Min. Requirement for SKS Version in the Pool

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Min. Requirement for SKS Version in the Pool

Kristian Fiskerstrand-7
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi,

In order to have the full pool being able to handle searching subkeys
using the Long KeyID, which is available in SKS 1.1.3, I intend on
making some changes to the minimum required SKS version of the servers
to be included in the main Pool.

- From the SKS CHANGELOG for 1.1.3: "
  - Allow &search with long subkey ID (16 digit) and subkey fingerprint
    subkey lookup was failing with other than a short key ID. However,
    public key lookup was working with short and long key ID and
    fingerprints.
    This patch makes subkey lookup behave the same as full key lookup.

http://lists.gnupg.org/pipermail/gnupg-users/2012-January/043495.html "

Currently only the sub-pool: subset.pool.sks-keyservers.net is already
LongID safe due to its min version requirement of 1.1.3 [0]

On the way to this goal, grandfathering 1.1.2 seems to be necessary to
maintain a higher number in the pool. Currently I see, of total of 64
servers included in the Pool at the time of writing;
alpha status # php debug_version.php  | sort | uniq -c
      3 GnuKS:0.9.2
     31 SKS:1.1.1
      7 SKS:1.1.2
     19 SKS:1.1.3
      4 SKS:1.1.3+

Id est 26 servers that are upgraded to 1.1.3 or later (1.1.3+ is
Trunk, and GnuKS 0.9.2 is based on 1.1.3).

As of *7. July 2012* I intend to change the minimum version for
qualification in the pool to 1.1.2, which would currently mean that
the number of servers in the pool will be 33 (although I expect more
servers to be upgrade by this time, one of the reasons being that
1.1.3 is now available in the Debian repository).

As of *1. August 2012* I intend to change the minimum version for
qualification in the pool to 1.1.3.

If there are strong objections to these changes, please let me know
before the aforementioned dates.

[0] http://sks-keyservers.net/overview-of-pools.php

- --
- ----------------------------
Kristian Fiskerstrand
http://www.sumptuouscapital.com
Twitter: @krifisk
- ----------------------------
Corruptissima re publica plurimæ leges
The greater the degeneration of the republic, the more of its laws
- ----------------------------
This email was digitally signed using the OpenPGP
standard. If you want to read more about this
The book: Sending Emails - The Safe Way: An
introduction to OpenPGP security is now
available in both Amazon Kindle and Paperback
format at
http://www.amazon.com/dp/B006RSG1S4/
- ----------------------------
Public PGP key 0xE3EDFAE3 at http://www.sumptuouscapital.com/pgp/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBCAAGBQJP52hmAAoJEBbgz41rC5UIEe4P/07fURGZ5kl5dinpEAP0smgr
1jfQ+SMCLCWA/mGqhxVrNY7DWOXSSvvblT8gQg+KKqP5WjeGuqesIYYAbuCaHbDf
mXSLZ4ekjh9o+SUKMlqw7upApzJJkBce8wQafZCXsXSwvquNtsrFS0VuT9nMEafw
uWUv8Au3yuD0ivaTGq73gEA5OgKLvQ5xuy8ljGtwl1R/vm1fRli2bg/s0CVnPhYV
wtxML/N3QjA4rZp6BPY0LIrGquLXuaEvuWl3AZL8kCEckGA0+kdJRnnUpISM2unF
OpmLEnvh/ehpjCXmspihky2V2Rf+OAb1+3jk57Qf4mY1assfKeVysMrQutACG/nl
TylI0bnwAbzjcILyYxnen+Jdnfmclzyh5sfINOJ2fauLzNNbSOFl98ZTSOX/WpWD
Ej8CGRxLPGMP+NAhWVmx4gy18z4FAMnbv9okJ+l/pPiUklIRKoH8dpy/yc4cotla
t4MFNtV7AjVPSQJ/9BxmxEKFebsNWUN+YiNgFiUsvuDfeluQNWEXdhkEPmAdAlWs
8OtPcwact1vaoOntIjSrVV1kkSK0DTQUBU8n+RxfMxTK018JhHhb8Nn1hdQG5rPV
+RWnOAoVUj/QqPYjGAxjyjUoTiTRnKyaHMQ+6R2mtTdAr+ylKN6IU4hM9ez4lH2Z
dvyQE36jWw4gzJ817WvI
=XLS3
-----END PGP SIGNATURE-----


_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

David Benfell
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/24/12 12:20, Kristian Fiskerstrand wrote:
>
> On the way to this goal, grandfathering 1.1.2 seems to be necessary
> to maintain a higher number in the pool.

<snip>

>
> As of *7. July 2012* I intend to change the minimum version for
> qualification in the pool to 1.1.2, which would currently mean
> that the number of servers in the pool will be 33 (although I
> expect more servers to be upgrade by this time, one of the reasons
> being that 1.1.3 is now available in the Debian repository).
>
> As of *1. August 2012* I intend to change the minimum version for
> qualification in the pool to 1.1.3.
>
I did take a quick look at the Arch User Repository package for SKS to
see why I'm not on 1.1.3 already. I tried pointing it at the 1.1.3 tar
ball and supplied a correct MD5SUM.

The build fails here:

cd cryptokit-1.0 && make all
make[1]: Entering directory
`/home/benfell/aur/sks/src/sks-1.1.3/cryptokit-1.0'
ocamlc -g -c -ccopt "-O -I/usr/include" rijndael-alg-fst.c
/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.0/cc1: error while loading
shared libraries: libisl.so.9: cannot open shared object file: No such
file or directory
make[1]: *** [rijndael-alg-fst.o] Error 2
make[1]: Leaving directory
`/home/benfell/aur/sks/src/sks-1.1.3/cryptokit-1.0'
make: *** [cryptokit-1.0/cryptokit.cma] Error 2
cd cryptokit-1.0 && make all
make[1]: Entering directory
`/home/benfell/aur/sks/src/sks-1.1.3/cryptokit-1.0'
ocamlc -g -c -ccopt "-O -I/usr/include" rijndael-alg-fst.c
/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.0/cc1: error while loading
shared libraries: libisl.so.9: cannot open shared object file: No such
file or directory
make[1]: *** [rijndael-alg-fst.o] Error 2
make[1]: Leaving directory
`/home/benfell/aur/sks/src/sks-1.1.3/cryptokit-1.0'
make: *** [cryptokit-1.0/cryptokit.cma] Error 2

I have isl 0.10-1 installed. *Sigh*

- --
David Benfell
[hidden email]


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJP56KpAAoJELT202JKF+xpM9IP/AkBNGKDfY+Lmyj4n5xJXk9K
QSPZh3WOOJWnUd+DOoNw7zv9QEUFu91Xzn9CpGUF75j4UH7vQpbM6jdnb/kI49UM
AjAV6Y6ya3fDzwhXsevOATonF7uAnJXyc9NM2UiE0NuiZzv1m88P1oZSfO+oflKd
lb5bQIFSOeHLgxE3Sjcry0IoKCzzohW/2L4fTUeRLEdg90Tkqr3oesZYjLDmFBtz
KanqKvTTWFxj2S2GBHxm2dTnQKZUiM+Maupi4qiBHWC0EZSCoTX9sP0iLJBypDlK
WMyeSybUPo879MkqiFiZSMqeCghq2Qhe9UFyIRJ4A5QOHHrTsvrijbaFmoD6u9ak
/TOgH+dDt34EOlQgmL6VN3hyxl7atfev6+9R6/TRcqaZGFjLjHRi4w/zAUGjnHgX
9mUg/KCSPcfjYfSYaEyoeAvJrbNVFBiypD6NAQ6UoCEoBF8f7OwP1bF03c3AUsD8
wUyZkC1rzPWL8aAPGf6inq0Ioie/ZI77+/7qyJeQKJh72UMyVYgs7uhPRLUBGTZW
2IXi91Z0bxI55m09GpaGX2Jj4e9yxDb0VH8oCnrr42fUS1xiXJezYunc+vcnaUBj
+rPokS+jv7HexhoEU1t40LpQpZ5Xe9/GyeGFhAVRQTWFDskq4V9b/q5beXY+PjBn
8Kqy82Px3OgU00Ul7+tv
=VA/t
-----END PGP SIGNATURE-----

_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

Kristian Fiskerstrand-7
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 2012-06-25 01:28, David Benfell wrote:
> On 06/24/12 12:20, Kristian Fiskerstrand wrote:
>
>> On the way to this goal, grandfathering 1.1.2 seems to be
>> necessary to maintain a higher number in the pool.
>
> <snip>
>

...

>
> I did take a quick look at the Arch User Repository package for SKS
> to see why I'm not on 1.1.3 already. I tried pointing it at the
> 1.1.3 tar ball and supplied a correct MD5SUM.
>
> The build fails here:
>

...


>
> I have isl 0.10-1 installed. *Sigh*
>

This seems to be an Arch bug, see [0]

[0] https://bugs.archlinux.org/task/30367


- --
- ----------------------------
Kristian Fiskerstrand
http://www.sumptuouscapital.com
Twitter: @krifisk
- ----------------------------
Corruptissima re publica plurimæ leges
The greater the degeneration of the republic, the more of its laws
- ----------------------------
This email was digitally signed using the OpenPGP
standard. If you want to read more about this
The book: Sending Emails - The Safe Way: An
introduction to OpenPGP security is now
available in both Amazon Kindle and Paperback
format at
http://www.amazon.com/dp/B006RSG1S4/
- ----------------------------
Public PGP key 0xE3EDFAE3 at http://www.sumptuouscapital.com/pgp/


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBCAAGBQJP57JAAAoJEBbgz41rC5UIWAYP/19zl06HqXW7PYDIg61+kIVA
dQ9Uydsj4/+cE8lBKlt+6yO2ohFoEYmYC/CF/vnf4orLPrwADFPP3csR0YlAd42v
PMSlZVGdTid2/pW/7K55KvNx2xG69XPDlij57eWd8QKB/OimPNEoctmHsPTN6ru8
X8V4ALkncxxOsm0C8qk6s5nVpkAGrVgEoEzQx2D8vZvI8V9WUvlRUmZuJIUTF5RD
cvfF4cwVz4bFQ1owVPjACsJA200pIvY7edjJpBXs12AMaoSUX7+BCWfGpiPD20yZ
qhJfxF3bY2SZCYR2Xdn76J44uzBq3tdhZH0S8LvTlQIqxXkKj3hkswt+aU3XDgWK
I/uA4FBN96Ofk7v8lqOwu0kZKcYJ54Rc+3vFsNHZNdY/iF7k7ayOD2Mu6QemhKOt
dYn4t8bUQchB7v0Yv3WXlnq1BGh8o3naXeFxWESgQ4vhk2smYD1tf9gZhs51B/Wq
QhnCjaWKrAgk1x9BX0gLIQU7ISLfpJL1oPPW3mWGReEKe5CchJ02cN55cfsU5COR
zO2XjUKw7jihUKwigGc3Ah4YZugA2T99+C9ILg4Q0Dke1tI6PWTunT4thDV4gNFl
KEQEkk0xftLrKOv61HdIkpBlySdiSY64M83DBu++0INPN0nM6qaD1+WeactXM5xo
mpQ0KuYarZlGtUnnRChV
=OqxW
-----END PGP SIGNATURE-----


_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

Brian D Heaton
In reply to this post by Kristian Fiskerstrand-7
Kristian,

Would it be a good idea to contact the operators of keyservers running
versions prior to v1.1.3 and let them know about this coming change and
perhaps get them to upgrade?  I'm guessing a great number of the
keyserver operators are not on the sks-devel list.

I'm willing to work through the servers in the NA pool in order by
weight to help take some of the load.

-Brian


On 6/24/2012 12:20 PM, Kristian Fiskerstrand wrote:

> Hi,
>
> In order to have the full pool being able to handle searching subkeys
> using the Long KeyID, which is available in SKS 1.1.3, I intend on
> making some changes to the minimum required SKS version of the servers
> to be included in the main Pool.
>
> From the SKS CHANGELOG for 1.1.3: "
>   - Allow &search with long subkey ID (16 digit) and subkey fingerprint
>     subkey lookup was failing with other than a short key ID. However,
>     public key lookup was working with short and long key ID and
>     fingerprints.
>     This patch makes subkey lookup behave the same as full key lookup.
>
> http://lists.gnupg.org/pipermail/gnupg-users/2012-January/043495.html "
>
> Currently only the sub-pool: subset.pool.sks-keyservers.net is already
> LongID safe due to its min version requirement of 1.1.3 [0]
>
> On the way to this goal, grandfathering 1.1.2 seems to be necessary to
> maintain a higher number in the pool. Currently I see, of total of 64
> servers included in the Pool at the time of writing;
> alpha status # php debug_version.php  | sort | uniq -c
>       3 GnuKS:0.9.2
>      31 SKS:1.1.1
>       7 SKS:1.1.2
>      19 SKS:1.1.3
>       4 SKS:1.1.3+
>
> Id est 26 servers that are upgraded to 1.1.3 or later (1.1.3+ is
> Trunk, and GnuKS 0.9.2 is based on 1.1.3).
>
> As of *7. July 2012* I intend to change the minimum version for
> qualification in the pool to 1.1.2, which would currently mean that
> the number of servers in the pool will be 33 (although I expect more
> servers to be upgrade by this time, one of the reasons being that
> 1.1.3 is now available in the Debian repository).
>
> As of *1. August 2012* I intend to change the minimum version for
> qualification in the pool to 1.1.3.
>
> If there are strong objections to these changes, please let me know
> before the aforementioned dates.
>
> [0] http://sks-keyservers.net/overview-of-pools.php
>
>
>
> _______________________________________________
> Sks-devel mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/sks-devel
>


_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

David Benfell
In reply to this post by Kristian Fiskerstrand-7
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/24/12 17:35, Kristian Fiskerstrand wrote:

>
> This seems to be an Arch bug, see [0]
>
> [0] https://bugs.archlinux.org/task/30367
>
Oops. I also notice I have cryptokit 1.5 installed. The last mention I
recall of that was right before our, ahem, discussion of Debian's
packaging of 1.1.3 (I see a sid package now exists).

- --
David Benfell
[hidden email]


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJP58O7AAoJELT202JKF+xp4jQP/RKuymh1Pl0qRFV4hqRC52kS
+1OzWJPLj9GwQLNY1rtemzjzgKt+2tibJ/bs6XycCIbbqS8TngusKlLQI5tQCVF3
60rdmWYFRfFIocPoH4aUJHh2K+axRdgjorY/TTV+FkpXz9b+soOEb/U6HIIxo7Xy
Q05kCix9oJ/kmAjdtRqEIUVEY3cln2osA/NlbRWHJnR1J8HQYW+SpeAKb+2SUg92
NuRWS4V9IXSEBwX0MPt7SjB5Z+YFVt5xB0OOllNetz+PUjIuDyDqC/F/QraEAwnb
jnHEwAeGfWwpkh4A4MhekuZx4SYho3lx1MmsI78X29z5gm8ZvsoBASWCRVjIq30d
LIgKVfRXi2C/r3obWXaewXnC8VrVdrg+PeCJt7T5J/rArx097fJkQif5FRStMo20
jpoHI46EZRIhhCyktAE1kyYm7OA2pUxPt6KuTyyKttLikh8bzFBmk8hbnAR94s+o
yEgtu1UJifctLf9/wPrw9xJQJ72VGyEWArX1tQFztGyFamxwXySninTHMd5iAjAz
l1I9CEYogzjixUkzeBMycMmamb0tADVo8a10lMJXoPHkcL+7ncoIjbXgE2V+e5+8
jS7jk1sE2aGfcudymULBYd96nVSWKYt9KMjSs81+zhGWlReCcyzj9ia4aa1MEl+U
+cMpmM1UYFOu2ociQH46
=dq6u
-----END PGP SIGNATURE-----

_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

Daniel Kahn Gillmor-7
In reply to this post by Kristian Fiskerstrand-7
On 06/24/2012 03:20 PM, Kristian Fiskerstrand wrote:
> As of *7. July 2012* I intend to change the minimum version for
> qualification in the pool to 1.1.2, which would currently mean that
> the number of servers in the pool will be 33 (although I expect more
> servers to be upgrade by this time, one of the reasons being that
> 1.1.3 is now available in the Debian repository).

1.1.3 is in debian unstable (sid).

Being in the unstable repository isn't the same as being in the stable
repo (squeeze), as most reasonable server administrators run stable, not
unstable on their systems.

Rather than expecting keyserver operators to run debian unstable, a
reasonable approach is to get it into squeeze-backports. (see
http://backports.debian,org/ for details about how debian backports work)

I'm pursuing this right now, and hope to have a backported package
tested and available in squeeze-backports in about a week.  When it
happens, i'll write a note to this list explaining how operators of
servers running debian stable can do a relatively easy upgrade without
switching distributions.

Thanks as always for maintaining the pool, Kristian.  I hope we can pull
this transition off cleanly without reducing the size of the pool too much.

        --dkg


_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel

signature.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

Kristian Fiskerstrand-6


Sent from my iPad

On Jun 25, 2012, at 5:38 AM, Daniel Kahn Gillmor <[hidden email]> wrote:

> On 06/24/2012 03:20 PM, Kristian Fiskerstrand wrote:
>> As of *7. July 2012* I intend to change the minimum version for
>> qualification in the pool to 1.1.2, which would currently mean that
>> the number of servers in the pool will be 33 (although I expect more
>> servers to be upgrade by this time, one of the reasons being that
>> 1.1.3 is now available in the Debian repository).
>
> 1.1.3 is in debian unstable (sid).
>
> Being in the unstable repository isn't the same as being in the stable
> repo (squeeze), as most reasonable server administrators run stable, not
> unstable on their systems.
>
> Rather than expecting keyserver operators to run debian unstable, a
> reasonable approach is to get it into squeeze-backports. (see
> http://backports.debian,org/ for details about how debian backports work)
>
> I'm pursuing this right now, and hope to have a backported package
> tested and available in squeeze-backports in about a week.  When it
> happens, i'll write a note to this list explaining how operators of
> servers running debian stable can do a relatively easy upgrade without
> switching distributions.

Thank you for the additional information on the debian process and for your effort ( http://lists.debian.org/debian-backports/2012/06/msg00067.html )

>
> Thanks as always for maintaining the pool, Kristian.  I hope we can pull
> this transition off cleanly without reducing the size of the pool too much.

Please let me know if we should push the timeline some for the 1.1.2 minimum to get more time for testing, as originally stated my primary goal is getting to 1.1.3, so this shouldn't necessarily affect too much, we can still keep that at August 1.

>

-KF
_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

Daniel Kahn Gillmor-7
On 06/25/2012 12:44 AM, Kristian Fiskerstrand wrote:
> Please let me know if we should push the timeline some for the 1.1.2 minimum to get more time for testing, as originally stated my primary goal is getting to 1.1.3, so this shouldn't necessarily affect too much, we can still keep that at August 1.

Another week or two of breathing room for the 1.1.2 limit might be nice,
thanks :)

i believe zimmermann.mayfirst.org is now running 1.1.3, from the
backported debian package.

I ran into two problems during the upgrade:

 (0) the post-installation script wants to back up the entire database
before upgrading it from 4.7 to 4.8.  This takes a long time and doubles
the amount of space consumed.  I think this is a debian-specific issue
(it derives from the packaging's upgrade scripts), so i've filed it at
http://bugs.debian.org/678924.

 (1) after the upgrade, some searches caused sks db to hang (hard!) with
the following message written to db.log:

 Error fetching uid during VIndex for keyid 0x29BE5D2268FD549F:
Bdb.DBError("unable to allocate memory for mutex; resize mutex region")

I've reported this to debian as well at http://bugs.debian.org/678927,
but i believe this is more of an upstream issue.  To resolve the
problem, i terminated both sks processes ("sks db" had to be killed with
SIGKILL once this mutex was hung), copied DB_CONFIG into the PTree and
DB directories, and then restarted the daemons.

This kind of hard hang makes the rest of the service completely
unresponsive.  is there some way to improve that behavior upon failure?
 I'd rather that the request that exhausts some resource simply fails
with some 500 error code, and allows other (less resource-intensive)
requests to succeed subsequently.

I'll post instructions for migration to 1.1.3 via squeeze-backports once
i'm confident in this setup, and the packaging has migrated into that
repository.

        --dkg


_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel

signature.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

Christoph Egger-9
Hi!

Daniel Kahn Gillmor <[hidden email]> writes:
> On 06/25/2012 12:44 AM, Kristian Fiskerstrand wrote:
>> Please let me know if we should push the timeline some for the 1.1.2 minimum to get more time for testing, as originally stated my primary goal is getting to 1.1.3, so this shouldn't necessarily affect too much, we can still keep that at August 1.
>  Error fetching uid during VIndex for keyid 0x29BE5D2268FD549F:
> Bdb.DBError("unable to allocate memory for mutex; resize mutex region")

the 65536 mutex count wasn't enough to stand a gpg --refresh-keys on a
~1k keys pubring here with a (probably) similarly backported package as
I still ran into this error so please test also "heavy" load ;-)

Regards

    Christoph

--
9FED 5C6C E206 B70A 5857  70CA 9655 22B9 D49A E731
Debian Developer | Lisp Hacker | CaCert Assurer

_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

unable to allocate memory for mutex; resize mutex region [was: Re: Min. Requirement for SKS Version in the Pool]

Daniel Kahn Gillmor-7
Hi Christoph--

On 06/25/2012 01:50 AM, Christoph Egger wrote:
> Daniel Kahn Gillmor <[hidden email]> writes:
>> On 06/25/2012 12:44 AM, Kristian Fiskerstrand wrote:
>>> Please let me know if we should push the timeline some for the 1.1.2 minimum to get more time for testing, as originally stated my primary goal is getting to 1.1.3, so this shouldn't necessarily affect too much, we can still keep that at August 1.
>>  Error fetching uid during VIndex for keyid 0x29BE5D2268FD549F:
>> Bdb.DBError("unable to allocate memory for mutex; resize mutex region")
>
> the 65536 mutex count wasn't enough to stand a gpg --refresh-keys on a
> ~1k keys pubring here with a (probably) similarly backported package as
> I still ran into this error so please test also "heavy" load ;-)

Testing with a large keyring refresh now, thanks for the suggestion.

Is your test package running behind a reverse proxy, as recommended at:

http://lists.nongnu.org/archive/html/sks-devel/2012-03/msg00006.html

?

        --dkg

_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: unable to allocate memory for mutex; resize mutex region [was: Re: Min. Requirement for SKS Version in the Pool]

Christoph Egger-9
Hi!

Daniel Kahn Gillmor <[hidden email]> writes:

> On 06/25/2012 01:50 AM, Christoph Egger wrote:
>> Daniel Kahn Gillmor <[hidden email]> writes:
>>> On 06/25/2012 12:44 AM, Kristian Fiskerstrand wrote:
>>>> Please let me know if we should push the timeline some for the 1.1.2 minimum to get more time for testing, as originally stated my primary goal is getting to 1.1.3, so this shouldn't necessarily affect too much, we can still keep that at August 1.
>>>  Error fetching uid during VIndex for keyid 0x29BE5D2268FD549F:
>>> Bdb.DBError("unable to allocate memory for mutex; resize mutex region")
>>
>> the 65536 mutex count wasn't enough to stand a gpg --refresh-keys on a
>> ~1k keys pubring here with a (probably) similarly backported package as
>> I still ran into this error so please test also "heavy" load ;-)
>
> Testing with a large keyring refresh now, thanks for the suggestion.
>
> Is your test package running behind a reverse proxy, as recommended at:
>
> http://lists.nongnu.org/archive/html/sks-devel/2012-03/msg00006.html

Jep it's behind a nginx instance (keyserver.siccegge.de)

Regards

    CHristoph

--
9FED 5C6C E206 B70A 5857  70CA 9655 22B9 D49A E731
Debian Developer | Lisp Hacker | CaCert Assurer

_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

John Clizbe-3
In reply to this post by Christoph Egger-9
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1,SHA256

Christoph Egger wrote:

> Hi!
>
> Daniel Kahn Gillmor <[hidden email]> writes:
>> On 06/25/2012 12:44 AM, Kristian Fiskerstrand wrote:
>>> Please let me know if we should push the timeline some for the 1.1.2 minimum to get more time for testing, as originally stated my primary goal is getting to 1.1.3, so this shouldn't necessarily affect too much, we can still keep that at August 1.
>>  Error fetching uid during VIndex for keyid 0x29BE5D2268FD549F:
>> Bdb.DBError("unable to allocate memory for mutex; resize mutex region")
>
> the 65536 mutex count wasn't enough to stand a gpg --refresh-keys on a
> ~1k keys pubring here with a (probably) similarly backported package as
> I still ran into this error so please test also "heavy" load ;-)

After Christoph's last email re mutex_set_max I checked my own databases, both
of which were set to 64K.  PTree was almost equally split between in-use and
free.  KDB was very close to running out with only about 3K left of the 64K.

With the larger number of keys now in the DB, perhaps a better value for
KDB/DB_CONFIG is

    mutex_set_max 98304

- -John
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12-git-509fe4ce-2012-01-31 (Windows XP)
Comment: When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl!
Comment: Be part of the £€€7 ECHELON -- Use Strong Encryption.
Comment: It's YOUR right - for the time being.
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJP6AJBAAoJECMTMVxDW9A0BeIIAJa5Xpj+cZZPmPXv3YvCGQ6D
LEKW8iowBskF5wFB7/QSRZnFdz98mpliZGIRNlHZmUceks9TVAWokEHppNGUnEc0
h8+K7mQUQga/aWoLpOn1AByfVY4y1Tkfy7clU7v3zzwVojIY3ITWKVx5ZC+8BGgz
eWNLV1szx1o/Hbg6Wn1nroKw0iM+nQ7u8FgBMxhEQ1Ucx0JK5bfiuW3PUikHdWC7
MHD0DUa6zSQFs+QZsqKnhwbCukJkeoqCrvfq3RB7daUiIs1a/XI7W3xdQZK320ZM
llqEQp53PfYSL26n7QaaWnDeE3DcPT/VlAx9925FZ9pfOOaU4Fs6LTdv8FNOr0+I
XgQBEQgABgUCT+gCQQAKCRDrXhnz1laYJYfWAP0Qkpu6Dy49ooKS0cJLDtKLbpzr
i5IBJlBswFZR2qUkJAD+IzlSlhCceYOyHeER3ZIF9Pnmju4/zoZk/yXtuWc8UOc=
=ULOs
-----END PGP SIGNATURE-----

_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

Daniel Kahn Gillmor-7
On 06/25/2012 02:16 AM, John Clizbe wrote:
> After Christoph's last email re mutex_set_max I checked my own databases, both
> of which were set to 64K.  PTree was almost equally split between in-use and
> free.  KDB was very close to running out with only about 3K left of the 64K.

Would you mind showing how you did this check?  What architecture was
the machine you checked?  I'd be happy to corroborate.

> With the larger number of keys now in the DB, perhaps a better value for
> KDB/DB_CONFIG is
>
>     mutex_set_max 98304

Can this change be made live in the DB_CONFIG file, while sks is
operating?  or should we shut down sks, make the change, and then restart?

Is there some way that this can be auto-tuned?  What are the
consequences of setting the value even higher?

Thanks for helping me get my head around bdb configuration!

        --dkg


_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel

signature.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

John Clizbe-6
Daniel Kahn Gillmor wrote:

> On 06/25/2012 02:16 AM, John Clizbe wrote:
>> After Christoph's last email re mutex_set_max I checked my own databases, both
>> of which were set to 64K.  PTree was almost equally split between in-use and
>> free.  KDB was very close to running out with only about 3K left of the 64K.
>
> Would you mind showing how you did this check?  What architecture was
> the machine you checked?  I'd be happy to corroborate.
>
>> With the larger number of keys now in the DB, perhaps a better value for
>> KDB/DB_CONFIG is
>>
>>     mutex_set_max 98304
>
> Can this change be made live in the DB_CONFIG file, while sks is
> operating?  or should we shut down sks, make the change, and then restart?

DB_CONFIG is read when the database is opened. You can make the edit, then do
a shutdown-start. I do a shutdown, db_recover -h KDB, sks start

> Is there some way that this can be auto-tuned?  What are the
> consequences of setting the value even higher?

The mutex region grows from ~7MB to ~12MB. BDB sets an initial mutex count of
57435. It when then grow the mutex pool as it needs more. mutex_set_max is
just an upper bound on that growth. You can see the mutex usage with

    db_stat -xh DB

"Normally," BDB's defaults are adequate. Tuning is more of an
application-specific "black art".

> Thanks for helping me get my head around bdb configuration!

Thank Jeff Johnson for helping me.

-John

_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

Daniel Kahn Gillmor-7
On 06/25/2012 02:39 AM, John Clizbe wrote:
> The mutex region grows from ~7MB to ~12MB. BDB sets an initial mutex count of
> 57435. It when then grow the mutex pool as it needs more. mutex_set_max is
> just an upper bound on that growth.

So the main concern is just allocation of a few more MiB of RAM?  or are
there other consequences to raising it even higher?

> You can see the mutex usage with
>
>     db_stat -xh DB

OK, i'm seeing slightly lower percentages of mutexes in use than you
were reporting, but i've gone ahead and adopted the larger mutex_set_max
you recommended anyway.

>> Thanks for helping me get my head around bdb configuration!
>
> Thank Jeff Johnson for helping me.

Thanks, Jeff!

        --dkg


_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel

signature.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

Kristian Fiskerstrand-7
In reply to this post by Daniel Kahn Gillmor-7
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 2012-06-25 07:41, Daniel Kahn Gillmor wrote:
> On 06/25/2012 12:44 AM, Kristian Fiskerstrand wrote:
>> Please let me know if we should push the timeline some for the
>> 1.1.2 minimum to get more time for testing, as originally stated
>> my primary goal is getting to 1.1.3, so this shouldn't
>> necessarily affect too much, we can still keep that at August 1.
>>
>
> Another week or two of breathing room for the 1.1.2 limit might be
> nice, thanks :)

Lets keep the 18th of July as a tentative date for 1.1.2 migration
then, which gives another two weeks until the 1.1.2 servers fall out
and we're at 1.1.3 to give some buffer in between.

- --
- ----------------------------
Kristian Fiskerstrand
http://www.sumptuouscapital.com
Twitter: @krifisk
- ----------------------------
Corruptissima re publica plurimæ leges
The greater the degeneration of the republic, the more of its laws
- ----------------------------
This email was digitally signed using the OpenPGP
standard. If you want to read more about this
The book: Sending Emails - The Safe Way: An
introduction to OpenPGP security is now
available in both Amazon Kindle and Paperback
format at
http://www.amazon.com/dp/B006RSG1S4/
- ----------------------------
Public PGP key 0xE3EDFAE3 at http://www.sumptuouscapital.com/pgp/


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBCAAGBQJP6C4lAAoJEBbgz41rC5UInVQQALFJWDaau0s3Wb/1IdjAmcSL
wLD2/+uzvgtWK3tz3GsEpCwhT91w5vpTwc/fN8Du3mMOOnvV/WrmSZdwbBXwGmmO
Yn6My1/P+FfuvurhbH/mdmcj9KGhkS/l9Fu8hEvg35LmxwJvdhCOxuHP0NewvFwg
QfjJuWATi/CbxQWO5A4S1Lrwp3QsjPhRa/w5/tuwSqILX5dWBq35vp5Pkuit3atF
VvU+wCneMf4NwzDPLlHhQp4FhfnprGB2aR/smR85R34NLYvi1ZwNVLuAnJuHXw5U
wtd9pMHmhUZw+Bi+ekuZGYG+tM5ZwdeJA60yjlWK9vEBm1LTbJIbikIf0HMxkQXr
LQuSpbOe9ILG/nbvmNOCZTPYCSU1Kc14pIW0dxxWdxfDoyKqs2HSuMIvE8O2Pkba
u598NQZUWiGPIDw46h2bsqidKuRUM5Ip2byaGTQ1AB951Lbu78d/yGGyIc+0mfH0
iILnYhBnK+Hy9DwUbeZ6+hal8t5p3GpiBc7oyTn30/48MBjKcxwfoEASoyBOLftt
T4e/+LYNI0YXQxb5yW8IgvC1JKyZunHDZSusAx+jOa6Mc7MMPpK+4iz/WN9vwwv9
1kMEO5hyKIdzZBnANX6H3T+QTWmhAfVTLCVKEl7mlax+BzK71g771EF6fjJpUf5w
P2PLPYv2hvjn2RmV5lFy
=f6j2
-----END PGP SIGNATURE-----


_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

Jeffrey Johnson-5
In reply to this post by Daniel Kahn Gillmor-7

On Jun 25, 2012, at 2:59 AM, Daniel Kahn Gillmor <[hidden email]> wrote:

> On 06/25/2012 02:39 AM, John Clizbe wrote:
>> The mutex region grows from ~7MB to ~12MB. BDB sets an initial mutex count of
>> 57435. It when then grow the mutex pool as it needs more. mutex_set_max is
>> just an upper bound on that growth.
>
> So the main concern is just allocation of a few more MiB of RAM?  or are
> there other consequences to raising it even higher?
>
>> You can see the mutex usage with
>>
>>    db_stat -xh DB
>
> OK, i'm seeing slightly lower percentages of mutexes in use than you
> were reporting, but i've gone ahead and adopted the larger mutex_set_max
> you recommended anyway.
>
>>> Thanks for helping me get my head around bdb configuration!
>>
>> Thank Jeff Johnson for helping me.
>
> Thanks, Jeff!
>

;-)

np. The confusion is that there are 2 different values
that need to be set in DB_CONFIG and the resource cap
that one runs into is
        mutex_set_max
which should be ~10x (my rule of thumb) the other values.

Its kinda frustrating changing all the values to no effect
when one of the values (mutex_set_max) needs to be larger
than the others.

73 de Jeff
> --dkg
>
> _______________________________________________
> Sks-devel mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/sks-devel


_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

Phil Benchoff-2
In reply to this post by Kristian Fiskerstrand-7
On Sun, Jun 24, 2012 at 09:20:07PM +0200, Kristian Fiskerstrand wrote:
> As of *1. August 2012* I intend to change the minimum version for
> qualification in the pool to 1.1.3.

Our keyserver runs on Centos 5.7.  The distribution BDB is 4.3.29 and we
have ocaml 3.12.0.  The server is a VM.

* Is there a suggested version of BDB other than >= 4.6?  Any reason to
  pick 4 versus 5?
* Is there any need to build ocaml 3.12.1?
* It looks like there is a patch for the VM clock issue.  Is it in 1.1.3?

Phil


_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

Andy Ruddock-2
In reply to this post by Brian D Heaton
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

I'm one of those keyserver operators who's on the sks-devel list.

My keyserver is running on Debian stable, I'm loathe to start
installing/upgrading packages from outside of the Debian repositories.

One of the main reasons for running Linux (practically any distro) is
that all software can be installed & updated from the repositories
without having to check umpteen packages for updates every day,
downloading and updating/installing from umpteen sources.

A package being in Debian unstable really doesn't count - I don't even
run Sid on my desktop, let alone on a server.

If the version number requirements for being in the keyserver pool is
greater than that provided in the repositories then the package may just
as well not be in the repositories at all.

Will every distro (of those that have sks in their repositories) be
carrying versions greater than 1.1.1?

Brian D Heaton wrote:

> Kristian,
>
> Would it be a good idea to contact the operators of keyservers running
> versions prior to v1.1.3 and let them know about this coming change and
> perhaps get them to upgrade?  I'm guessing a great number of the
> keyserver operators are not on the sks-devel list.
>
>> As of *7. July 2012* I intend to change the minimum version for
>> qualification in the pool to 1.1.2, which would currently mean that
>> the number of servers in the pool will be 33 (although I expect more
>> servers to be upgrade by this time, one of the reasons being that
>> 1.1.3 is now available in the Debian repository).
>>
>> As of *1. August 2012* I intend to change the minimum version for
>> qualification in the pool to 1.1.3.
>>
>> If there are strong objections to these changes, please let me know
>> before the aforementioned dates.

- --
Andy Ruddock
- ------------
[hidden email] (GPG Key ID 0xB0324245)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBCgAGBQJP6HgAAAoJECqtbbewMkJFGFUP/RHIBLaEkcW8jdxY+3K+7Ybd
CbYSfD6GoW+fn28EGlxHQza1OXO0jEkIhbxlCg+sRKipTWHvON5v2/s+DGz8ygsE
TfTkXELT+BvG/ndQQq6LEO3exDDQxUt7C1mH7r380ODPDdla2z/yxPsLgKbVVeOg
j/8i5x4a04yc1z+xIpDjMyaxmLq2HFl+DBogYaGTTUczdQ3zS1PawLAAvmMI0RH4
+y+a9mdyaHUEd1Ib4Y0UVoDOVPvnVXYFK8DfIkik7lKfsm/3ANRerUXN7IsMHM5t
X5C0RBKv0hMoxA5/5KB5r5HQM8DmbTS6QAQsSZJGoAiiyIUTh8U7BPrKxLznUBH/
xVQ/HpJvRez43CCqhythhceKoQm+VRQ1yTGZ4QoPInxiqgMMd3B2mpIA/9yaVl0x
zLC27yx0Z613DbxtFgJSkzHjzfNPoSeMJQ+j4N7wl3JZggn0t+hjagyjd1fjHUpn
aAWScrKo5vavIzPPRPqxuO221ck61irl5pDq3p0BNAwrrQzv5KUvX8usxzQsXCHU
q4huDcc96XTB25Wf1BJKba8/cF6zF1a/zqV+ZJ7qix44r7wHegdi0qj856BcEt5S
nVB1Vt6mNvv8OTDpMlZwRVouDB8SSAykDEooNOHEnjeC92YtH92go4WCbeEHeFo/
s/149kFycBMiohlWhYAY
=nfeh
-----END PGP SIGNATURE-----

_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Min. Requirement for SKS Version in the Pool

Jeffrey Johnson-5
In reply to this post by Phil Benchoff-2

On Jun 25, 2012, at 10:24 AM, Phil Benchoff wrote:

> On Sun, Jun 24, 2012 at 09:20:07PM +0200, Kristian Fiskerstrand wrote:
>> As of *1. August 2012* I intend to change the minimum version for
>> qualification in the pool to 1.1.3.
>
> Our keyserver runs on Centos 5.7.  The distribution BDB is 4.3.29 and we
> have ocaml 3.12.0.  The server is a VM.
>
> * Is there a suggested version of BDB other than >= 4.6?  Any reason to
>  pick 4 versus 5?

Centos 5.7 ships many versions of Berkeley DB for "compatibility",
not just db-4.3.29 uses in "system" db for clueless/busted versions
of software that continues to build with
        #include <db.h>

Instead, add db-5.3.x that installs into /usr/include/db53 and
add -I/usr/include/db53 to build flags.

Shared llbraries will not collide, and linkage flags should be  -ldb-5.3,
and pristine "vendor" software will not be disturbed whatsoever.

> * Is there any need to build ocaml 3.12.1?

This I can't answer for sure, but I suspect you will need 3.12.1.

> * It looks like there is a patch for the VM clock issue.  Is it in 1.1.3?

VM time skew was a big deal around RHEL5 iirc, and it may be fixed
in the kernel. I personally haven'y seen time skew issues running in
CentOS VM's in 2y without any patching.

But Phil Pennock's patch was posted in the last 4 weeks, quite easy,
in archives.

hth

73 de Jeff


_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
12