-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512 Hi, I'm currently rebuilding the database for keyserver.rainydayz.org. I'm using the normalbuild rather than the fastbuild. I'm presuming that the process displays the time taken after it's processed each file, which means that the rebuild is going to take about three weeks. As it's a slow machine I reckoned the extra time spent building the database would be compensated for it running faster when it was finished. How much faster is the fastbuild process and how much slower does the server run when it's finished? Would it not be possible to get a file dump of the database and start with that? - -- Andy Ruddock - ------------ [hidden email] (GPG Key ID 0xEEC3AFB3) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBCgAGBQJL5o6+AAoJEAC80KTuw6+zl4YP/jBKTjD2TkGrdnDC7FpDmWGd 1biWFQ4OA3G71BG8YwQ1MybC5N4vXGjDtpDdIbQpMdK143RBL/+XVbpr74xY8xG+ Oha6IRclDb0lJXa0qz4JbC02QHZ9wTsDLj5RkbCbNdEQxm4Rd/H6VVS+vLHOpVMj 1GoSaUui1318DKxW0EXfBzflNorEMr4SgnKATIHp7N23hCbjGQ25HUb9cFBYcc9d sjJiGx65RYpIdzM1fOfGj4AOIAZkdF74WHbHR2vdo+BHokdJLtA4jkCbxxZHPdGN 8aLjsIWAtAs6ZfUW3qckoNmHc/hXZUcD6BxOButJVY0w/7jviXAPwH9F0x1sEvXK 1DRAbRtX8m0i8ovn/lUkab+MpSKi5+xc2B2gUbdOrIcEZllmtvYDmSFmz8RoJC63 U093mq1SDSnictypvbJkJGoxukac8usHdozyoCwets1XJG9/Pv65dibm+RAEA7nE abrwehO51cvlrwChMaJ8ls+X8f35S+87oHI7NSz77kfRIT3Gj/WRckCvOEl85uQi CEJpC0bBtJrh+/1tki9F3XUFP4/E9/+BEzEmkWNgURCa4hqbX4z8Q1vnij2mbY2e Lyea7kPS12o+hPGX7FjOKbu3wxdGmPd5zv0neG8WHmzOL6qLP0bNSvu0afmc60Ho 6yATm3B1V3TqlDtK0iUu =/TK2 -----END PGP SIGNATURE----- _______________________________________________ Sks-devel mailing list [hidden email] http://lists.nongnu.org/mailman/listinfo/sks-devel |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256 Andy Ruddock wrote: > Hi, > > I'm currently rebuilding the database for keyserver.rainydayz.org. > I'm using the normalbuild rather than the fastbuild. > I'm presuming that the process displays the time taken after it's > processed each file, which means that the rebuild is going to take about > three weeks. It's the time after processing each set of files, i.e. the value you have -n set to on the sks build command line. On a 1.5G Hz Sempron box with 1GB RAM, I can load just under 200,000 keys per pass without starting to swap. So, if I'm using a dump from pramberger.at where each file is 25k keys, I use -n 7. From linux.prato.it, each dump is 15k keys, and then I use -n 13. ls -ltu on your dump directory will tell you the last access time for each file. Add -r and you'll easily get the last file read in. build takes about 6 hours on this box and pbuild approx 2.5 hrs. > As it's a slow machine I reckoned the extra time spent building the > database would be compensated for it running faster when it was finished. > How much faster is the fastbuild process and how much slower does the > server run when it's finished? I don't know that this has ever been measured. I've never built with fastbuild > Would it not be possible to get a file dump of the database and start > with that? I guess that could be done, but it would be significantly larger than the existing key dumps. - -- John P. Clizbe Inet:John (a) gingerbear.net You can't spell fiasco without SCO. hkp://keyserver.gingerbear.net or mailto:[hidden email]?subject=HELP Q:"Just how do the residents of Haiku, Hawai'i hold conversations?" A:"An odd melody / island voices on the winds / surplus of vowels" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11-svn5331-2010-05-07 (Windows XP) Comment: When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl! Comment: Be part of the £€37 ECHELON -- Use Strong Encryption. Comment: It's YOUR right - for the time being. Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iF4EAREIAAYFAkvnGp8ACgkQ614Z89ZWmCV4dgD/W4wRWGaxV6CeXWXklYEcEMiq d7WdVfnmO7TqJwd6/9QA/2a9//vjnn5J0Dbw2izWGV9djWTdRDSuQe2c1XIVRVI4 =VgPN -----END PGP SIGNATURE----- _______________________________________________ Sks-devel mailing list [hidden email] http://lists.nongnu.org/mailman/listinfo/sks-devel |
In reply to this post by Andy Ruddock-2
Andy Ruddock writes:
> Hi, > > I'm currently rebuilding the database for keyserver.rainydayz.org. > I'm using the normalbuild rather than the fastbuild. > I'm presuming that the process displays the time taken after it's > processed each file, which means that the rebuild is going to take about > three weeks. > As it's a slow machine I reckoned the extra time spent building the > database would be compensated for it running faster when it was finished. > How much faster is the fastbuild process and how much slower does the > server run when it's finished? When I had a quick glance at this[1], the default settings for Berkeley DB's page size seamed way too low; you could try increasing it. But once the key database is fully populated it occupies more room than one built with fastbuild. I suspect that performances will not change much in the day to day operation of your SKS server, but please tell us what your experience is. > Would it not be possible to get a file dump of the database and start > with that? It should be but I am not sure what kind of gain you'd expect. [1] http://lists.gnu.org/archive/html/sks-devel/2009-08/msg00058.html -- Kim Minh _______________________________________________ Sks-devel mailing list [hidden email] http://lists.nongnu.org/mailman/listinfo/sks-devel |
On May 12, 2010, at 3:14 PM, Kim Minh Kaplan wrote: > > It should be but I am not sure what kind of gain you'd expect. > The gain is not in performance, but rather "referential integrity" and a simpler easier to maintain data store w/o a key dump. The KDB store is tied to one specific key dump. (aside) There's another conscious design choice related to "referential intergrity" splitting the PTRee from the KDB store. Both stores must be maintained in parallel (or the PTree store must be rebuilt when/as needed). But -fastbuild (what I use) performs quite well, and the PTree store can be rebuilt if/when needed, and a new keydump can be loaded whenever, gud enuf. 73 de Jeff _______________________________________________ Sks-devel mailing list [hidden email] http://lists.nongnu.org/mailman/listinfo/sks-devel |
Free forum by Nabble | Edit this page |