Has anyone looked into bdb alternatives?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Has anyone looked into bdb alternatives?

Alex Roper-2
I've been having terrible corruption issues (need to run reconstruction every day), as I
usually notice with bdb-based applications. I was wondering if anyone has experimented
with patching sks to use something more reliable like mysql, postgres, sqlite, etc?

mysql and postgres are exceptionally nice because then I wouldn't need to back up the
database manually, it would just be rolled into our normal backups, but even sqlite would
be an improvement.

Al "as a cs student, the thought of a single threaded single process spinlocking the db
engine is just sickening"

P.S. If you emailed me I think I added you to our membership file, let me know if you're
not showing up and you wanted to: http://dionysus.ugcs.caltech.edu:11371/pks/lookup?op=stats


_______________________________________________
Sks-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/sks-devel

signature.asc (300 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Has anyone looked into bdb alternatives?

Joseph Oreste Bruni-3
On Feb 14, 2009, at 3:59 PM, Alex Roper wrote:

> on issues (need to run reconstruction every day), as I usually  
> notice with bdb-based applications.

In defense of BDB, I have written a multi-process application (in C)  
that handles billions of transactions per year, and I have never  
encountered any database corruption. The application has been running  
non-stop for the past four years.

It wasn't always so, however. It took me some time to make sure I was  
doing everything correctly.

So perhaps what you're asking for is not a database that is more  
reliable, but rather something that is more tolerant of sloppy  
programming.

Personally, I'd like to see SKS re-implemented in C.



_______________________________________________
Sks-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/sks-devel

smime.p7s (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Has anyone looked into bdb alternatives?

Alex Roper-2
That would be nice. Personally I always use a database abstraction layer of some kind so
it's easy to choose your backend. As I mentioned, mysql and postgres would be a far
cleaner setup for us in particular, and I'd like to see that here.

Joseph Oreste Bruni wrote:

> On Feb 14, 2009, at 3:59 PM, Alex Roper wrote:
>
>> on issues (need to run reconstruction every day), as I usually notice
>> with bdb-based applications.
>
> In defense of BDB, I have written a multi-process application (in C)
> that handles billions of transactions per year, and I have never
> encountered any database corruption. The application has been running
> non-stop for the past four years.
>
> It wasn't always so, however. It took me some time to make sure I was
> doing everything correctly.
>
> So perhaps what you're asking for is not a database that is more
> reliable, but rather something that is more tolerant of sloppy programming.
>
> Personally, I'd like to see SKS re-implemented in C.
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Sks-devel mailing list
> [hidden email]
> http://lists.nongnu.org/mailman/listinfo/sks-devel

_______________________________________________
Sks-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/sks-devel

signature.asc (300 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Has anyone looked into bdb alternatives?

Dinko Korunic
In reply to this post by Joseph Oreste Bruni-3
On Sat, Feb 14, 2009 at 09:47:40PM -0700, Joseph Oreste Bruni wrote:
> encountered any database corruption. The application has been running  
> non-stop for the past four years.

OpenLDAP, another rather popular BDB-based application has been suffering
from same DB corruption problem for a long time -- I've seen problems with
BDB 4.1, 4.2, 4.3... :/

> So perhaps what you're asking for is not a database that is more  
> reliable, but rather something that is more tolerant of sloppy  
> programming.

When I've migrated SKS to BDB 4.4 a few years ago, I haven't had any
problems whatsoever -- neither with any kind of memory leaking nor with DB
corruption.

--
NAME:Dinko.kreator.Korunic   DISCLAIMER:Standard.disclaimer.applies
ICQ:16965294        JAB:[hidden email]        PGP:0xEA160D0B
HOME:http://dkorunic.net    QUOTE:Eat.right.stay.fit.and.die.anyway


_______________________________________________
Sks-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Has anyone looked into bdb alternatives?

John Marshall-2
In reply to this post by Alex Roper-2
On Sat, 14 Feb 2009, 14:59 -0800, Alex Roper wrote:
> I've been having terrible corruption issues (need to run reconstruction
> every day), as I usually notice with bdb-based applications. I was

Perhaps the port of BDB to your platform isn't very good; or perhaps
there's some underlying disc sub-system problem; or perhaps you are
using an older BDB?

I've only been running SKS for a few weeks but haven't seen any
corruption issues.  The logs are clean.  I'm running SKS 1.1.0 on BDB
4.6.21 on FreeBSD 7.1.

> wondering if anyone has experimented with patching sks to use something
> more reliable like mysql, postgres, sqlite, etc?

If the issue is indeed the porting of BDB to your platform, one of these
might provide a more portable solution for SKS.

--
John Marshall

_______________________________________________
Sks-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/sks-devel

attachment0 (203 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Has anyone looked into bdb alternatives?

Alex Roper-2
We're using bdb 4.6.21 on Debian Stable so I doubt it's a software issue. Hardware is
possible but unlikely, as we're properly raided and lvm-ed.

I'm really not sure what to do here. I suppose I could cronjob a shutdown sks, recover,
start sks, but this really shouldn't be necessary. I could also try rebuilding the
database, but I already did that once.

If no one has any other suggestions, I'll cronjob the recovery once per hour. Not ideal
but I guess it works.

Alex

John Marshall wrote:

> On Sat, 14 Feb 2009, 14:59 -0800, Alex Roper wrote:
>> I've been having terrible corruption issues (need to run reconstruction
>> every day), as I usually notice with bdb-based applications. I was
>
> Perhaps the port of BDB to your platform isn't very good; or perhaps
> there's some underlying disc sub-system problem; or perhaps you are
> using an older BDB?
>
> I've only been running SKS for a few weeks but haven't seen any
> corruption issues.  The logs are clean.  I'm running SKS 1.1.0 on BDB
> 4.6.21 on FreeBSD 7.1.
>
>> wondering if anyone has experimented with patching sks to use something
>> more reliable like mysql, postgres, sqlite, etc?
>
> If the issue is indeed the porting of BDB to your platform, one of these
> might provide a more portable solution for SKS.
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Sks-devel mailing list
> [hidden email]
> http://lists.nongnu.org/mailman/listinfo/sks-devel

_______________________________________________
Sks-devel mailing list
[hidden email]
http://lists.nongnu.org/mailman/listinfo/sks-devel

signature.asc (300 bytes) Download Attachment