Extreme memory usage

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Extreme memory usage

Gabor Kiss
Dear folks,

I experienced in the past months that my SKS instance dies almost every day.
Now I think I found the reason.

Program crashes when e-mail address to be search is broken
into mostly 1-3 letter words.
These very short words - that are legitim search patters - make
sks db process eat the whole 6 GiB RAM of the VM and the OOM killer
reapes it.
Is this normal? What are your experiences?

Cheers

Gabor

_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
Reply | Threaded
Open this post in threaded view
|

Re: Extreme memory usage

Skip Carter
If you do an 'lsof' you will also see all the dump files (223 when I
built it) are open, even when having done a normal build.  The only way
to prevent this is to move the dump directory after the build finishes.
 This could also be contributing to the resource consumption.

On Wed, 2019-07-17 at 19:47 +0200, Kiss Gabor (Bitman) wrote:

> Dear folks,
>
> I experienced in the past months that my SKS instance dies almost
> every day.
> Now I think I found the reason.
>
> Program crashes when e-mail address to be search is broken
> into mostly 1-3 letter words.
> These very short words - that are legitim search patters - make
> sks db process eat the whole 6 GiB RAM of the VM and the OOM killer
> reapes it.
> Is this normal? What are your experiences?
>
> Cheers
>
> Gabor
>
> _______________________________________________
> Sks-devel mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/sks-devel
--
Dr Everett (Skip) Carter
[hidden email]

Taygeta Scientific Inc
607 Charles Ave
Seaside CA 93955
831-641-0645 x103



_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Extreme memory usage

Todd Fleisher
To Gabor’s point, I see some similar behavior on my nodes intermittently: https://i.imgur.com/aEePr6J.jpg

It seems to clear up on its own get automatically restarted by systemd after OOM kills it so for the most part it doesn’t appear to impact my pool from serving requests from the outside, presumably thanks to load balancing and/or caching. Also, it does not happen to all nodes at the same time.

FWIW - my nodes do not have the dump directory so that shouldn’t be at play here.

-T

On Jul 17, 2019, at 6:14 PM, Skip Carter <[hidden email]> wrote:

Signed PGP part
If you do an 'lsof' you will also see all the dump files (223 when I
built it) are open, even when having done a normal build.  The only way
to prevent this is to move the dump directory after the build finishes.
This could also be contributing to the resource consumption.

On Wed, 2019-07-17 at 19:47 +0200, Kiss Gabor (Bitman) wrote:
Dear folks,

I experienced in the past months that my SKS instance dies almost
every day.
Now I think I found the reason.

Program crashes when e-mail address to be search is broken
into mostly 1-3 letter words.
These very short words - that are legitim search patters - make
sks db process eat the whole 6 GiB RAM of the VM and the OOM killer
reapes it.
Is this normal? What are your experiences?

Cheers

Gabor

_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel
--
Dr Everett (Skip) Carter
[hidden email]

Taygeta Scientific Inc
607 Charles Ave
Seaside CA 93955
831-641-0645 x103





_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Extreme memory usage

Gabor Kiss
In reply to this post by Skip Carter
> If you do an 'lsof' you will also see all the dump files (223 when I
> built it) are open, even when having done a normal build.  The only way
> to prevent this is to move the dump directory after the build finishes.
>  This could also be contributing to the resource consumption.

Dump files are not open.

Gabor
--
A mug of beer, please. Shaken, not stirred.

_______________________________________________
Sks-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/sks-devel