Building SKS on Alpine Linux 3.12 with ocaml 4.08

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Building SKS on Alpine Linux 3.12 with ocaml 4.08

Jeremy T. Bouse
Okay, so I have completed my move and finally had some time to take a look back at trying to get sks.undergrid.net back online and started with trying to rebuild my Docker images which I found the build to be failing in part to the repo change. So I remembered the email here and updated my Dockerfile download location and got the build going again. I was of course getting the latest released version 1.1.6. This time when it built if failed with the following:

ocamlmklib -o cryptokit rijndael-alg-fst.o stubs-aes.o d3des.o stubs-des.o arcfour.o stubs-arcfour.o sha1.o stubs-sha1.o sha256.o stubs-sha256.o ripemd160.o stubs-ripemd160.o blowfish.o stubs-blowfish.o keccak.o stubs-sha3.o stubs-md5.o stubs-zlib.o stubs-misc.o stubs-rng.o -L/usr/lib -lz
ocamlc -g -c  cryptokit.mli
ocamlc -g -c  cryptokit.ml
File "cryptokit.ml", line 16, characters 5-8:
16 | open Nat
          ^^^
Error: Unbound module Nat
make[1]: *** [Makefile:95: cryptokit.cmo] Error 2
make[1]: Leaving directory '/sks-keyserver/cryptokit-1.7/src'
make: *** [Makefile:292: cryptokit-1.7/src/cryptokit.cma] Error 2

Doing some research I found this related to the Num library so I started trying to figure out how to solve that. Some mentions to OPAM but haven't used it and zero mention of actual commands needed to resolve the issue were found. So I started looking at the master branch and just grabbing SKS via git thinking what the hell I'll give that a shot. This time I get the failure on cryptokit which I kind of expected given the other but this error was different:

make[1]: Leaving directory '/tmp/sks-keyserver/bdb'
cc -O3 -Werror-implicit-function-declaration -I`ocamlc -where` -I . -c crc.c
ocamlfind ocamlc -package cryptokit,unix,str,bigarray,num -I bdb -ccopt -L/usr/lib  -ccopt -Lbdb -annot -bin-annot -warn-error A -linkpkg -g bdb.cma -c pSet.mli
ocamlfind: Package `cryptokit' not found
make: *** [Makefile:342: pSet.cmi] Error 2

So reading up some I see it was moved to be expected as an external link but couldn't see any updates to changes in the build pre-requirements. So anyone else ran into this and solved the build issues?
Reply | Threaded
Open this post in threaded view
|

Re: Building SKS on Alpine Linux 3.12 with ocaml 4.08

Dan Egli
On 10/14/2020 9:15 PM, Jeremy T. Bouse wrote:
Okay, so I have completed my move and finally had some time to take a look back at trying to get sks.undergrid.net back online and started with trying to rebuild my Docker images which I found the build to be failing in part to the repo change. So I remembered the email here and updated my Dockerfile download location and got the build going again. I was of course getting the latest released version 1.1.6. This time when it built if failed with the following:

[ much deleted for brevity

So reading up some I see it was moved to be expected as an external link but couldn't see any updates to changes in the build pre-requirements. So anyone else ran into this and solved the build issues?

I had a lot of problems getting SKS up and running. I finally found a good idea after talking to some of the gentoo devs on IRC. Add net-misc/sks to your /etc/portage/package.accept_keywords, then emerge sks again, it should grab a DATED version of sks (sks_1.1.6_p20200624). That one compiled fine where as all others seemed to want to die somewhere. Just be sure you have the dev-ml/num package installed first, because that i will be needed and I can't recall if this version includes that or not. Previous versions omitted that dependency.

-- 
Dan Egli
On my Test server

OpenPGP_0xF8A7B3F2AAB08F9D.asc (1K) Download Attachment
OpenPGP_signature (505 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Building SKS on Alpine Linux 3.12 with ocaml 4.08

Dan Egli
On 10/14/2020 9:40 PM, Dan Egli wrote:
On 10/14/2020 9:15 PM, Jeremy T. Bouse wrote:
Okay, so I have completed my move and finally had some time to take a look back at trying to get sks.undergrid.net back online and started with trying to rebuild my Docker images which I found the build to be failing in part to the repo change. So I remembered the email here and updated my Dockerfile download location and got the build going again. I was of course getting the latest released version 1.1.6. This time when it built if failed with the following:

[ much deleted for brevity

So reading up some I see it was moved to be expected as an external link but couldn't see any updates to changes in the build pre-requirements. So anyone else ran into this and solved the build issues?

I had a lot of problems getting SKS up and running. I finally found a good idea after talking to some of the gentoo devs on IRC. Add net-misc/sks to your /etc/portage/package.accept_keywords, then emerge sks again, it should grab a DATED version of sks (sks_1.1.6_p20200624). That one compiled fine where as all others seemed to want to die somewhere. Just be sure you have the dev-ml/num package installed first, because that i will be needed and I can't recall if this version includes that or not. Previous versions omitted that dependency.

-- 
Dan Egli
On my Test server

Wait, ignore me. Sorry. I was thinking of Gentoo and you mentioned Alpine Linux. Sorry about that. Wasn't paying attention to where I was.
(slaps himself)


-- 
Dan Egli
On my Test server

OpenPGP_0xF8A7B3F2AAB08F9D.asc (1K) Download Attachment
OpenPGP_signature (505 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Building SKS on Alpine Linux 3.12 with ocaml 4.08

Todd Fleisher
In reply to this post by Dan Egli
I personally recommend an Ubuntu 18.04LTS system, using the somewhat patched package found @ https://launchpad.net/~canonical-sysadmins/+archive/ubuntu/sks-public/+packages to protect against the so-called “poison keys” that will almost certainly cause your system to be unstable & use much more bandwidth & IO than is necessary. This path will render compilation unnecessary.

-T

On Oct 14, 2020, at 20:40, Dan Egli <[hidden email]> wrote:

On 10/14/2020 9:15 PM, Jeremy T. Bouse wrote:
Okay, so I have completed my move and finally had some time to take a look back at trying to get sks.undergrid.net back online and started with trying to rebuild my Docker images which I found the build to be failing in part to the repo change. So I remembered the email here and updated my Dockerfile download location and got the build going again. I was of course getting the latest released version 1.1.6. This time when it built if failed with the following:

[ much deleted for brevity

So reading up some I see it was moved to be expected as an external link but couldn't see any updates to changes in the build pre-requirements. So anyone else ran into this and solved the build issues?

I had a lot of problems getting SKS up and running. I finally found a good idea after talking to some of the gentoo devs on IRC. Add net-misc/sks to your /etc/portage/package.accept_keywords, then emerge sks again, it should grab a DATED version of sks (sks_1.1.6_p20200624). That one compiled fine where as all others seemed to want to die somewhere. Just be sure you have the dev-ml/num package installed first, because that i will be needed and I can't recall if this version includes that or not. Previous versions omitted that dependency.

-- 
Dan Egli
On my Test server
<OpenPGP_0xF8A7B3F2AAB08F9D.asc>


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Building SKS on Alpine Linux 3.12 with ocaml 4.08

Dan Egli
On 10/14/2020 10:05 PM, Todd Fleisher wrote:
> I personally recommend an Ubuntu 18.04LTS system, using the somewhat
> patched package found
> @ https://launchpad.net/~canonical-sysadmins/+archive/ubuntu/sks-public/+packages
> <https://launchpad.net/~canonical-sysadmins/+archive/ubuntu/sks-public/+packages> to
> protect against the so-called “poison keys” that will almost certainly
> cause your system to be unstable & use much more bandwidth & IO than
> is necessary. This path will render compilation unnecessary.
>
> -T


To each their own I suppose. I know a lot of people use Ubuntu. Hell,
it's running on my VPS. But the ONLY reason Gentoo ISN'T running is that
it wasn't offered as an option and I didn't (and don't) want to go to
the hassle of manually installing Gentoo and removing Ubuntu LTS at the
same time. When I finally move from a VPS to a physical server, I can
promise it will be Gentoo. But let's not get into a distro war. The
above patch sounds interesting. I'm running SKS on the VPS and if that
package has patches that fix flaws the main Debian package doesn't, then
I'll look into it myself. Thanks for the tip!

--
Dan Egli
On my Test server


OpenPGP_0xF8A7B3F2AAB08F9D.asc (1K) Download Attachment
OpenPGP_signature (505 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Building SKS on Alpine Linux 3.12 with ocaml 4.08

Todd Fleisher
For sure I’m not trying to preach to or convert anyone. Just (re-)offering my $0.02 regarding my experiences with SKS in particular. The same package would probably run just as well on Debian or maybe even other OS’s if you convert it to a native package. But since you brought it up … hopefully you get a laugh out of this classic adaptation:


-T

On Oct 15, 2020, at 12:30, Dan Egli <[hidden email]> wrote:

But let's not get into a distro war.


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Building SKS on Alpine Linux 3.12 with ocaml 4.08

Ángel
In reply to this post by Todd Fleisher
On 2020-10-14 at 21:05 -0700, Todd Fleisher wrote:
> I personally recommend an Ubuntu 18.04LTS system, using the somewhat
> patched package found @
> https://launchpad.net/~canonical-sysadmins/+archive/ubuntu/sks-public/+packages
>  to protect against the so-called “poison keys” that will almost
> certainly cause your system to be unstable & use much more bandwidth
> & IO than is necessary. This path will render compilation
> unnecessary.
>
> -T

First of all, those patches protect against a single poison key,
0xE41ED3A107A7DBC7. By skipping the merge of changes to it, I think.

Second, this may actually not be a good idea at all. sks key
reconciliation works by having two servers with different contents for
a "file" end up with the same one. If one of the parties is picky and
reject some keys the other has, the system might fall apart.
Ideally, a rejection of certain keys would have to be network-wide.
Otherwise, the reconciliation could fail, or the servers might be
continuously retrying that key which is actually rejected by the other
party. I'm not sure if this is actually a problem with this patch (I
hope someone better understanding the protocol can chime in and
explain), but seems a reason for concern.
Also, I expect that if you started from a dump which already has the
forbidden key, this patch was probably a no-op and that reconciliation
issue would go unnoticed.


Best regards


Reply | Threaded
Open this post in threaded view
|

Re: Building SKS on Alpine Linux 3.12 with ocaml 4.08

Todd Fleisher


> On Oct 15, 2020, at 17:58, Ángel <[hidden email]> wrote:
>
> First of all, those patches protect against a single poison key,
> 0xE41ED3A107A7DBC7. By skipping the merge of changes to it, I think.

I suppose one is better than none. I also block several other (popular?) keys that are problematic at the NGINX level after having issues with them in the past causing server instability. It’s far from a perfect system, but between that and automatic service restarts when SKS crashes I rarely have to touch anything anymore. *knocks on wood*

> Second, this may actually not be a good idea at all. sks key
> reconciliation works by having two servers with different contents for
> a "file" end up with the same one. If one of the parties is picky and
> reject some keys the other has, the system might fall apart.
> Ideally, a rejection of certain keys would have to be network-wide.
> Otherwise, the reconciliation could fail, or the servers might be
> continuously retrying that key which is actually rejected by the other
> party. I'm not sure if this is actually a problem with this patch (I
> hope someone better understanding the protocol can chime in and
> explain), but seems a reason for concern.
> Also, I expect that if you started from a dump which already has the
> forbidden key, this patch was probably a no-op and that reconciliation
> issue would go unnoticed.
Meh. I understand your thought process here, but don’t see it as a problem. At worst, servers will keep thinking they need to recon a problem key with a remote server that will never accept it. I cannot foresee how this would cause any real strain on the network no less to the point it might fall apart … any more so than it is already at risk of anyway.

-T

> Best regards
>
>


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Building SKS on Alpine Linux 3.12 with ocaml 4.08

ygrek
In reply to this post by Jeremy T. Bouse
Hi,

 cryptokit is now an external dependency, there are really 2 ways to make it work :
 * system way : if cryptokit is packaged for your distribution - install it (e.g. apt install libcryptokit-ocaml-dev on debian)
 * opam way : install opam through your system package manager and then install cryptokit with opam.
     opam init
     opam install cryptokit
   it is recommended to remove all ocaml packages installed via system package manager in this case

 Documentation shall be updated for the next release

--

Reply | Threaded
Open this post in threaded view
|

Re: Building SKS on Alpine Linux 3.12 with ocaml 4.08

Skip Carter
In reply to this post by Todd Fleisher
What are the characteristics of a poison key ? What makes it bad ?
I wonder if there is an algorithmic way to deal with them instead of a
blacklist.

On Thu, 2020-10-15 at 21:45 -0700, Todd Fleisher wrote:

> >
> > On Oct 15, 2020, at 17:58, Ángel <[hidden email]> wrote:
> >
> > First of all, those patches protect against a single poison key,
> > 0xE41ED3A107A7DBC7. By skipping the merge of changes to it, I
> > think.
>
> I suppose one is better than none. I also block several other
> (popular?) keys that are problematic at the NGINX level after having
> issues with them in the past causing server instability. It’s far
> from a perfect system, but between that and automatic service
> restarts when SKS crashes I rarely have to touch anything anymore.
> *knocks on wood*
>

--
Dr Everett (Skip) Carter  0xF29BF36844FB7922
[hidden email]

Taygeta Scientific Inc
607 Charles Ave
Seaside CA 93955
831-641-0645 x103



signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Building SKS on Alpine Linux 3.12 with ocaml 4.08

Todd Fleisher
On Oct 16, 2020, at 08:46, Skip Carter <[hidden email]> wrote:

What are the characteristics of a poison key ?

A large number of bogus 3rd party signatures applied to the public key and uploaded to the network

What makes it bad ?

The key size becomes too large for GPG to process it

I wonder if there is an algorithmic way to deal with them instead of a
blacklist.

This has been discussed to death on the list previously. Check the archives if you’d like more info. The short answer is no due to a lack of development resources. GNUPG has already mitigated against this by stripping 3rd party signatures & numerous GPG implementations have also moved to keys.openpgp.org as the default keyserver in response to this issue.

-T

--
Dr Everett (Skip) Carter  0xF29BF36844FB7922
[hidden email]

Taygeta Scientific Inc
607 Charles Ave
Seaside CA 93955
831-641-0645 x103




signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Building SKS on Alpine Linux 3.12 with ocaml 4.08

Jeremy T. Bouse
So I've spent the weekend working on my SKS Docker image build... The repo is available at https://github.com/UGNS/sks-docker and the image itself is available at https://hub.docker.com/r/jtbouse/sks

I'd welcome some further sets of eyes on it. I've ran several tests against it all weekend with keydumps from https://mirror.cyberbits.eu/sks/dump/ and https://sks.pod02.fleetstreetops.com/dump/2020-10-15/ after I had some initial Seg Fault issues during the key import step. I've got a couple patches applied based on works I was able to pull together from various sources in an attempt to clean up the build and harden the deployment.

Next step is to work on my Terraform deployment now that I have the image rebuilt and seemingly working without issues. Once I get it up and running I'll be looking to find some peers again.

On Fri, Oct 16, 2020 at 12:42 PM Todd Fleisher <[hidden email]> wrote:
On Oct 16, 2020, at 08:46, Skip Carter <[hidden email]> wrote:

What are the characteristics of a poison key ?

A large number of bogus 3rd party signatures applied to the public key and uploaded to the network

What makes it bad ?

The key size becomes too large for GPG to process it

I wonder if there is an algorithmic way to deal with them instead of a
blacklist.

This has been discussed to death on the list previously. Check the archives if you’d like more info. The short answer is no due to a lack of development resources. GNUPG has already mitigated against this by stripping 3rd party signatures & numerous GPG implementations have also moved to keys.openpgp.org as the default keyserver in response to this issue.

-T

--
Dr Everett (Skip) Carter  0xF29BF36844FB7922
[hidden email]

Taygeta Scientific Inc
607 Charles Ave
Seaside CA 93955
831-641-0645 x103