Analyzing dumps (Was: 6 million)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Analyzing dumps (Was: 6 million)

Gabor Kiss
On Sat, 2 May 2020, Wiktor Kwapisiewicz wrote:

> On 02.05.2020 07:55, Gabor Kiss wrote:
> > I would create such a programs from the scratch but I cannot
> > find even the format description of the dump file. :-(
>
> Last time I checked dumps where just packet piles so any OpenPGP tool
> could read it.

Thanks again for the hint.

I wrote a small Perl script to see what is in dump files
at http://keys.niif.hu/keydump/. (Server is managed by me.)

I found broken dumps. Certain RFC-4880 packets are truncated. For example
let's see signatures of key 0x7cec0e7c93115f7e:

00483ad0     89 01 22 04 10 01 02  00 0c 05 02 44 cf db 85  |..."........D...|
00483ae0  05 03 00 93 89 01 22 04  10 01 02 00 0c 05 02 4d  |......"........M|

We can see a signature packet starting at 00483ad1.
(89 01 22 is a typical old style packet header.) Its length should be
0x122 octets however it breaks in middle of the second subpacket starting
at 00483ae0. A new packet starts at 00483ae4 but my simple parser cannot
detect this and gets confused.
(Unfortunately such a truncated packet may block the import procedure
also on a newly set up key server, I guess.)

I cannot imagine how this dump could be created.
Could the attacker upload broken packets or is it "sks dump"
who garbled the dump file? Or file became bad during
compression/decompression?

Another observation: some keys have enermous amount of signatures.
"Yegor Timoshenko <[hidden email]>" may be a recorder
with 174612 sigs. This is one of the poisoned keys, isn't it?

Gabor
--
No smoke, no drugs, no vindoze.

Reply | Threaded
Open this post in threaded view
|

Re: Analyzing dumps (Was: 6 million)

Wiktor Kwapisiewicz
Hi,

Just a wild guess but maybe the packet continues in the next dump file?
That is the thing that breaks down dumps just cuts them at byte
boundaries not packet boundaries.

On 03.05.2020 11:15, Kiss Gabor (Bitman) wrote:
> Another observation: some keys have enermous amount of signatures.
> "Yegor Timoshenko <[hidden email]>" may be a recorder
> with 174612 sigs. This is one of the poisoned keys, isn't it?

Wow, that's bad, surprisingly it seems riseup doesn't offer keys via Web
Key Directory so it's quite hard to get the clean key.

Kind regards,
Wiktor

--
https://metacode.biz/@wiktor

Reply | Threaded
Open this post in threaded view
|

Re: Analyzing dumps (Was: 6 million)

William Hay
In reply to this post by Gabor Kiss
Thus spake "Kiss Gabor (Bitman)":
> I cannot imagine how this dump could be created.
> Could the attacker upload broken packets or is it "sks dump"
> who garbled the dump file? Or file became bad during
> compression/decompression?

I don't think sks does much validation of packets so anything could
be uploaded. I've found in the past with broken dumps that hkt from
hopenpgp-tools works fairly well as a filter.  It manages to skip
broken keys and carry on while GnuPG just aborts at the first badly
broken key.

William

Reply | Threaded
Open this post in threaded view
|

Re: Analyzing dumps (Was: 6 million)

Ángel
In reply to this post by Gabor Kiss
On 2020-05-03 at 11:15 +0200, Kiss Gabor (Bitman) wrote:

> On Sat, 2 May 2020, Wiktor Kwapisiewicz wrote:
>
> > On 02.05.2020 07:55, Gabor Kiss wrote:
> > > I would create such a programs from the scratch but I cannot
> > > find even the format description of the dump file. :-(
> >
> > Last time I checked dumps where just packet piles so any OpenPGP tool
> > could read it.
>
> Thanks again for the hint.
>
> I wrote a small Perl script to see what is in dump files
> at http://keys.niif.hu/keydump/. (Server is managed by me.)
>
> I found broken dumps. Certain RFC-4880 packets are truncated. For example
> let's see signatures of key 0x7cec0e7c93115f7e:
>
> 00483ad0     89 01 22 04 10 01 02  00 0c 05 02 44 cf db 85  |..."........D...|
> 00483ae0  05 03 00 93 89 01 22 04  10 01 02 00 0c 05 02 4d  |......"........M|
>
> We can see a signature packet starting at 00483ad1.
> (89 01 22 is a typical old style packet header.) Its length should be
> 0x122 octets however it breaks in middle of the second subpacket starting
> at 00483ae0. A new packet starts at 00483ae4 but my simple parser cannot
> detect this and gets confused.
> (Unfortunately such a truncated packet may block the import procedure
> also on a newly set up key server, I guess.)
>
> I cannot imagine how this dump could be created.
> Could the attacker upload broken packets or is it "sks dump"
> who garbled the dump file? Or file became bad during
> compression/decompression?

My own script can handle them fine.
There is always a container key around the packets, so it's not a
problem. But yes, sometimes there is a container that claims to have
more data than it has available, particularly around subpackets. Seems
that part is not verified by sks.

I see exactly one error on your first file (0000), on this key of
'Florian Streibelt':
Public Key v4 DSA - Key 3B8EA41F82F61240 - Created 2004-10-22 19:25:43
    #ERROR: INVALID PACKET: Packet of 70 bytes but at least 1165 bytes needed
    # Creation ERROR: INVALID PACKET: Packet of 70 bytes but at least 1165 bytes needed
    - ERROR: INVALID PACKET: Packet of 70 bytes but at least 1165 bytes needed
ERROR: INVALID PACKET: Packet of 70 bytes but at least 1165 bytes needed


While the whole keys are manageable, I do see a large number of errors than I used to.

So I took a mattrude dump from 80 days ago. That had 5978288 keys vs 6008063 of your dump
(an increase of 0.5% keys). Then I compared the number of keys with an error in both cases.
I considered a key to have an error if it detected a problem anywhere,
regardless of the number of signatures it had (the top one was a key
with 110 diagnostics).

On the February dump, there were 135 "broken keys" out of 5978288
(0,002258%)
On your dump, there are 134 "broken keys" out of 6008063 keys (0,00223%)


So not a problem of your server/dump, and not an increase of garbled
keys, either.

As for the 0x7cec0e7c93115f7e key (Terry Lansdown) I can import it from
your keyserver and use fine, also exporting with no error.
That key lives on keydump-sks-0054.pgp at 5956981 with a length of 29111
bytes.

I can find a funny signature there:
  - Signature v4 sig RSA SHA1
    # Creation 2006-08-02 00:53:57
    # SigExpiration 111d21h47m45s
ERROR: INVALID PACKET: Packet of 290 bytes but at least 8710 bytes
needed

This seems the opposite to your problem, though.

that is followed by
  - Signature v4 sig RSA SHA1
    # Creation 2006-08-15 12:17:53
    # SigExpiration 14d
    - Issuer 9710B89BCA57AD7C


Interestingly, pgpdump does get confused by it

> Old: Signature Packet(tag 2)(290 bytes)
>         Ver 4 - new
>         Sig type - Generic certification of a User ID and Public Key packet(0x10).
>         Pub alg - RSA Encrypt or Sign(pub 1)
>         Hash alg - SHA1(hash 2)
>         Hashed Sub: signature creation time(sub 2)(4 bytes)
>                 Time - Wed Aug  2 00:53:57 CEST 2006
>         Hashed Sub: signature expiration time(sub 3)(4 bytes)
>                 Time - Tue Nov 21 21:41:42 CET 2006
>         Sub: reserved(sub 1)(15 bytes)
>         Sub: additional decryption key(sub 10) WARNING: see CA-2000-18!!!(-1 bytes)
>                 Class - Unknown class(09)
>                 Pub alg - ElGamal Encrypt-Only(pub 16)
>                 Fingerprint - 97 10 b8 9b ca 57 bf 6b 5b 5b 4f 74 8d 81 87 63 79 34 9b 83
>         Sub: unknown(sub 46, critical)(13301 bytes)
>         Hash left 2 bytes - c6 6e
>         RSA m^d mod n(13079 bits) - ...
>                 -> PKCS-1
> New: unknown(tag 56)(11 bytes)
> Old: Signature Packet(tag 2)(until eof)
>         Ver 218 - unknown


gpg --list-packets view is slightly different:


> :signature packet: algo 1, keyid 9710B89BCA57AD7C
>         version 4, created 1154472837, md5len 0, sigclass 0x10
>         digest algo 2, begin of digest 47 45
>         hashed subpkt 2 len 4 (sig created 2006-08-01)
>         hashed subpkt 3 len 4 (sig expires after 14d0h0m)
>         subpkt 16 len 8 (issuer key ID 9710B89BCA57AD7C)
>         data: [2047 bits]
> :signature packet: algo 1, keyid 0000000000000000
>         version 4, created 1154472837, md5len 0, sigclass 0x10
>         digest algo 2, begin of digest b7 6d
>         hashed subpkt 2 len 4 (sig created 2006-08-01)
>         hashed subpkt 3 len 4 (sig expires after 111d21h47m)
>         subpkt 1 len 15 (?)
>         subpkt 10 len 0 (additional recipient request)
> WARNING: PGP versions > 5.0 and < 6.5.8 will automagically encrypt to this key and thereby reveal the plaintext to the owner of this ARR key. Detailed info follows:
>         subpkt 10 len 4294967295 (too short: buffer is only 8690)
>         subpkt 9 len 9 (key expires after 8y301d11h27m)
>         subpkt 91 len 90 (?)
>         subpkt 37 len 128 (?)
>         subpkt 14 len 167 (?)
>         critical subpkt 87 len 6 (?)
>         subpkt 95 len 73 (?)
>         critical subpkt 100 len 157 (experimental / private subpacket)
>         subpkt 9 len 95 (key expires after 109y205d21h52m)
>         subpkt 68 len 160 (?)
>         data: [MPI_NULL]


I then extracted the key and performed a gpgsplit. Those above are 000013-002.sig and 000014-002.sig

If you compare them, these two signature packets are extremely similar:

13:
00000010  03 00[12 75 00 00 0a 09  10 97 10 b8 9b ca 57 ad  |...u..........W.|
00000020  7c 47 45 07 ff 78 8c c9  b5 f5 81 3f f0 1e ea 97  ||GE..x.....?....|
00000030  73 79] bf 6b 5b 5b 4f 74  8d 81 87 63 79 34 9b 83  |sy.k[[Ot...cy4..|

14:
00000010  03 00[93 89 01 22 04 10  01 02 00 0c 05 02 4d 98  |....."........M.|
00000020  35 09 05 03 00 12 75 00  00 0a 09 10 97 10 b8 9b  |5.....u.........|
00000030  ca 57]bf 6b 5b 5b 4f 74  8d 81 87 63 79 34 9b 83  |.W.k[[Ot...cy4..|

This second one is identified by pgpdump as
Old: Signature Packet(tag 2)(290 bytes)
        Ver 4 - new
        Sig type - Generic certification of a User ID and Public Key packet(0x10).
        Pub alg - RSA Encrypt or Sign(pub 1)
        Hash alg - SHA1(hash 2)
        Hashed Sub: signature creation time(sub 2)(4 bytes)
                Time - Wed Aug  2 00:53:57 CEST 2006
        Hashed Sub: signature expiration time(sub 3)(4 bytes)
                Time - Tue Nov 21 21:41:42 CET 2006
        Sub: reserved(sub 1)(15 bytes)
        Sub: additional decryption key(sub 10) WARNING: see CA-2000-18!!!(-1 bytes)
                Class - Unknown class(09)
                Pub alg - ElGamal Encrypt-Only(pub 16)
                Fingerprint - 97 10 b8 9b ca 57 bf 6b 5b 5b 4f 74 8d 81 87 63 79 34 9b 83
        Sub: unknown(sub 46, critical)(13301 bytes)

That CA-2000-18 reference is about CERT Advisory about Additional Decryption Keys on unhashed subpacket
https://resources.sei.cmu.edu/asset_files/whitepaper/2000_019_001_496188.pdf



00000000  89 01 22 04 10 01 02 00  
00 0c → 12 hashed subpacket bytes
05 02 44 cf db 85 05
    ^ 2 = Signature Creation Time:  Wed Aug  2 00:53:57 CEST 2006
05 03 00 93 89 01
    ^ 3 = Signature Expiration Time: 111 days later

22 04
Unhashed packet length of 8708 (invalid)

10 length of 16 bytes
 01 Subpacket type  1 (reserved)
 02 00 0c 05 02 4d 98 35 09 05 03 00 12 75 00
00 invalid length of 0 bytes
 0a Subpacket type 10 additional decryption key (Placeholder for backward compatibility in rfc4880)
 etc.


However, if we discard the first 20 bytes of this signature, we get a quite differen signature:

> Old: Signature Packet(tag 2)(290 bytes)
> Ver 4 - new
> Sig type - Generic certification of a User ID and Public Key packet(0x10).
> Pub alg - RSA Encrypt or Sign(pub 1)
> Hash alg - SHA1(hash 2)
> Hashed Sub: signature creation time(sub 2)(4 bytes)
> Time - Sun Apr  3 10:51:21 CEST 2011
> Hashed Sub: signature expiration time(sub 3)(4 bytes)
> Time - Sun Apr 17 10:51:21 CEST 2011
> Sub: key flags(sub 27)(critical)(51 bytes)
> Flag - This key may be used to certify other keys
> Flag - This key may be used to sign data
> Flag - The private component of this key may be in the possession of more than one person
> Hash left 2 bytes - b8 de
> RSA m^d mod n(54585 bits) - ....
>

Those key flags are certainly weird, and the stated mpi is larger than expected, but these signature times make much more sense.

So it seems that at a given point, a mid-written signature got replaced by a different one, and so we ended up with this broken signature item.


Best regards