Git's usage of SHA1

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Git's usage of SHA1

grarpamp
A long thread on the below list with the above subject (that SHA1
is more or less broken and should be proactively replaced along
the lines of other general global movements off of SHA1) mentioned
the snippet below. The same subject should be given consideration
in monotone as well.

https://lists.sonic.net/mailman/private/crypto-practicum/2016q1/thread.html
  Also, historically speaking git's usage of SHA1 almost certainly came
from monotone (monotone.ca), which Linus used as inspiration for git.
They still use SHA1, but it sounds like it would be much easier to
replace there than in git.


There were some design talk and thread links in there for anyone
interested to subscribe to the sonic c-p archives.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=813157
https://git.wiki.kernel.org/index.php/LinusTalk200705Transcript
https://github.com/jayphelps/git-blame-someone-else

_______________________________________________
Monotone-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/monotone-devel
Reply | Threaded
Open this post in threaded view
|

Re: Git's usage of SHA1

Brian May-11
grarpamp <[hidden email]> writes:

> A long thread on the below list with the above subject (that SHA1
> is more or less broken and should be proactively replaced along
> the lines of other general global movements off of SHA1) mentioned
> the snippet below.

Is the git community going to try to change this?

Years ago, Linus said there was no need for concern:
http://marc.info/?l=git&m=115678778717621&w=2

(not a view I agree with myself)

> https://lists.sonic.net/mailman/private/crypto-practicum/2016q1/thread.html

Can't read that without being subscribed :-(
--
Brian May <[hidden email]>

_______________________________________________
Monotone-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/monotone-devel
Reply | Threaded
Open this post in threaded view
|

Re: Git's usage of SHA1

Markus Wanner-2
In reply to this post by grarpamp
Hi,

On 02/08/2016 07:52 PM, grarpamp wrote:
> A long thread on the below list with the above subject (that SHA1
> is more or less broken and should be proactively replaced along
> the lines of other general global movements off of SHA1) mentioned
> the snippet below. The same subject should be given consideration
> in monotone as well.

I agree and have thought about teaching monotone to use other hash
functions. There's no need to rush, though. I'd rather like to think
this through well.

OTOH at the current rate of development, we better hurry up a bit. ;-)

> https://lists.sonic.net/mailman/private/crypto-practicum/2016q1/thread.html

No access.

>   Also, historically speaking git's usage of SHA1 almost certainly came
> from monotone (monotone.ca), which Linus used as inspiration for git.

Yes.

> They still use SHA1, but it sounds like it would be much easier to
> replace there than in git.

I cannot speak for git, but I've already tried in monotone and it's not
trivial, either.

> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=813157

Sounds like an entirely different issue. Monotone took the conservative
approach, here, and has always checked everything it receives via the
network.

> https://git.wiki.kernel.org/index.php/LinusTalk200705Transcript

Here, Linus' argument is that git only uses SHA-1 as a consistency
check, not for security. (I haven't checked how git's OpenPGP
integration works.)

That's certainly not the case for monotone, where certs reference the
revision id (a sha-1 hash).

> https://github.com/jayphelps/git-blame-someone-else

WTF is that? You're not trying to blame monotone for git's usage of
SHA-1, are you?

Kind Regards

Markus Wanner


_______________________________________________
Monotone-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/monotone-devel
Reply | Threaded
Open this post in threaded view
|

Re: Git's usage of SHA1

grarpamp
> https://lists.sonic.net/mailman/private/crypto-practicum/2016q1/thread.html

> Can't read that without being subscribed :-(

> No access.

Subscription to the archives is required as said, and is also
documented on the list page. It's free, no human is involved.
Bug them on policy, not me. The context for this thread begins
there and would be of interest to those with interest.

https://lists.sonic.net/mailman/listinfo/crypto-practicum

> WTF

Maybe malleability in git repos, or fun with features.

_______________________________________________
Monotone-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/monotone-devel
Reply | Threaded
Open this post in threaded view
|

Monotone's usage of SHA1

Markus Wanner-2
Hi,

On 02/09/2016 09:45 PM, grarpamp wrote:
> Subscription to the archives is required as said, and is also
> documented on the list page. It's free, no human is involved.
> Bug them on policy, not me. The context for this thread begins
> there and would be of interest to those with interest.
>
> https://lists.sonic.net/mailman/listinfo/crypto-practicum

Okay, thanks, I've read through the archives, now.

One thing I'm curious about is the proposal to use Argon2 (a password
hash) over SHA3 or Blake2b for user facing hashes (or portions thereof).
Do I understand correctly that this "only" makes it (proportionally)
harder for Mallory to come up with a collision on the first few bytes of
the resulting hash? Or put another way: Is there any point in using
Argon2 (compared to Keccak or Blake2), if the full hash is used?

Monotone is pretty rigorous in checking its data's hashes. For example,
it checks not just after receiving from another node, but even after
loading a revision from disk. Therefore, I'd be pretty hesitant to
impose that additional computational cost for the normal user.

I rather thought about using a more compact encoding, like base58 as
used by Bitcoin. That way you'd get 45% more information into those 5-7
chars that humans can comfortably pass around (compared to hex),
resulting in 29 - 40 bits of hash.

I'm not saying that's enough, either. But in the case of monotone, I'm
less concerned, because there we have integrated certs, which check
against the full hash. And just to identify a revision out of the set of
already validated revisions, 5-7 chars usually are enough. (Sounds
suspiciously similar to Linus' argument, except that certs are external
to git, AFAIUI.)

Kind Regards

Markus Wanner


_______________________________________________
Monotone-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/monotone-devel
Reply | Threaded
Open this post in threaded view
|

Re: [Crypto-practicum] Monotone's usage of SHA1

Bill Cox
On Tue, Feb 16, 2016 at 8:19 AM, Markus Wanner <[hidden email]> wrote:
Hi,

On 02/09/2016 09:45 PM, grarpamp wrote:
> Subscription to the archives is required as said, and is also
> documented on the list page. It's free, no human is involved.
> Bug them on policy, not me. The context for this thread begins
> there and would be of interest to those with interest.
>
> https://lists.sonic.net/mailman/listinfo/crypto-practicum

Okay, thanks, I've read through the archives, now.

One thing I'm curious about is the proposal to use Argon2 (a password
hash) over SHA3 or Blake2b for user facing hashes (or portions thereof).
Do I understand correctly that this "only" makes it (proportionally)
harder for Mallory to come up with a collision on the first few bytes of
the resulting hash? Or put another way: Is there any point in using
Argon2 (compared to Keccak or Blake2), if the full hash is used?


SHA3 is slower than SHA1, and would slow down Monotone somewhat.  BLAKE2b is faster than MD5, and would speed up Monotone, but it has a bit less formal analysis at this point than SHA3.

Showing Argon2 derived hashes to users is an interesting concept.  One problem: do we need to ever verify these hashes, or do can we generate them once during a commit (when an extra 100ms or so wont hurt much) and keep them around.  Verification of Argon2 hashes is very expensive.  I would not want to download 1 million commits and have to verify 1 million Argon2 hashes.  There are some algorithms with "short-cuts" where verification is faster than hashing.  One of those might be better.  The best ones are only "bandwidth hard" rather than memory hard, so their security is a bit lower.
 
Monotone is pretty rigorous in checking its data's hashes. For example,
it checks not just after receiving from another node, but even after
loading a revision from disk. Therefore, I'd be pretty hesitant to
impose that additional computational cost for the normal user.

An Argon2 hash here would be a real problem if it had to be verified every time.  Maybe the resulting hash could be signed by the committer so you only have to verify a signature?
 
I rather thought about using a more compact encoding, like base58 as
used by Bitcoin. That way you'd get 45% more information into those 5-7
chars that humans can comfortably pass around (compared to hex),
resulting in 29 - 40 bits of hash.

That might be a small improvement.
 
I'm not saying that's enough, either. But in the case of monotone, I'm
less concerned, because there we have integrated certs, which check
against the full hash. And just to identify a revision out of the set of
already validated revisions, 5-7 chars usually are enough. (Sounds
suspiciously similar to Linus' argument, except that certs are external
to git, AFAIUI.)

Kind Regards

Markus Wanner

It sounds like switching away from SHA1 is even more important for Monotone than git, since Monotone does intend to provide data integrity against malicious attackers.

Bill 

_______________________________________________
Monotone-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/monotone-devel
Reply | Threaded
Open this post in threaded view
|

Re: [Crypto-practicum] Monotone's usage of SHA1

Hendrik Boom-2
On Tue, Feb 16, 2016 at 11:53:55AM -0800, Bill Cox wrote:

> than git, since Monotone does intend to provide data integrity against
> malicious attackers.

Yay!

-- hendrik

_______________________________________________
Monotone-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/monotone-devel
Reply | Threaded
Open this post in threaded view
|

Re: Monotone's usage of SHA1

Lapo Luchini
In reply to this post by Markus Wanner-2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

BTW: I've been long lurking (and even that with loong hiatuses) but
I'm still using monotone for private projects and am still some kind
of a crypto-freak, so this thread is particularly interesting to me.

As far as modern hash go… I can see the reasons why NaCl library chose
SHA-512 as his default hash. I still have to read that thread (and
might well convince me otherwise), but SHA-512 or maybe SHA-512/256
(truncated to 256 bit) is what I would suggest to use at this moment.

- --
Lapo Luchini - http://lapo.it/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJW8/yuAAoJEIOwuMbL2nHwhIwQANxiU0Qkp6vFPbNMKSPKkQVH
ZmKdOsoMAjbIsOhHw4umslPp1vSABGC/nxEQSwcxl7NtEjSf1XjMXsr/4BPuw9q0
NoKc2+pLtnCnDeUAq9DYDahbxMgbnsbJHxbnsxaQwHCqjvTSP7zxg/yYp7WP7z+7
WD6a7YH3+rJ2+rrdh8NhgsT+kGFkNGSlOWAs1ez2x3qasueRsJQzqvPJK3iS2Qvg
6bdOn1YplbwQWQRQE0pKt4ahhgWYMyr1ShMpiUDY5JPbwev0ja2K5XDhNIceSk7f
D+wVTqIkaCElVV3J+UUQRVcY61SQFUmXEKXKZwpTS2VRLZD7xUOQUMBtypjnNM04
XZCVRi3QYg4qCYktlH75d//khGpmLOMfvJQmg6hn5scdM/jB9w15ofB8mxwYv/tv
jGUXUwoWFmYVNaMshj0Vk28Vn2/r8Rh7pdm3PnUQrWo6ITpT01Brd/+Y9HMxQPyS
1DQxersZmlINkPSi6eHGCymKXa4uRuMBrOhCBKaBzKeHH91hhNG1bxSBYcbaCHZK
IrNRdpAR31wp/bHubJsC3Jw6zstlQbsuN2IHLa9T9mmB6C7imiiVSPeuVMPb4LVs
D0TdRswG46FhiiT/i8OcKNtNsRDTDMYmzyL6JwZReWfxpy6UIwLi9QCcxMSHnYbA
57f6g7YKQOdW6KRm6v8/
=Jpz7
-----END PGP SIGNATURE-----


_______________________________________________
Monotone-devel mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/monotone-devel