Hash has arbitrary seed-independent multicollisions, is not DoS resistant

[orlp]

There are probably other issues as well, but this line is particularly problematic:

gxhash/src/gxhash/platform/x86.rs

Line 86 in 8bee61e

let mut tmp1 = aes_encrypt(v0, v2);

This is trivial to invert and allows you to create arbitrary seed-independent multicollisions. I would suggest not advertising DoS resistance on this hash at all.

// Not an endorsement of aes_crypto, just the first crate
// I could find that allows cross-platform single-round encryption.
use aes_crypto::AesBlock;

fn main() {
    let zero_key = AesBlock::zero();

    let mut s0 = [0u8; 192];
    let mut s1 = [0u8; 192];

    s0[64] = 100;
    s1[64] = 42;
    
    let v0 = AesBlock::new(s0[64..64 + 16].try_into().unwrap());
    v0.enc(zero_key).store_to(&mut s0[64 + 32..]);

    let v0 = AesBlock::new(s1[64..64 + 16].try_into().unwrap());
    v0.enc(zero_key).store_to(&mut s1[64 + 32..]);

    // Different strings.
    assert!(s0 != s1);
    
    // Collide regardless of seed.
    assert!(gxhash::gxhash128(&s0, 0) == gxhash::gxhash128(&s1, 0));
    assert!(gxhash::gxhash128(&s0, 0xdeadbeef) == gxhash::gxhash128(&s1, 0xdeadbeef));
}

[ogxd]

Thanks a lot for this analysis @orlp! Let's see if we can improve DoS resistance without compromising performance. I don't know if it can be made 100% proof but I'm sure it can be improved (maybe using the seed in the main construction).

Btw DoS resistance is not advertised (unless I missed something?). See in readme:

This does not mean however that it is completely DOS resistant. This has to be analyzed further.

[orlp]

Btw DoS resistance is not advertised

I disagree. The way it is worded it sounds like it has some DoS resistance when there is none at all. Similarly with multicollision resistance.

[ogxd]

Let's see first if we can address this 😄 Feel free to submit a PR if you want.
After that, depending on the results, we can reword the DoS resistance section more accurately.

[ogxd]

Did not has much time to work on this, so in the meantime I've clarified the DOS resistance part in the readme https://github.com/ogxd/gxhash?tab=readme-ov-file#security

[purplesyringa]

There's probably a lot more problems with DoS resistance here. The most obvious one, IMO, is

https://github.com/ogxd/gxhash/blob/main/src/hasher.rs#L113-L117

compress_all is completely independent from the seed, so a collision in compress_all affects all seeds.

[ogxd]

Mixing the seed at the start instead of the end should significantly improve DoS resistance. I'm currently (slowly) exploring this path.