Allow Rust implementations of MerkleTree / Proof / SparseMerkleTree to use non-keccak256 hash functions

[serejke]

Overview

For non-EVM chains integration it may be necessary to use a hash function other than keccak256. For instance, we're working on Cardano integration. Cardano does not have a native support of keccak256, and implementing it in a smart contract is computationally/gas unfeasible. We've opened a CIP for keccak256 support, but it may take several months to a year to make into the Cardano mainnet. For now the only option is to use blake2b (256) for MerkleTree (on-chain and off-chain) on Cardano side.

Description

This task is about refactoring the off-chain agents to make the hash-function customizable. For simplicity, the new hash function must return H256. Returning a different value would require a massive refactoring throughout the whole codebase. So hash functions with a bigger hash size will not be supported for now.

hyperlane-monorepo/rust/hyperlane-core/src/accumulator/mod.rs

Line 18 in 1fff74e

pub(super) fn hash_concat(left: impl AsRef<[u8]>, right: impl AsRef<[u8]>) -> H256 {

hash_concat(left, right) strictly depends on Keccak256.

Suggestion

I see a couple of ways to resolving this:

Add a trait HashFunction

pub trait HashFunction {
    fn hash_concat(&self, left: impl AsRef<[u8]>, right: impl AsRef<[u8]>) -> H256;
}

pub struct Keccak256HashFunction;

impl HashFunction for Keccak256HashFunction {
    fn hash_concat(&self, left: impl AsRef<[u8]>, right: impl AsRef<[u8]>) -> H256 {
        H256::from_slice(Keccak256::new().chain(left).chain(right).finalize().as_slice())
    }
}

pub struct Blake2b256HashFunction;

impl HashFunction for Blake2b256HashFunction {
    fn hash_concat(&self, left: impl AsRef<[u8]>, right: impl AsRef<[u8]>) -> H256 {
        let mut hasher = Blake2b::new();        
        hasher.update(left);
        hasher.update(right);
        let result = hasher.finalize();
        H256::from_slice(result.as_slice())
    }
}

Then pass the HashFunction trait to the hash_concat.

• dynamically as &dyn HashFunction — then we need to store the object reference in all dependent structs
• statically as &HashFunction — then all the dependent structs need to have a generic parameter and own the function

Using cargo feature flags

// accumulator/mod.rs

#[cfg(feature = "keccak256")]
pub use keccak256::hash_concat;

#[cfg(feature = "blake2b256")]
pub use blake2b256::hash_concat;

// accumulator/mod.rs Cargo.toml

[features]
default = ["keccak256"]

// Cargo.toml
[dependencies]
accumulator = { version = "*", features = ["blake2b256"] }

[nambrot]

Related to #2258, I'll try to capture it more broadly as well

[nambrot]

Tried to capture it comprehensively at #2399