Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add option to select KZG library for PeerDAS #6107

Open
Tracked by #4983
jimmygchen opened this issue Jul 16, 2024 · 2 comments
Open
Tracked by #4983

Add option to select KZG library for PeerDAS #6107

jimmygchen opened this issue Jul 16, 2024 · 2 comments
Labels
blocked das Data Availability Sampling

Comments

@jimmygchen
Copy link
Member

jimmygchen commented Jul 16, 2024

Description

We've recently switched from the c-kzg library to the Rust peerdas-kzg library on the das branch (#5941). The purpose for the switch was to:

  1. Try out a new alternative to c-kzg;
  2. Allow us to merge das branch to unstable without having to update c-kzg to a development version

However we've now lost the option to run the das branch with ckzg, which has been quite stable for a while. We've recently run into a few issues with peerdas-kzg, including slow startup time and stack overflow during block production (#6105). Therefore I think it would be great to have the option to use ckzg library, especially for public denvets, where a small number of reliable supernodes is required.

Ideally we want to be able to:

  1. maintain the ckzg library version for the mainnet code path
  2. optionally include the ckzg peerdas library when it's enabled via a feature flag, so both ckzg library will co-exist when this ckzg-peerdas feature is enabled, however this library will only be referenced by the PeerDAS portion of the code.
  3. allow selecting the ckzg-peerdas library via a hidden runtime flag
@jimmygchen jimmygchen added the das Data Availability Sampling label Jul 16, 2024
@realbigsean realbigsean changed the title Add option to select KZG library for PeerDAS Add option to select KZG library for PeerDAS Dec 17, 2024
@jimmygchen jimmygchen added das Data Availability Sampling and removed das Data Availability Sampling labels Jan 28, 2025
@jimmygchen
Copy link
Member Author

We currently use c-kzg for Deneb, and rust-eth-kzg for PeerDAS. To fully support this we'd need:

  • Re-integrate PeerDAS with c-kzg, so we can have the option to load c-kzg only
  • Use rust-eth-kzg to perform deneb KZG operations, so we can have the option to load rust-eth-kzg only
  • If we complete both changes, then we can have option to choose which KZG library to use, or feature gate them (Add option to select KZG library for PeerDAS #6107)

rust-eth-kzg seems to perform better in PeerDAS - I'm not sure if this has changed.

Alternatively, an option is to maintain the status quo for now, and gradually phase out c-kzg over time? We can't completely switch over to rust KZG yet, AFAIK it hasn't been audited, but I'd assume there are plans to do this.

@jimmygchen
Copy link
Member Author

@pawanjay176 also expressed his preference to maintain the status quo now and eventually get rid of c-kzg. It's unclear whether both libraries will be maintained in the longer term.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
blocked das Data Availability Sampling
Projects
None yet
Development

No branches or pull requests

1 participant