Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat: lz4 compression #71

Merged
merged 3 commits into from
Feb 2, 2024
Merged

feat: lz4 compression #71

merged 3 commits into from
Feb 2, 2024

Conversation

merklefruit
Copy link
Contributor

LZ4 is even faster than Snappy while keeping the same compression ratio. Very impressive!

Mainnet Capella beacon block compression & decompression test results:

uncompressed data size: 287038 bytes
gzip compression shrank the data by 334.39% in 4.06275ms
zstd compression shrank the data by 389.28% in 2.413542ms
snappy compression shrank the data by 202.82% in 454.625µs
lz4 compression shrank the data by 199.92% in 333.125µs
------
gzip decompression took 849.875µs
zstd decompression took 405.666µs
snappy decompression took 152.25µs
lz4 decompression took 128.833µs

Additional changes

  • chore: moved compression-related crates to msg-wire instead of workspace manifest
  • chore: forgot to export the connection module in lib.rs, made imports cleaner.

@mempirate
Copy link
Contributor

mempirate commented Jan 31, 2024

This looks great, could you add lz4 to the book as a supported compression algo?

@merklefruit
Copy link
Contributor Author

@mempirate yep!

Copy link
Contributor

@mempirate mempirate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mempirate mempirate merged commit 5651bf7 into main Feb 2, 2024
7 checks passed
@mempirate mempirate deleted the feat/lz4 branch February 2, 2024 15:08
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants