Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[Relayer] Implement dedicated relay store to optimize memory usage #1114

Draft
wants to merge 3 commits into
base: fix/relayminer-goroutine-leak
Choose a base branch
from

Conversation

red-0ne
Copy link
Contributor

@red-0ne red-0ne commented Mar 10, 2025

Summary

Adds persisted relay store to decouple relay data from SMST storage

Primary Changes:

  • Introduces RelayStore interface for persisting relay data separately from SMST
  • Implements file-based relay storage with buffered I/O optimizations
  • Modifies proof submission to include full relay data for signature verification

Secondary Changes:

  • Updates proto definitions to include proof_relay field
  • Adds new error types for relay storage operations
  • Modifies test utilities to handle relay proof verification

Issue

  • Description: Currently, the relayer's SMST stores the complete relay data within its nodes in memory alongside the SMST leads to high memory usage. Considering that relay can be several kilobytes in size.
  • Issue: [Load Testing] 10B Relay Load Test #1106

Type of change

Select one or more from the following:

Sanity Checklist

  • I have updated the GitHub Issue assignees, reviewers, labels, project, iteration and milestone
  • For docs, I have run make docusaurus_start
  • For code, I have run make go_develop_and_test and make test_e2e
  • For code, I have added the devnet-test-e2e label to run E2E tests in CI
  • For configurations, I have update the documentation
  • I added TODOs where applicable

@red-0ne red-0ne added relayminer Changes related to the Relayminer loadtest Work related to load testing scalability smt Sprase Merkle Tree Related proof Claim & Proof life cycle consensus-breaking IMPORTANT! If the PR with this tag is merged, next release WILL HAVE TO BE an upgrade. labels Mar 10, 2025
@red-0ne red-0ne added this to the Beta TestNet Iteration milestone Mar 10, 2025
@red-0ne red-0ne requested review from Olshansk and okdas March 10, 2025 10:59
@red-0ne red-0ne self-assigned this Mar 10, 2025
@red-0ne red-0ne removed request for Olshansk and okdas March 11, 2025 18:14
@red-0ne red-0ne marked this pull request as draft March 11, 2025 18:14
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
consensus-breaking IMPORTANT! If the PR with this tag is merged, next release WILL HAVE TO BE an upgrade. loadtest Work related to load testing proof Claim & Proof life cycle relayminer Changes related to the Relayminer scalability smt Sprase Merkle Tree Related
Projects
Status: 🏗 In progress
Development

Successfully merging this pull request may close these issues.

2 participants