Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat(tests/scripts): create script to retrieve trie state via rpc #3714

Merged
merged 18 commits into from
Jan 29, 2024

Conversation

jimjbrettj
Copy link
Contributor

@jimjbrettj jimjbrettj commented Jan 18, 2024

Changes

  • Created a script to query a running node via rpc to retrieve the trie state at a given block and write it to a file
  • I also added the ability to compute and verify the state root from the retrieved state

Tests

Start up a gossamer node and execute the script against it (or any running gossamer node)

Issues

#3713

Primary Reviewer

@EclesioMeloJunior

Copy link

codecov bot commented Jan 18, 2024

Codecov Report

Attention: 55 lines in your changes are missing coverage. Please review.

Comparison is base (4566b14) 50.56% compared to head (a8f77d0) 50.45%.

Additional details and impacted files
@@               Coverage Diff               @@
##           development    #3714      +/-   ##
===============================================
- Coverage        50.56%   50.45%   -0.12%     
===============================================
  Files              230      231       +1     
  Lines            28654    28728      +74     
===============================================
+ Hits             14489    14494       +5     
- Misses           12649    12711      +62     
- Partials          1516     1523       +7     

@jimjbrettj jimjbrettj force-pushed the jimmy/trieStateScript branch from e483d83 to 297003c Compare January 18, 2024 22:40
@jimjbrettj jimjbrettj added T-enhancement this issue/pr covers improvement of existing functionality. S-tests issue related to adding new tests. S-sync-westend related to particular network syncing. T-feat this issue/pr is a new feature or functionality. and removed T-enhancement this issue/pr covers improvement of existing functionality. labels Jan 18, 2024
Copy link
Member

@EclesioMeloJunior EclesioMeloJunior left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice script! 🔥 🚀 🚀

Few comments and suggestions to improve the script versatility, also could you write down some test cases?

@dimartiro
Copy link
Contributor

What if we move it to a scripts folder (at root level) instead of tests/scripts
Also we can create a make script to run it

@jimjbrettj
Copy link
Contributor Author

Responded to all feedback except @dimartiro given how simple of a script this is I did not see the benefit of using a makefile

@jimjbrettj jimjbrettj force-pushed the jimmy/trieStateScript branch from 8df1f80 to a98aec7 Compare January 22, 2024 23:49
@jimjbrettj jimjbrettj force-pushed the jimmy/trieStateScript branch from a98aec7 to a8f3f48 Compare January 23, 2024 19:30
Copy link
Contributor

@dimartiro dimartiro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think using trie.TrieLayout could make it more scalable and safety.

Also, I think the file name should be trie_state_script.go and trie_state_script_test.go

@jimjbrettj jimjbrettj force-pushed the jimmy/trieStateScript branch from a8f3f48 to d4b678c Compare January 29, 2024 18:16
@jimjbrettj jimjbrettj requested a review from dimartiro January 29, 2024 18:17
@jimjbrettj jimjbrettj force-pushed the jimmy/trieStateScript branch from d4b678c to a8f77d0 Compare January 29, 2024 18:22
@jimjbrettj jimjbrettj merged commit 5ccea40 into development Jan 29, 2024
24 checks passed
@jimjbrettj jimjbrettj deleted the jimmy/trieStateScript branch January 29, 2024 18:36
github-actions bot pushed a commit that referenced this pull request Mar 1, 2024
# [0.9.0](v0.8.0...v0.9.0) (2024-3-1)

### Bug Fixes

* add a limit of number of bytes while scale decoding a slice ([#3733](#3733)) ([5edbf89](5edbf89))
* **docs:** Fixing link to polkadot runtime fundamentals to the right one ([#3763](#3763)) ([a785d32](a785d32))
* don't panic if we fail to convert hex to bytes ([#3734](#3734)) ([12234de](12234de))
* **dot/sync:** execute p2p handshake when there is no target ([#3695](#3695)) ([a9db0ec](a9db0ec))
* fix index out of range undeterministic error in rpc test ([#3718](#3718)) ([d099384](d099384))
* fix non deterministic  panic during TestStableNetworkRPC integration test ([#3756](#3756)) ([ee3d243](ee3d243))
* **lib/trie:** use `MustBeHashed` for V1 trie nodes with larger storage values ([#3739](#3739)) ([f5e48a9](f5e48a9))
* **mocks:** Set fixed version for uber mockgen in CI ([#3656](#3656)) ([ea9877e](ea9877e))
* **runtime/storage:** support nested storage transactions ([#3670](#3670)) ([3e99f6d](3e99f6d))
* segfault on node restart ([#3736](#3736)) ([d1ca7aa](d1ca7aa))
* **state-version:** should be uint8 instead of uint32 ([#3779](#3779)) ([c8fdb14](c8fdb14))
* update paseo chain spec ([#3770](#3770)) ([6a54f28](6a54f28))
* use last finalized block on startup ([#3737](#3737)) ([c262642](c262642))

### Features

* **config:** dynamically set version based on environment ([#3693](#3693)) ([5c534c9](5c534c9))
* **staging:** Expose RPC on Westend Staging Node ([#3687](#3687)) ([c374eaa](c374eaa))
* **tests/scripts:** create script to retrieve trie state via rpc ([#3714](#3714)) ([5ccea40](5ccea40))
Copy link

github-actions bot commented Mar 1, 2024

🎉 This PR is included in version 0.9.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

timwu20 pushed a commit that referenced this pull request Apr 19, 2024
# [0.9.0](v0.8.0...v0.9.0) (2024-3-1)

### Bug Fixes

* add a limit of number of bytes while scale decoding a slice ([#3733](#3733)) ([5edbf89](5edbf89))
* **docs:** Fixing link to polkadot runtime fundamentals to the right one ([#3763](#3763)) ([a785d32](a785d32))
* don't panic if we fail to convert hex to bytes ([#3734](#3734)) ([12234de](12234de))
* **dot/sync:** execute p2p handshake when there is no target ([#3695](#3695)) ([a9db0ec](a9db0ec))
* fix index out of range undeterministic error in rpc test ([#3718](#3718)) ([d099384](d099384))
* fix non deterministic  panic during TestStableNetworkRPC integration test ([#3756](#3756)) ([ee3d243](ee3d243))
* **lib/trie:** use `MustBeHashed` for V1 trie nodes with larger storage values ([#3739](#3739)) ([f5e48a9](f5e48a9))
* **mocks:** Set fixed version for uber mockgen in CI ([#3656](#3656)) ([ea9877e](ea9877e))
* **runtime/storage:** support nested storage transactions ([#3670](#3670)) ([3e99f6d](3e99f6d))
* segfault on node restart ([#3736](#3736)) ([d1ca7aa](d1ca7aa))
* **state-version:** should be uint8 instead of uint32 ([#3779](#3779)) ([c8fdb14](c8fdb14))
* update paseo chain spec ([#3770](#3770)) ([6a54f28](6a54f28))
* use last finalized block on startup ([#3737](#3737)) ([c262642](c262642))

### Features

* **config:** dynamically set version based on environment ([#3693](#3693)) ([5c534c9](5c534c9))
* **staging:** Expose RPC on Westend Staging Node ([#3687](#3687)) ([c374eaa](c374eaa))
* **tests/scripts:** create script to retrieve trie state via rpc ([#3714](#3714)) ([5ccea40](5ccea40))
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
S-sync-westend related to particular network syncing. S-tests issue related to adding new tests. T-feat this issue/pr is a new feature or functionality.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants