perf: Make kura drop old blocks from memory #5103
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dima74
requested review from
s8sato,
mversic,
SamHSmith,
outoftardis,
Erigara,
0x009922 and
DCNick3
as code owners
dima74
force-pushed
the
diralik/kura-drop-old-blocks
branch
from
3aaecd1 to
bcd75a9
Compare
Erigara
approved these changes
Sep 27, 2024
SamHSmith
force-pushed
the
diralik/kura-drop-old-blocks
branch
from
bcd75a9 to
2b5c07a
Compare
|
I broke DCO by pushing the rebase button. Please rebase locally and then force push so that we can merge. |
Signed-off-by: Dmitry Murzin <diralik@yandex.ru>
dima74
force-pushed
the
diralik/kura-drop-old-blocks
branch
from
2b5c07a to
492fd5c
Compare
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
Meta: #5083
Fixes #4954
Solution
Currently kura stores all blocks in memory, which results in memory usage approximately 1.6KB per transaction, or 1.6GB per 1 million transactions. This PR changes to store only N last blocks in memory (128 by default). Other blocks will be loaded from the disk if needed. This improves memory usage to approximately 280 bytes per transaction or 0.26GB per 1 million transactions. Note that memory usage is still unbounded because of
State::transactionsmap.Checklist
CONTRIBUTING.md.