core: respect history cutoff in txindexer #31393
Merged
+541
−200
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rjl493456442
force-pushed
the
txlookup-cutoff
branch
from
05624a8 to
85b6921
Compare
rjl493456442
force-pushed
the
txlookup-cutoff
branch
from
85b6921 to
e9e5ab1
Compare
fjl
approved these changes
Mar 21, 2025
sivaratrisrinivas
pushed a commit
to sivaratrisrinivas/go-ethereum
that referenced
this pull request
Apr 21, 2025
In ethereum#31384 we unindex TXes prior to the merge block. However when the node starts up it will try to re-index those back if the config is to index the whole chain. This change makes the indexer aware of the history cutoff block, avoiding reindexing in that segment. --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com> Co-authored-by: Felix Lange <fjl@twurst.com>
rjl493456442
added a commit
that referenced
this pull request
May 5, 2025
Fixes #31732. This logic was removed in the recent refactoring in the txindexer to handle history cutoff (#31393). It was first introduced in this PR: #28908. I have tested it and it works as an alternative to #31745. This PR packs 3 changes to the flow of fetching txs from the API: - It caches the indexer tail after each run is over to avoid hitting the db all the time as was done originally in #28908. - Changes `backend.GetTransaction`. It doesn't return an error anymore when tx indexer is in progress. It shifts the responsibility to the caller to check the progress. The reason is that in most cases we anyway check the txpool for the tx. If it was indeed a pending tx we can avoid the indexer progress check. --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com>
0g-wh
pushed a commit
to 0g-wh/0g-geth
that referenced
this pull request
May 8, 2025
Fixes ethereum#31732. This logic was removed in the recent refactoring in the txindexer to handle history cutoff (ethereum#31393). It was first introduced in this PR: ethereum#28908. I have tested it and it works as an alternative to ethereum#31745. This PR packs 3 changes to the flow of fetching txs from the API: - It caches the indexer tail after each run is over to avoid hitting the db all the time as was done originally in ethereum#28908. - Changes `backend.GetTransaction`. It doesn't return an error anymore when tx indexer is in progress. It shifts the responsibility to the caller to check the progress. The reason is that in most cases we anyway check the txpool for the tx. If it was indeed a pending tx we can avoid the indexer progress check. --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Rampex1
pushed a commit
to streamingfast/go-ethereum
that referenced
this pull request
May 15, 2025
In ethereum#31384 we unindex TXes prior to the merge block. However when the node starts up it will try to re-index those back if the config is to index the whole chain. This change makes the indexer aware of the history cutoff block, avoiding reindexing in that segment. --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com> Co-authored-by: Felix Lange <fjl@twurst.com>
| func (indexer *txIndexer) report(head uint64) TxIndexProgress { | ||
| // Special case if the head is even below the cutoff, | ||
| // nothing to index. | ||
| if head < indexer.cutoff { |
There was a problem hiding this comment.
Should we check for head == 0 too here because if someone calls some json rpc api methods of getting receipt on an empty blockchain, they will get error txn indexing is in progress which is a bit misleading because there is no indexing on genesis block @s1na
There was a problem hiding this comment.
Have you seen this error in practice?
There was a problem hiding this comment.
Umm yes, I have observed it when writing unit tests with a chain that starts from genesis. When I commit the first transaction to the chain, and wait for its receipt, I keep on getting error txn indexing is in progress and it keeps on going until the first block is committed. After that getting transaction receipt is smooth and fast
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In the #31384 we unindex txes prior to the merge block. However when the node starts up it will try to re-index those back if the config is to index the whole chain.