Hardhat fork error in v2.21.0 - "missing trie node"

[DeFiFoFum]

Version of Hardhat

v2.21.0

What happened?

Issue

When I upgrade to v2.21.0 and then try to use the forking configuration below, I get the error: ProviderError: The response reported error -32000: missing trie node. (optional data: None).

Fork Configuration

I use a helper function called setupFork which follows the forking documentation in the docs: https://hardhat.org/hardhat-network/docs/guides/forking-other-networks#pinning-a-block

Here is an example: https://github.com/DeFiFoFum/hardhat-template/blob/main/lib/evm/forkHelper.ts#L39

  await network.provider.request({
    method: 'hardhat_reset',
    params: [
      {
        forking: {
          jsonRpcUrl: forkingNetworkConfig.url,
          blockNumber: blockNumber,
        },
      },
    ],
  })

Minimal reproduction steps

Workaround: use hardhat@2.14.0

To fix this issue, I downgraded to hardhat@2.14.0 and pinned that version. Now the fork configuration works as expected.

Reproduce

Install hardhat@2.21.0 and then run the setupFork function I provided above.

Search terms

trie node 2.21.0

[ChristopherDedominici]

Hi @DeFiFoFum, which service/node are you connecting to?

[RealJohnnyTime]

Also got the same issue while trying forking with Fantom / Avalanche chains

[Wodann]

Related to #387

[Wodann]

@DeFiFoFum and @RealJohnnyTime which node provider are you using when this problem occurs? Infura? Alchemy? Something else?

[DeFiFoFum]

Hi @DeFiFoFum, which service/node are you connecting to?

@DeFiFoFum and @RealJohnnyTime which node provider are you using when this problem occurs? Infura? Alchemy? Something else?

I generally go down the list on ChainList.org until I find an RPC node which can support archive requests. I also work on multiple networks such as Ethereum, Binance Smart Chain, Polygon, Arbitrum and Linea primarily. Off the top of my head I don't remember exactly which ones were causing the issue, I just know that my workflow started failing after I upgraded above 2.14.0 on CICD and I couldn't get it working again without reverting back to 2.14.0 using the same RPC endpoint. 🤔

[Wodann]

Based on our investigation, this problem is an internal server error originating from archive providers. To minimise the end-user effect, we've introduced an application-level retry specifically for when the "missing trie node" JSON-RPC error is returned.

We only retry once, so there is still a slim change that the internal error occurs consecutively and is returned to the user.

We'll release a patch release including this change as soon as possible.

Note: Once this issue is resolved on the archive provider's side, we'll remove the application-level retry.

[agostbiro]

We've released EDR v0.3.8 with a fix for this issue.

To upgrade to the latest EDR version immediately, you can remove your node_modules directory and npm/yarn/pnpm lock file and reinstall your dependencies. Otherwise you can wait for the next Hardhat release which will automatically use the latest EDR version.

You can verify that you’re using the latest version of EDR by running the following command: npm ls @nomicfoundation/edr.

[wade-liwei]

@agostbiro

wuban@wuban:~/hardhat$ npx hardhat node --hostname 0.0.0.0 --fork https://base-rpc.publicnode.com

Error HH604: Error running JSON-RPC server: The response reported error -32000: missing trie node 767f8b54c954826780d412c6aa03603b70b3f1a9ed5cca2a14610377e1d0fac2 (path ) state 0x767f8b54c954826780d412c6aa03603b70b3f1a9ed5cca2a14610377e1d0fac2 is not available. (optional data: None). Request: {"id":5,"jsonrpc":"2.0","method":"eth_getCode","params":["0x3c44cdddb6a900fa2b585dd299e03d12fa4293bc","0xdf1c74"]}

HardhatError: HH604: Error running JSON-RPC server: The response reported error -32000: missing trie node 767f8b54c954826780d412c6aa03603b70b3f1a9ed5cca2a14610377e1d0fac2 (path ) state 0x767f8b54c954826780d412c6aa03603b70b3f1a9ed5cca2a14610377e1d0fac2 is not available. (optional data: None). Request: {"id":5,"jsonrpc":"2.0","method":"eth_getCode","params":["0x3c44cdddb6a900fa2b585dd299e03d12fa4293bc","0xdf1c74"]}
at SimpleTaskDefinition.action (/home/wuban/hardhat/node_modules/hardhat/src/builtin-tasks/node.ts:372:17)
at Environment._runTaskDefinition (/home/wuban/hardhat/node_modules/hardhat/src/internal/core/runtime-environment.ts:359:14)
at Environment.run (/home/wuban/hardhat/node_modules/hardhat/src/internal/core/runtime-environment.ts:192:14)
at main (/home/wuban/hardhat/node_modules/hardhat/src/internal/cli/cli.ts:323:7)

Caused by: ProviderError: The response reported error `-32000`: `missing trie node 767f8b54c954826780d412c6aa03603b70b3f1a9ed5cca2a14610377e1d0fac2 (path ) state 0x767f8b54c954826780d412c6aa03603b70b3f1a9ed5cca2a14610377e1d0fac2 is not available`. (optional data: None). Request: {"id":5,"jsonrpc":"2.0","method":"eth_getCode","params":["0x3c44cdddb6a900fa2b585dd299e03d12fa4293bc","0xdf1c74"]}
    at EdrProviderWrapper.request (/home/wuban/hardhat/node_modules/hardhat/src/internal/hardhat-network/provider/provider.ts:438:19)
    at SimpleTaskDefinition.action (/home/wuban/hardhat/node_modules/hardhat/src/builtin-tasks/node.ts:146:9)
    at Environment._runTaskDefinition (/home/wuban/hardhat/node_modules/hardhat/src/internal/core/runtime-environment.ts:359:14)
    at Environment.run (/home/wuban/hardhat/node_modules/hardhat/src/internal/core/runtime-environment.ts:192:14)
    at SimpleTaskDefinition.action (/home/wuban/hardhat/node_modules/hardhat/src/builtin-tasks/node.ts:303:44)
    at Environment._runTaskDefinition (/home/wuban/hardhat/node_modules/hardhat/src/internal/core/runtime-environment.ts:359:14)
    at Environment.run (/home/wuban/hardhat/node_modules/hardhat/src/internal/core/runtime-environment.ts:192:14)
    at main (/home/wuban/hardhat/node_modules/hardhat/src/internal/cli/cli.ts:323:7)

wuban@wuban:/hardhat$
wuban@wuban:/hardhat$
wuban@wuban:/hardhat$
wuban@wuban:/hardhat$
wuban@wuban:~/hardhat$ npm ls @nomicfoundation/edr
hardhat-project@ /home/wuban/hardhat
└─┬ hardhat@2.22.4
└── @nomicfoundation/edr@0.3.8

[fvictorio]

@wade-liwei https://base-rpc.publicnode.com/ is not an archive node

[matthiasgeihs]

@wade-liwei https://base-rpc.publicnode.com/ is not an archive node

Is forking only supposed to work with archive nodes? Getting the same error when trying to fork https://rpc.sepolia.org/ :(

     Error: The response reported error `-32000`: `missing trie node a1c93ae3e858256e406eea8a153f8ffd5227b1943b250aa255d8c53c2502a3f8 (path ) state 0xa1c93ae3e858256e406eea8a153f8ffd5227b1943b250aa255d8c53c2502a3f8 is not available, not found`. (optional data: None). Request: [{"id":4,"jsonrpc":"2.0","method":"eth_getBalance","params":["0x3c44cdddb6a900fa2b585dd299e03d12fa4293bc","0x60ffb7"]},{"id":5,"jsonrpc":"2.0","method":"eth_getTransactionCount","params":["0x3c44cdddb6a900fa2b585dd299e03d12fa4293bc","0x60ffb7"]},{"id":6,"jsonrpc":"2.0","method":"eth_getCode","params":["0x3c44cdddb6a900fa2b585dd299e03d12fa4293bc","0x60ffb7"]}]

[SwapnilSoni1999]

@wade-liwei https://base-rpc.publicnode.com/ is not an archive node

Is forking only supposed to work with archive nodes? Getting the same error when trying to fork https://rpc.sepolia.org/ :(

     Error: The response reported error `-32000`: `missing trie node a1c93ae3e858256e406eea8a153f8ffd5227b1943b250aa255d8c53c2502a3f8 (path ) state 0xa1c93ae3e858256e406eea8a153f8ffd5227b1943b250aa255d8c53c2502a3f8 is not available, not found`. (optional data: None). Request: [{"id":4,"jsonrpc":"2.0","method":"eth_getBalance","params":["0x3c44cdddb6a900fa2b585dd299e03d12fa4293bc","0x60ffb7"]},{"id":5,"jsonrpc":"2.0","method":"eth_getTransactionCount","params":["0x3c44cdddb6a900fa2b585dd299e03d12fa4293bc","0x60ffb7"]},{"id":6,"jsonrpc":"2.0","method":"eth_getCode","params":["0x3c44cdddb6a900fa2b585dd299e03d12fa4293bc","0x60ffb7"]}]

got any solution?

[fvictorio]

You can fork a non-archive node, but it will eventually stop working.

The way forking works is that you start forking at some block N and use that block to get the state you need to simulate transactions locally.

Eventually, though, new blocks will be mined, N will get old, and the node will start rejecting queries about the state at that node.

You could argue that in that case we should always fetch the state from the latest block, but that would lead to a lot of weird, wrong behaviors, because you'd be using a world state that is continually changing. For most use cases, this doesn't make sense.