forge test --rerun should rerun failing fuzz test input

[tynes]

Component

Forge

Describe the feature you would like

I am currently experiencing a situation in which forge test --rerun does indeed rerun a failed test but the failed test is a fuzz test and it is not always capturing the failure in the fuzz test. Ideally the cached output for the failure should include the fuzz inputs that triggered the failure and those are specifically used as inputs during the call with --rerun so that the failure trace can be inspected with -vvv

Additional context

No response

[tynes]

Here is an example of this happening in CI: https://app.circleci.com/pipelines/github/ethereum-optimism/optimism/60845/workflows/9d3cf6ec-3be1-4da1-98f8-30b786def971/jobs/2551903

[grandizzy]

@tynes does your CI use same cache dir for subsequent runs? The saved fuzz failure counterexample needs to be available in cache dir when rerun, pls see

https://book.getfoundry.sh/forge/replay-testing#fuzz-tests-failures

you can locally check this functionality by using a test contract like

contract ContractTest is Test {
    function testA() public {
        require(true);
    }

    function testB() public {
        require(true);
    }

    function testC(uint256 a) public {
        require(a != 0);
    }
}

on the first forge test --rerun will show failure caught after 31 runs

[PASS] testA() (gas: 187)
[PASS] testB() (gas: 165)
[FAIL. Reason: EvmError: Revert; counterexample: calldata=0xf280cf450000000000000000000000000000000000000000000000000000000000000000 args=[0]] testC(uint256) (runs: 31, μ: 312, ~: 312)

while the next one runs only the failed test with failed counterexample (runs is 0)

Ran 1 test for test/ContractTest.sol:ContractTest
[FAIL. Reason: EvmError: Revert; counterexample: calldata=0xf280cf450000000000000000000000000000000000000000000000000000000000000000 args=[0]] testC(uint256) (runs: 0, μ: 0, ~: 0)

[mds1]

@tynes Going to close this issue since I believe --rerun was working properly and what we really saw is #8639

[tynes]

Thank you @grandizzy for the help!