Creating test filler

If you want to add a test to existing Ethereum test repository (https://github.com/ethereum/tests/) be aware that a lot of the tests in there are created from test fillers. This means that actual test file would be overwritten with next core update and all test in it would be refilled. This means that you have to get latest ethereum-cpp develop testeth to fill the tests manually.

To check how your test is working use testeth (https://github.com/ethereum/tests/wiki/Using-Testeth) with --filltests --checkstate option. Define your test suite with -t <TestSuite>/<TestCase> where is the name of the filler file without 'Filler' word at the end.

So, you have to add test to the test filler file that contains instructions for your test. First you have to decide to which section (TestSuite) your test belongs to:

StateTests - if you want to test a single or multiple contract interaction, or how the transaction is executed, you should add a StateTest (https://github.com/ethereum/tests/wiki/State-tests). State test fillers are located at (https://github.com/ethereum/libethereum/tree/develop/test/libethereum/StateTestsFiller). You could choose a filler that corresponds to the topic of your test and add your test in there, than make a PR. StateTest define pre-state with contracts and it's code, a transaction and tests how that single transaction is executed on a pre-state. (note that single transaction could call the contract that would call another contract and so on...)

BlockchainTests - the most complex test suite (https://github.com/ethereum/tests/wiki/Blockchain-Tests). It's fillers located at (https://github.com/ethereum/libethereum/tree/develop/test/libethereum/BlockchainTestsFiller). The global idea is that you could set genesis block, genesis pre state, and mina a couple of blocks on top of it. With blockchain tests you could test all values of the blockheaders, you could create and mine blocks with transactions and uncles, and test how this blocks are added into blockchain. The expect section is also present in this tests. Read the wiki page for detailed explanation on how to create blockchain test filler.

TransactionTests - a small test suite just to test various combinations of transaction parameters. Like to check if transaction with given nonce, value, hashes would produce a valid RLP or not. Test fillers are located here (https://github.com/ethereum/libethereum/tree/develop/test/libethereum/TransactionTestsFiller). See the wiki (https://github.com/ethereum/tests/wiki/Transaction-Tests)

RLP Tests - this tests does not have fillers. feel free to add it directly to the test repo (https://github.com/ethereum/tests/tree/develop/RLPTests). But be sure to check it's correctness with the testeth before doing that. RLP tests are for testing the RLP protocol and it's encoding/decoding function. Wiki here (https://github.com/ethereum/tests/wiki/RLP-Tests)

VM Tests - currently not supported.

Note that some test scenarios could not be filled with fillers. For example Transaction tests has test with invalid RLP of a transaction that are forged manually (https://github.com/ethereum/tests/blob/develop/TransactionTests/ttWrongRLPTransaction.json). This file could not be refilled with testeth. Same with bcInvalidRLPTest (https://github.com/ethereum/tests/blob/develop/BlockchainTests/bcInvalidRLPTest.json).

I hope this wiki answers some questions on how to add a new test to the Ethereum repo. If you have any more questions join skype #DEV Tests channel and feel free to ask.