Simulation Testing

Testing is not about proving a system is correct. It’s a search problem; the search for errors. We look for paths through state space that result in errors. Unit testing explores a tiny subset of possible pathways. Even 100% unit test coverage doesn’t guarantee 100% state space coverage. Scripted automated tests are only as rigorous as your most devious testers’ imaginations.

We can improve our results with controlled randomness. We simulate inputs to the system under test, using randomness to broaden our search of the state space. At the same time, we control the randomness to ensure our tests are repeatable and that we can verify when bugs are fixed.

This talk will introduce the structure of simulation testing using the open-source framework Simulant. We will walk through a complete simulation lifecycle with source code.