Regression Testing and Fingerprints Tutorial¶
During the development of simulation models, it is important to detect regressions, i.e. to notice that a change in a correctly working model led to incorrect behavior.
Regressions can be detected by manually examining simulations in Qtenv. However, some behavioral changes might be subtle and go unnoticed, but still would invalidate the model. Regression testing automates this process, so the model doesn’t have to be manually checked as thoroughly and as often. It might also detect regressions that would have been missed during manual examination.
Fingerprint testing is a useful and low-cost tool that can be used for this purpose. A fingerprint is a hash value, which is calculated during a simulation run and is characteristic of the simulation’s trajectory. After the change in the model, a change in the fingerprint can indicate that the model no longer works along the same trajectory. This may or may not mean that the model is incorrect. When the fingerprints stay the same, the model can be assumed to be still working correctly (thus protecting against regressions). Fingerprints can be calculated in different ways, using so called ‘ingredients’ that specify which aspects of the simulation to be taken into account when calculating a hash value.
The steps in this tutorial contain examples for typical changes in the model during development, and describe the model validation workflow using fingerprint tests.
Information about OMNeT++’s fingerprint support can be found in the corresponding section of the OMNeT++ Simulation Manual. Also, the Testing section in the INET Developer’s Guide describes regression testing.
The tutorial contains the following steps: