In this paper, the authors describe their experience with automatically testing Java virtual machines and describe two specific techniques for generating test cases.
In this paper, the authors describe their experience with automatically testing Java virtual machines. They outline two automatic testing techniques for JVMs.
The first technique is comparative evaluation with mutations, where randomly perturbed test inputs are used to identify discrepancies between different versions of JVMs. They show that this fast and effective technique achieves broad code coverage and discuss its shortcomings.
Second, they present a well-structured technique for generating complex test cases from cogent grammar descriptions. They describe lava, a special purpose language we developed to specify production grammars for testing and show that grammar-based test generation can produce very complex test cases from compact specifications.
This testing process is easy to steer and can generate targeted test cases. Most importantly, grammars enable the tester to reason about the expected system behavior on the test inputs.
