A simple benchmark comparing two popular automatic test generation tools
- To generate all tests, use "gradle generateTests"
- To run all generated tests, use "gradle runAllTests"
- To create coverage reports for the generated tests, use "gradle allCoverages"
- To create mutation reports for the generated tests, use "gradle allMutations"
- To create a csv file with coverage information for the tests, use "gradle exportCSV"
- To clean up generated files and tests, use "gradle cleanUp"
Please note that some of these tasks depend on a previous task. i.e: generateTests must be called before runAllTests
Several convenience methods are available and can be listed by using "gradle tasks"
Available parameters:
- Time spent creating tests: -PbudgetTime=X
- Amount of test suites to generate for each class: -PtestSuites=X
Generate a csv with information for 10 tests suites, using 2 minutes as a budget
gradle generateTests runAllTests allCoverages allMutations exportCSV -PbudgetTime=120 -PtestSuites=10
Generate and measure only randoop tests
gradle randoop runRandoopTests randoopCoverage randoopMutation exportCSV
class name, tool name, line coverage, branch coverage, mutation score