diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index 7b37b414..92d96035 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -20,6 +20,7 @@ - [Using nextest](nextest.md) - [Baseline tests](baseline.md) - [Testing in-place](in-place.md) + - [Iterating on missed mutants](iterate.md) - [Generating mutants](mutants.md) - [Error values](error-values.md) - [Improving performance](performance.md) diff --git a/book/src/iterate.md b/book/src/iterate.md new file mode 100644 index 00000000..e258cb61 --- /dev/null +++ b/book/src/iterate.md @@ -0,0 +1,31 @@ +# Iterating on missed mutants + +When you're working to improve test coverage in a tree, you might use a process like this: + +1. Run `cargo-mutants` to find code that's untested, possibly filtering to some selected files. + +2. Think about why some mutants are missed, and then write tests that will catch them. + +3. Run cargo-mutants again to learn whether your tests caught all the mutants, or if any remain. + +4. Repeat until everything is caught. + +You can speed up this process by using the `--iterate` option. This tells cargo-mutants to skip mutants that were either caught or unviable in a previous run, and to accumulate the results. + +You can run repeatedly with `--iterate`, adding tests each time, until all the missed mutants are caught (or skipped.) + +## How it works + +When `--iterate` is given, cargo-mutants reads `mutants.out/caught.txt`, `previously_caught.txt`, and `unviable.txt` before renaming that directory to `mutants.out.old`. If those files don't exist, the lists are assumed to be empty. + +Mutants are then tested as usual, but excluding all the mutants named in those files. `--list --iterate` also applies this exclusion and shows you which mutants will be tested. + +Mutants are matched based on their file name, line, column, and description, just as shown in `--list` and in those files. As a result, if you insert or move text in a source file, some mutants may be re-tested. + +After testing, all the previously caught, caught, and unviable are written into `previously_caught.txt` so that they'll be excluded on future runs. + +`previously_caught.txt` is only written when `--iterate` is given. + +## Caution + +`--iterate` is a heuristic, and makes the assumption that any new changes you make won't reduce coverage, which might not be true. After you think you've caught all the mutants, you should run again without `--iterate` to make sure. diff --git a/book/src/mutants-out.md b/book/src/mutants-out.md index b58c2f71..32450ffe 100644 --- a/book/src/mutants-out.md +++ b/book/src/mutants-out.md @@ -25,8 +25,10 @@ The output directory contains: * `caught.txt`, `missed.txt`, `timeout.txt`, `unviable.txt`, each listing mutants with the corresponding outcome. +* `previously_caught.txt` accumulates a list of mutants caught in previous runs with [`--iterate`](iterate.md). + The contents of the directory and the format of these files is subject to change in future versions. These files are incrementally updated while cargo-mutants runs, so other programs can read them to follow progress. -There is generally no reason to include this directory in version control, so it is recommended that you add `/mutants.out*` to your `.gitignore` file. This will exclude both `mutants.out` and `mutants.out.old`. \ No newline at end of file +There is generally no reason to include this directory in version control, so it is recommended that you add `/mutants.out*` to your `.gitignore` file. This will exclude both `mutants.out` and `mutants.out.old`.