Merge remote-tracking branch 'origin/main' into 53-iterate

.github/workflows/tests.yml

@@ -160,6 +160,7 @@ jobs:
           install cargo-mutants ~/.cargo/bin/
       - name: Mutants in-diff
         # Normally this would have --in-place, but for the sake of exercising more cases, it does not.
+        # TODO: Pass --profile=mutants when supported
         run: >
           cargo mutants --no-shuffle -vV --in-diff git.diff --test-tool=${{matrix.test_tool}} --timeout=500 --build-timeout=500
       - name: Archive mutants.out
@@ -200,6 +201,7 @@ jobs:
           cargo mutants --no-shuffle -vV --shard ${{ matrix.shard }}/10
           --test-tool ${{ matrix.test_tool }} --baseline=skip --timeout=500
           --build-timeout=500 --in-place
+          --cargo-arg=--profile=mutants
       - name: Archive mutants.out
         uses: actions/upload-artifact@v4
         if: always()

Cargo.toml

@@ -48,7 +48,6 @@ regex = "1.10"
 serde_json = "1.0.117"
 similar = "2.1"
 strum = { version = "0.26", features = ["derive"] }
-subprocess = "0.2.8"
 tempfile = "3.4"
 time = "0.3"
 toml = "0.8"
@@ -158,6 +157,10 @@ pr-run-mode = "plan"
 inherits = "release"
 lto = "thin"
 
+[profile.mutants]
+inherits = "test"
+debug = "none"
+
 # Config for <https://github.com/crate-ci/cargo-release/blob/master/docs/reference.md>
 [workspace.metadata.release]
 pre-release-replacements = [

NEWS.md

@@ -6,6 +6,12 @@
 
 - Fixed: The auto-set timeout for building mutants is now 2 times the baseline build time times the number of jobs, with a minimum of 20 seconds. This was changed because builds of mutants contend with each other for access to CPUs and may be slower than the baseline build.
 
+- Changed: No build timeouts by default. Previously, cargo-mutants set a default build timeout based on the baseline build, but experience showed that this would sometimes make builds flaky, because build times can be quite variable. If mutants cause builds to hang, then you can still set a timeout using `--build-timeout` or `--build-timeout-multiplier`.
+
+- Fixed: Don't error if the `--in-diff` file is empty.
+
+- Changed: cargo-mutants no longer passes `--cap-lints=allow` to rustc. This was previously done so that mutants would not unnecessarily be unviable due to triggering compiler warnings in trees configured to deny some lints, but it had the undesirable effect of disabling rustc's detection of long running const evaluations. If your tree treats some lints as errors then the previous behavior can be restored with `--cap-lints=true` (or the equivalent config option), or you can use `cfg_attr` and a feature flag  to accept those warnings when testing under cargo-mutants.
+
 ## 24.5.0
 
 - Fixed: Follow `path` attributes on `mod` statements.

book/src/SUMMARY.md

@@ -21,6 +21,7 @@
   - [Baseline tests](baseline.md)
   - [Testing in-place](in-place.md)
   - [Iterating on missed mutants](iterate.md)
+  - [Strict lints](lints.md)
 - [Generating mutants](mutants.md)
   - [Error values](error-values.md)
 - [Improving performance](performance.md)

book/src/lints.md

@@ -0,0 +1,12 @@
+# Strict lints
+
+Because cargo-mutants builds versions of your tree with many heuristically injected errors, it may not work well in trees that are configured to treat warnings as errors.
+
+For example, mutants that delete code are likely to cause some parameters to be seen as unused, which will cause problems with trees that configure `#[deny(unused)]`. This will manifest as an excessive number of mutants being reported as "unviable".
+
+There are a few possible solutions:
+
+1. Define a feature flag for mutation testing, and use `cfg_attr` to enable strict warnings only when not testing mutants.
+2. Use the `cargo mutants --cap-lints=true` command line option, or the `cap_lints = true` config option.
+
+`--cap_lints=true` also disables rustc's detection of long-running const expression evaluation, so may cause some builds to fail. If that happens in your tree, you can set a [build timeout](timeouts.md).

book/src/timeouts.md

@@ -22,32 +22,31 @@ continue to the next mutant.
 By default, the timeouts are set automatically, relative to the times taken to
 build and test the unmodified tree (baseline).
 
-The default timeouts are:
-
-- `cargo build`/`cargo check`: 2 times the baseline build time times the number of jobs (with a minimum of 20 seconds)
-- `cargo test`: 5 times baseline test time (with a minimum of 20 seconds)
-
-The build timeout scales with the number of jobs to reflect that cargo often spawns many jobs, and so builds run in parallel are likely to take longer than the baseline, which has no external parallelism.
+The default test timeout is 5 times the baseline test time, with a minimum of 20 seconds.
 
 The minimum of 20 seconds for the test timeout can be overridden by the
 `--minimum-test-timeout` option or the `CARGO_MUTANTS_MINIMUM_TEST_TIMEOUT`
 environment variable, measured in seconds.
 
-You can set explicit timeouts with the `--build-timeout`, and `--timeout`
-options, also measured in seconds. If these options are specified then they
-are applied to the baseline build and test as well.
+You can set an explicit timeouts with the `--timeout` option, also measured in seconds.
+
+You can also set the test timeout as a multiple of the duration of the baseline test, with the `--timeout-multiplier` option and the `timeout_multiplier` configuration key.
+The multiplier only has an effect if the baseline is not skipped and if `--timeout` is not specified.
+
+## Build timeouts
+
+`const` expressions may be evaluated at compile time. In the same way that mutations can cause tests to hang, mutations to const code may potentially cause the compiler to enter an infinite loop.
+
+rustc imposes a time limit on evaluation of const expressions. This is controlled by the `long_running_const_eval` lint, which by default will interrupt compilation: as a result the mutants will be seen as unviable.
+
+If this lint is configured off in your program, or if you use the `--cap-lints=true` option to turn off all lints, then the compiler may hang when constant expressions are mutated.
+
+In this case you can use the `--build-timeout` or `--build-timeout-multiplier` options, or their corresponding configuration keys, to impose a limit on overall build time. However, because build time can be quite variable there's some risk of this causing builds to be flaky, and so it's off by default.
 
-You can set timeout multipliers that are relative to the duration of the
-baseline build or test with `--build-timeout-multiplier` and
-`--timeout-multiplier`, respectively.  Additionally, these can be configured
-with `build_timeout_multiplier` and `timeout_multiplier` in
-`.cargo/mutants.toml` (e.g. `timeout-multiplier = 1.5`).  These options are only
-applied if the baseline is not skipped and no corresponding
-`--build-timeout`/`--timeout` option is specified, otherwise they are ignored.
+You might also choose to skip mutants that can cause long-running const evaluation.
 
 ## Exceptions
 
 The multiplier timeout options cannot be used when the baseline is skipped
 (`--baseline=skip`), or when the build is in-place (`--in-place`). If no
-explicit timeouts is provided in these cases, then a default of 300 seconds
-will be used.
+explicit timeouts is provided in these cases, then there is no build timeout and the test timeout default of 300 seconds will be used.