--colors and CARGO_TERM_COLORS support

NEWS.md

@@ -1,6 +1,9 @@
 # cargo-mutants changelog
 
 ## Unreleased
+
+- New: Colored output can be enabled in CI or other noninteractive situations by passing `--colors=always`, or setting `CARGO_TERM_COLOR=always`, or `CLICOLOR_FORCE=1`. Colors can similarly be forced off with `--colors=never`, `CARGO_TERM_COLOR=never`, or `NO_COLOR=1`.
+
 ## 24.1.2
 
 - New: `--in-place` option tests mutations in the original source tree, without copying the tree. This is faster and uses less disk space, but it's incompatible with `--jobs`, and you must be careful not to edit or commit the source tree while tests are running.

book/src/SUMMARY.md

@@ -12,6 +12,7 @@
   - [Filtering files](skip_files.md)
   - [Filtering functions and mutants](filter_mutants.md)
 - [Controlling cargo-mutants](controlling.md)
+  - [Display and output](output.md)
   - [Listing and previewing mutations](list.md)
   - [Workspaces and packages](workspaces.md)
   - [Passing options to Cargo](cargo-args.md)

book/src/ci.md

@@ -23,6 +23,9 @@ Here is an example of a GitHub Actions workflow that runs mutation tests and upl
 ```yml
 name: cargo-mutants
 
+env:
+  CARGO_TERM_COLOR: always
+
 on:
   push:
     branches:

book/src/controlling.md

@@ -26,14 +26,4 @@ the source.
 `-d`, `--dir`: Test the Rust tree in the given directory, rather than the source tree
 enclosing the working directory where cargo-mutants is launched.
 
-## Console output
-
-`-v`, `--caught`: Also print mutants that were caught by tests.
-
-`-V`, `--unviable`: Also print mutants that failed `cargo build`.
-
-`--no-times`: Don't print elapsed times.
-
-`-L`, `--level`, and `$CARGO_MUTANTS_TRACE_LEVEL`: set the verbosity of trace
-output to stdout. The default is `info`, and it can be increased to `debug` or
-`trace`.
+`--manifest-path`: Also selects the tree to test, but takes a path to a Cargo.toml file rather than a directory. (This is less convenient but compatible with other Cargo commands.)

book/src/output.md

@@ -0,0 +1,23 @@
+# Display and output
+
+cargo-mutants writes a list of missed or timed-out mutants to stderr, and optionally mutants that were caught (with `--caught`) or failed to build (with `--unviable`) to stdout. It writes error or debug messages to stderr.
+
+The following options control what is printed to stdout and stderr.
+
+`-v`, `--caught`: Also print mutants that were caught by tests.
+
+`-V`, `--unviable`: Also print mutants that failed `cargo build`.
+
+`--no-times`: Don't print elapsed times. (This is intended mostly to make the output more stable for testing.)
+
+## Colors
+
+`--colors=always|never|auto`: Control whether to use colors in output. The default is `auto`, which will write colors if the output is a terminal that supports colors. Color support is detected independently for stdout and stderr, so you should still see colors on stderr if stdout is redirected.
+
+The same values can be set with the `CARGO_TERM_COLOR` environment variable, which is respected by many Cargo commands.
+
+cargo-mutants also respects the [`NO_COLOR`](https://no-color.org/) and [`CLICOLOR_FORCE`](https://bixense.com/clicolors/) environment variables. If they are set to a value other than `0` then colors will be disabled or enabled regardless of any other settings.
+
+## Debug trace
+
+`-L`, `--level`, and `$CARGO_MUTANTS_TRACE_LEVEL`: set the verbosity of trace output to stderr. The default is `info`, and it can be increased to `debug` or `trace`.

book/src/pr-diff.md

@@ -10,6 +10,9 @@ name: Tests
 permissions:
   contents: read
 
+env:
+  CARGO_TERM_COLOR: always
+
 on:
   push:
     branches:

src/console.rs

@@ -18,6 +18,7 @@ use tracing::Level;
 use tracing_subscriber::fmt::MakeWriter;
 use tracing_subscriber::prelude::*;
 
+use crate::options::Colors;
 use crate::outcome::{LabOutcome, SummaryOutcome};
 use crate::scenario::Scenario;
 use crate::tail_file::TailFile;
@@ -42,6 +43,14 @@ impl Console {
         }
     }
 
+    pub fn set_colors_enabled(&self, colors: Colors) {
+        if let Some(colors) = colors.forced_value() {
+            ::console::set_colors_enabled(colors);
+            ::console::set_colors_enabled_stderr(colors);
+        }
+        // Otherwise, let the console crate decide, based on isatty, etc.
+    }
+
     pub fn walk_tree_start(&self) {
         self.view
             .update(|model| model.walk_tree = Some(WalkModel::default()));
@@ -241,9 +250,12 @@ impl Console {
     /// Configure tracing to send messages to the console and debug log.
     ///
     /// The debug log is opened later and provided by [Console::set_debug_log].
-    pub fn setup_global_trace(&self, console_trace_level: Level) -> Result<()> {
+    pub fn setup_global_trace(&self, console_trace_level: Level, colors: Colors) -> Result<()> {
         // Show time relative to the start of the program.
         let uptime = tracing_subscriber::fmt::time::uptime();
+        let stderr_colors = colors
+            .forced_value()
+            .unwrap_or_else(::console::colors_enabled_stderr);
         let debug_log_layer = tracing_subscriber::fmt::layer()
             .with_ansi(false)
             .with_file(true) // source file name
@@ -252,7 +264,7 @@ impl Console {
             .with_writer(self.make_debug_log_writer());
         let level_filter = tracing_subscriber::filter::LevelFilter::from_level(console_trace_level);
         let console_layer = tracing_subscriber::fmt::layer()
-            .with_ansi(true)
+            .with_ansi(stderr_colors)
             .with_writer(self.make_terminal_writer())
             .with_target(false)
             .without_time()

src/list.rs

@@ -1,4 +1,4 @@
-// Copyright 2023 Martin Pool
+// Copyright 2023-2024 Martin Pool
 
 //! List mutants and files as text.
 
@@ -45,12 +45,11 @@ pub(crate) fn list_mutants<W: fmt::Write>(
         }
         out.write_str(&serde_json::to_string_pretty(&list)?)?;
     } else {
+        // TODO: Do we need to check this? Could the console library strip them if they're not
+        // supported?
+        let colors = options.colors.active_stdout();
         for mutant in mutants {
-            writeln!(
-                out,
-                "{}",
-                mutant.name(options.show_line_col, options.colors)
-            )?;
+            writeln!(out, "{}", mutant.name(options.show_line_col, colors))?;
             if options.emit_diffs {
                 writeln!(out, "{}", mutant.diff())?;
             }

src/main.rs

@@ -54,7 +54,7 @@ use crate::list::{list_files, list_mutants, FmtToIoWrite};
 use crate::log_file::LogFile;
 use crate::manifest::fix_manifest;
 use crate::mutate::{Genre, Mutant};
-use crate::options::{Options, TestTool};
+use crate::options::{Colors, Options, TestTool};
 use crate::outcome::{Phase, ScenarioOutcome};
 use crate::scenario::Scenario;
 use crate::shard::Shard;
@@ -120,6 +120,16 @@ struct Args {
     #[arg(long, help_heading = "Execution")]
     check: bool,
 
+    /// draw colors in output.
+    #[arg(
+        long,
+        value_enum,
+        help_heading = "Output",
+        default_value_t,
+        env = "CARGO_TERM_COLOR"
+    )]
+    colors: Colors,
+
     /// show the mutation diffs.
     #[arg(long, help_heading = "Filters")]
     diff: bool,
@@ -327,7 +337,8 @@ fn main() -> Result<()> {
     }
 
     let console = Console::new();
-    console.setup_global_trace(args.level)?;
+    console.setup_global_trace(args.level, args.colors)?; // We don't have Options yet.
+    console.set_colors_enabled(args.colors);
     interrupt::install_handler();
 
     let start_dir: &Utf8Path = if let Some(manifest_path) = &args.manifest_path {

src/options.rs

@@ -92,7 +92,7 @@ pub struct Options {
     pub error_values: Vec<String>,
 
     /// Show ANSI colors.
-    pub colors: bool,
+    pub colors: Colors,
 
     /// List mutants in json, etc.
     pub emit_json: bool,
@@ -125,6 +125,44 @@ fn join_slices(a: &[String], b: &[String]) -> Vec<String> {
     v
 }
 
+/// Should ANSI colors be drawn?
+#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Display, Deserialize, ValueEnum)]
+#[strum(serialize_all = "snake_case")]
+#[serde(rename_all = "snake_case")]
+pub enum Colors {
+    #[default]
+    Auto,
+    Always,
+    Never,
+}
+
+impl Colors {
+    /// If colors were forced on or off by the user through an option or
+    /// environment variable, return that value.
+    ///
+    /// Otherwise, return None, meaning we should decide based on the
+    /// detected terminal characteristics.
+    pub fn forced_value(&self) -> Option<bool> {
+        // From https://bixense.com/clicolors/
+        if env::var("NO_COLOR").map_or(false, |x| x != "0") {
+            Some(false)
+        } else if env::var("CLICOLOR_FORCE").map_or(false, |x| x != "0") {
+            Some(true)
+        } else {
+            match self {
+                Colors::Always => Some(true),
+                Colors::Never => Some(false),
+                Colors::Auto => None, // library should decide
+            }
+        }
+    }
+
+    pub fn active_stdout(&self) -> bool {
+        self.forced_value()
+            .unwrap_or_else(::console::colors_enabled)
+    }
+}
+
 impl Options {
     /// Build options by merging command-line args and config file.
     pub(crate) fn new(args: &Args, config: &Config) -> Result<Options> {
@@ -146,7 +184,7 @@ impl Options {
             ),
             baseline: args.baseline,
             check_only: args.check,
-            colors: true, // TODO: An option for this and use CLICOLORS.
+            colors: args.colors,
             emit_json: args.json,
             emit_diffs: args.diff,
             error_values: join_slices(&args.error, &config.error_values),

src/snapshots/cargo_mutants__visit__test__expected_mutants_for_own_source_tree.snap

@@ -207,6 +207,13 @@ src/options.rs: replace join_slices -> Vec<String> with vec![String::new()]
 src/options.rs: replace join_slices -> Vec<String> with vec!["xyzzy".into()]
 src/options.rs: replace + with - in join_slices
 src/options.rs: replace + with * in join_slices
+src/options.rs: replace Colors::forced_value -> Option<bool> with None
+src/options.rs: replace Colors::forced_value -> Option<bool> with Some(true)
+src/options.rs: replace Colors::forced_value -> Option<bool> with Some(false)
+src/options.rs: replace != with == in Colors::forced_value
+src/options.rs: replace != with == in Colors::forced_value
+src/options.rs: replace Colors::active_stdout -> bool with true
+src/options.rs: replace Colors::active_stdout -> bool with false
 src/options.rs: replace or_slices -> &'c[T] with Vec::leak(Vec::new())
 src/options.rs: replace or_slices -> &'c[T] with Vec::leak(vec![Default::default()])
 src/options.rs: replace build_glob_set -> Result<Option<GlobSet>> with Ok(None)

src/visit.rs

@@ -1,4 +1,4 @@
-// Copyright 2021-2023 Martin Pool
+// Copyright 2021-2024 Martin Pool
 
 //! Visit all the files in a source tree, and then the AST of each file,
 //! to discover mutation opportunities.
@@ -590,9 +590,9 @@ mod test {
     fn expected_mutants_for_own_source_tree() {
         let config = Config::read_file(Path::new("./.cargo/mutants.toml")).expect("Read config");
         let args =
-            Args::try_parse_from(["mutants", "--list", "--line-col=false"]).expect("Parse args");
-        let mut options = Options::new(&args, &config).expect("Build options");
-        options.colors = false; // TODO: Use a command-line arg.
+            Args::try_parse_from(["mutants", "--list", "--line-col=false", "--colors=never"])
+                .expect("Parse args");
+        let options = Options::new(&args, &config).expect("Build options");
         let mut list_output = String::new();
         let console = Console::new();
         let workspace = Workspace::open(

tests/cli/colors.rs

@@ -0,0 +1,107 @@
+// Copyright 2024 Martin Pool
+
+//! Tests for color output.
+
+// Testing autodetection seems hard because we'd have to make a tty, so we'll rely on humans noticing
+// for now.
+
+use predicates::prelude::*;
+
+use super::run;
+
+fn has_color_listing() -> impl Predicate<str> {
+    predicates::str::contains("with \x1b[33m0\x1b[0m")
+}
+
+fn has_ansi_escape() -> impl Predicate<str> {
+    predicates::str::contains("\x1b[")
+}
+
+fn has_color_debug() -> impl Predicate<str> {
+    predicates::str::contains("\x1b[34mDEBUG\x1b[0m")
+}
+
+/// The test fixtures force off colors, even if something else tries to turn it on.
+#[test]
+fn no_color_in_test_subprocesses_by_default() {
+    run()
+        .args(["mutants", "-d", "testdata/small_well_tested", "--list"])
+        .assert()
+        .success()
+        .stdout(has_ansi_escape().not())
+        .stderr(has_ansi_escape().not());
+}
+
+/// Colors can be turned on with `--color` and they show up in the listing and
+/// in trace output.
+#[test]
+fn colors_always_shows_in_stdout_and_trace() {
+    run()
+        .args([
+            "mutants",
+            "-d",
+            "testdata/small_well_tested",
+            "--list",
+            "--colors=always",
+            "-Ltrace",
+        ])
+        .assert()
+        .success()
+        .stdout(has_color_listing())
+        .stderr(has_color_debug());
+}
+
+#[test]
+fn cargo_term_color_env_shows_colors() {
+    run()
+        .env("CARGO_TERM_COLOR", "always")
+        .args([
+            "mutants",
+            "-d",
+            "testdata/small_well_tested",
+            "--list",
+            "-Ltrace",
+        ])
+        .assert()
+        .success()
+        .stdout(has_color_listing())
+        .stderr(has_color_debug());
+}
+
+#[test]
+fn invalid_cargo_term_color_rejected_with_message() {
+    run()
+        .env("CARGO_TERM_COLOR", "invalid")
+        .args([
+            "mutants",
+            "-d",
+            "testdata/small_well_tested",
+            "--list",
+            "-Ltrace",
+        ])
+        .assert()
+        .stderr(predicate::str::contains(
+            // The message does not currently name the variable due to <https://github.com/clap-rs/clap/issues/5202>.
+            "invalid value 'invalid'",
+        ))
+        .code(1);
+}
+
+/// Colors can be turned on with `CLICOLOR_FORCE`.
+#[test]
+fn clicolor_force_shows_in_stdout_and_trace() {
+    run()
+        .env("CLICOLOR_FORCE", "1")
+        .args([
+            "mutants",
+            "-d",
+            "testdata/small_well_tested",
+            "--list",
+            "--colors=never",
+            "-Ltrace",
+        ])
+        .assert()
+        .success()
+        .stdout(has_color_listing())
+        .stderr(has_color_debug());
+}