Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for proc-macro crates #406

Closed
chesedo opened this issue Sep 12, 2024 · 8 comments · Fixed by #415
Closed

Support for proc-macro crates #406

chesedo opened this issue Sep 12, 2024 · 8 comments · Fixed by #415
Labels
enhancement New feature or request small

Comments

@chesedo
Copy link

chesedo commented Sep 12, 2024

I tried running cargo-mutants on a proc-macro crate and was surprised to see it found no mutants. When I turned log level to tracing I saw this in the logs:

DEBUG package{name=despatma-dependency-container}: skipping target "despatma_dependency_container" of kinds ["proc-macro"]

Is there a reason proc-macros are skipped that I might be missing?

@sourcefrog
Copy link
Owner

I don't think there's any positive reason, it's just not something that we recognize yet.

It looks like this is from https://github.com/chesedo/despatma

Looking at the code, it does seem to have tests that cargo-mutants could run, and code that it would understand. So this might be as simple as just changing the code in cargo.rs that decides which cargo targets will be mutated, where this trace line comes from.

@sourcefrog sourcefrog added enhancement New feature or request small labels Sep 12, 2024
@sourcefrog
Copy link
Owner

@chesedo it looks like the tests fail at head in despatma:

test tests/fail/missing_types.rs ... mismatch

EXPECTED:
┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
error[E0405]: cannot find trait `Circle` in this scope
 --> tests/fail/missing_types.rs:6:14
  |
6 | visitor!(dyn Circle, Rectangle, dyn Sphere, Arc, dyn Cube);
  |              ^^^^^^ not found in this scope
  |
help: consider importing this trait
  |
4 + use crate::library::shapes::Circle;
  |

error[E0412]: cannot find type `Rectangle` in this scope
 --> tests/fail/missing_types.rs:6:22
  |
6 | visitor!(dyn Circle, Rectangle, dyn Sphere, Arc, dyn Cube);
  |                      ^^^^^^^^^ not found in this scope
  |
help: consider importing this struct
  |
4 + use crate::library::shapes::Rectangle;
  |

error[E0405]: cannot find trait `Sphere` in this scope
 --> tests/fail/missing_types.rs:6:37
  |
6 | visitor!(dyn Circle, Rectangle, dyn Sphere, Arc, dyn Cube);
  |                                     ^^^^^^ not found in this scope
  |
help: consider importing this trait
  |
4 + use crate::library::shapes::Sphere;
  |

error[E0412]: cannot find type `Arc` in this scope
 --> tests/fail/missing_types.rs:6:45
  |
6 | visitor!(dyn Circle, Rectangle, dyn Sphere, Arc, dyn Cube);
  |                                             ^^^ not found in this scope
  |
help: consider importing one of these items
  |
4 + use crate::library::shapes::Arc;
  |
4 + use std::sync::Arc;
  |

error[E0405]: cannot find trait `Cube` in this scope
 --> tests/fail/missing_types.rs:6:54
  |
6 | visitor!(dyn Circle, Rectangle, dyn Sphere, Arc, dyn Cube);
  |                                                      ^^^^ not found in this scope
  |
help: consider importing this trait
  |
4 + use crate::library::shapes::Cube;
  |
┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈

ACTUAL OUTPUT:
┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
error[E0405]: cannot find trait `Circle` in this scope
 --> tests/fail/missing_types.rs:6:14
  |
6 | visitor!(dyn Circle, Rectangle, dyn Sphere, Arc, dyn Cube);
  |              ^^^^^^ not found in this scope
  |
help: consider importing this trait
  |
4 + use crate::library::shapes::Circle;
  |

error[E0412]: cannot find type `Rectangle` in this scope
 --> tests/fail/missing_types.rs:6:22
  |
6 | visitor!(dyn Circle, Rectangle, dyn Sphere, Arc, dyn Cube);
  |                      ^^^^^^^^^ not found in this scope
  |
help: consider importing this struct
  |
4 + use crate::library::shapes::Rectangle;
  |

error[E0405]: cannot find trait `Sphere` in this scope
 --> tests/fail/missing_types.rs:6:37
  |
6 | visitor!(dyn Circle, Rectangle, dyn Sphere, Arc, dyn Cube);
  |                                     ^^^^^^ not found in this scope
  |
help: consider importing this trait
  |
4 + use crate::library::shapes::Sphere;
  |

error[E0412]: cannot find type `Arc` in this scope
 --> tests/fail/missing_types.rs:6:45
  |
6 | visitor!(dyn Circle, Rectangle, dyn Sphere, Arc, dyn Cube);
  |                                             ^^^ not found in this scope
  |
help: consider importing one of these structs
  |
4 + use crate::library::shapes::Arc;
  |
4 + use std::sync::Arc;
  |

error[E0405]: cannot find trait `Cube` in this scope
 --> tests/fail/missing_types.rs:6:54
  |
6 | visitor!(dyn Circle, Rectangle, dyn Sphere, Arc, dyn Cube);
  |                                                      ^^^^ not found in this scope
  |
help: consider importing this trait
  |
4 + use crate::library::shapes::Cube;
  |
┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
note: If the actual output is the correct output you can bless it by rerunning
      your test with the environment variable TRYBUILD=overwrite



test fail ... FAILED
tests/expand/compound_name.rs - ok
tests/expand/helper_tmpl.rs - ok
tests/expand/multiple.rs - ok
tests/expand/no_default.rs - ok
tests/expand/simple.rs - ok
tests/expand/visitor_mut.rs - ok
test expand ... ok

failures:

---- fail stdout ----
thread 'fail' panicked at /Users/mbp/.cargo/registry/src/index.crates.io-6f17d22bba15001f/trybuild-1.0.99/src/run.rs:101:13:
1 of 4 tests failed
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

@chesedo
Copy link
Author

chesedo commented Sep 12, 2024

@chesedo it looks like the tests fail at head in despatma:

A newer Rust version recently changed some of the error messages. Can't quite remember if I updated the repo yet.

I assume you are trying to test if cargo-mutants can run against a proc-macro crate?

@chesedo
Copy link
Author

chesedo commented Sep 12, 2024

I don't think there's any positive reason

If there is no solid, known reason then I can make a fork to test locally. And can make a PR if it works

@sourcefrog
Copy link
Owner

sourcefrog commented Sep 12, 2024 via email

@sourcefrog sourcefrog linked a pull request Sep 14, 2024 that will close this issue
3 tasks
@chesedo
Copy link
Author

chesedo commented Sep 17, 2024

The linked branch here has a change to enable it, it just needs some manual tests, and a test added to the tree!

Okay, I was able to manually test the linked branch and the mutants went from 16 to 49. I also fixed the repo if you also wanted to perform a manual test.

@sourcefrog
Copy link
Owner

OK yep this seems to work well on your crate:

; c r mutants -d ../despatma/
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.14s
     Running `target/debug/cargo-mutants mutants -d ../despatma/`
Found 72 mutants to test
ok       Unmutated baseline in 7.7s build + 26.7s test
 INFO Auto-set test timeout to 2m 14s
MISSED   despatma-lib/src/key_value.rs:51:31: replace && with || in <impl PartialEq for KeyValue>::eq in 0.4s build + 0.2s test
MISSED   despatma-lib/src/key_value.rs:51:9: replace <impl PartialEq for KeyValue>::eq -> bool with true in 0.3s build + 0.2s test
MISSED   despatma-dependency-container/src/processing/visitor/extract_async.rs:20:33: replace |= with ^= in <impl VisitorMut for ExtractAsync>::visit_dependency_mut in 0.9s build + 8.7s test
MISSED   despatma-dependency-container/src/processing/mod.rs:54:9: replace <impl PartialEq for Lifetime>::eq -> bool with true in 0.8s build + 9.4s test
72 mutants tested in 3m 20s: 4 missed, 55 caught, 13 unviable

@chesedo
Copy link
Author

chesedo commented Oct 1, 2024

Awesome! 🥳 Thanks for the update!

sourcefrog added a commit that referenced this issue Oct 1, 2024
- [x] Docs explaining that proc macro implementations can be mutated,
but this only mutates the definitions and not the usage of macros
- [x] News
- [x] Maybe test that the testdata crate is fully covered?

Fixes #406
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request small
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants