Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable warnings as errors for weekly run in test suite #6217

Open
wants to merge 70 commits into
base: master
Choose a base branch
from

Conversation

mlxd
Copy link
Member

@mlxd mlxd commented Sep 4, 2024

Before submitting

Please complete the following checklist when submitting a PR:

  • All new features must include a unit test.
    If you've fixed a bug or added code that should be tested, add a test to the
    test directory!

  • All new functions and code must be clearly commented and documented.
    If you do make documentation changes, make sure that the docs build and
    render correctly by running make docs.

  • Ensure that the test suite passes, by running make test.

  • Add a new entry to the doc/releases/changelog-dev.md file, summarizing the
    change, and including a link back to the PR.

  • The PennyLane source code conforms to
    PEP8 standards.
    We check all of our code against Pylint.
    To lint modified files, simply pip install pylint, and then
    run pylint pennylane/path/to/file.py.

When all the above are checked, delete everything above the dashed
line and fill in the pull request template.


Context: This PR enables a weekly run of the test suite with Python-level warnings promoted to errors (exceptions). This will allow the test suite to regularly report when hitting warnings through general execution.

Description of the Change: Add a new workflow to run the test suite with Python warnings as errors on a weekly basis,

Benefits: Enable us to catch warnings when and where they occur during testing.

Possible Drawbacks: Additional overhead of running the CI at the intended occurrence.

Related GitHub Issues:

@mlxd mlxd added the do not merge ⚠️ Do not merge the pull request until this label is removed label Sep 5, 2024
pennylane/devices/tests/pytest.ini Outdated Show resolved Hide resolved
tests/pytest.ini Outdated Show resolved Hide resolved
@mlxd mlxd marked this pull request as ready for review September 12, 2024 19:59
@mlxd mlxd changed the title Enable warnings as errors globally in test suite Enable warnings as errors for weekly run in test suite Sep 12, 2024
@mlxd mlxd removed the do not merge ⚠️ Do not merge the pull request until this label is removed label Sep 12, 2024
Copy link
Contributor

@mudit2812 mudit2812 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Left a bunch of suggestions. I would also suggest adding a pull_request trigger to the workflow once it's reasonably complete so you can test it out in action before the PR gets merged.

tests/pytest.ini Show resolved Hide resolved
.github/workflows/package_warnings_as_errors.yml Outdated Show resolved Hide resolved
.github/workflows/package_warnings_as_errors.yml Outdated Show resolved Hide resolved
.github/workflows/package_warnings_as_errors.yml Outdated Show resolved Hide resolved
.github/workflows/unit-test.yml Outdated Show resolved Hide resolved
.github/workflows/interface-unit-tests.yml Show resolved Hide resolved
Copy link

codecov bot commented Sep 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.58%. Comparing base (060bb9a) to head (01df56b).
Report is 3 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6217      +/-   ##
==========================================
- Coverage   99.59%   99.58%   -0.02%     
==========================================
  Files         444      443       -1     
  Lines       42321    42230      -91     
==========================================
- Hits        42149    42053      -96     
- Misses        172      177       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mlxd and others added 2 commits September 13, 2024 14:05
Co-authored-by: Rashid N H M <95639609+rashidnhm@users.noreply.github.com>
Copy link
Contributor

@mudit2812 mudit2812 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of more changes to tidy up the workflow, but functionally looks great!

.github/workflows/interface-unit-tests.yml Show resolved Hide resolved
.github/workflows/interface-unit-tests.yml Outdated Show resolved Hide resolved
.github/workflows/interface-unit-tests.yml Outdated Show resolved Hide resolved
.github/workflows/interface-unit-tests.yml Outdated Show resolved Hide resolved
.github/workflows/package_warnings_as_errors.yml Outdated Show resolved Hide resolved
.github/workflows/unit-test.yml Show resolved Hide resolved
mlxd and others added 2 commits September 13, 2024 14:27
Co-authored-by: Mudit Pandey <mudit.pandey@xanadu.ai>
Co-authored-by: Mudit Pandey <mudit.pandey@xanadu.ai>
Copy link
Contributor

@mudit2812 mudit2812 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. We have an epic for next quarter with stories for updating workflows. I'll add a story there to implement the suggestions in this PR that are not able to be done right now.

.github/workflows/interface-unit-tests.yml Show resolved Hide resolved
.github/workflows/interface-unit-tests.yml Outdated Show resolved Hide resolved
@mlxd mlxd enabled auto-merge (squash) September 13, 2024 18:35
@mlxd mlxd disabled auto-merge September 13, 2024 19:42
@mlxd
Copy link
Member Author

mlxd commented Sep 13, 2024

Note: splitting seems to be affected in this PR. Follow-up re-eval needed.

@mudit2812
Copy link
Contributor

Note: splitting seems to be affected in this PR. Follow-up re-eval needed.

Maybe the splits need to be set up in the setup-ci-load step too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants