Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add labeled outlier dataframe output capability #16

Merged
merged 9 commits into from
May 31, 2024

Conversation

d33bs
Copy link
Member

@d33bs d33bs commented May 22, 2024

Description

This PR adds capabilities to output a full dataframe with scores and a column indicating whether threshold(s) pass for rows of the provided dataframe. Along the way I found I need to further modularize the work and increase typing / documentation throughout. I also took a chance to increase testing overall with a "basic" / simulated dataset + parquet-based output (which would otherwise be many lines of data within a Python file) to help increase our testing coverage and make changes more visible.

Thanks for any feedback you may have!

Closes #10

What kind of change(s) are included?

  • Feature (adds or updates new capabilities)
  • Bug fix (fixes an issue).
  • Enhancement (adds functionality).
  • Breaking change (these changes would cause existing functionality to not work as expected).

Checklist

Please ensure that all boxes are checked before indicating that this pull request is ready for review.

  • I have read and followed the CONTRIBUTING.md guidelines.
  • I have searched for existing content to ensure this is not a duplicate.
  • I have performed a self-review of these additions (including spelling, grammar, and related).
  • These changes pass all pre-commit checks.
  • I have added comments to my code to help provide understanding
  • I have added a test which covers the code changes found within this PR
  • I have deleted all non-relevant text in this pull request template.

@d33bs d33bs requested a review from jenna-tomkinson May 22, 2024 21:45
Copy link
Member

@jenna-tomkinson jenna-tomkinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! My comments are mainly regarding documentation and some questions that will help guide my understanding.

Also, we might consider updating the function names to make them a bit more distinct since they all have outliers in the names. But, exciting to see the different options including adding the boolean for if a cell passed or not for a given condition.

I am really wanting to try this out now!

src/cosmicqc/analyze.py Outdated Show resolved Hide resolved
src/cosmicqc/analyze.py Outdated Show resolved Hide resolved
src/cosmicqc/analyze.py Show resolved Hide resolved
d33bs and others added 2 commits May 29, 2024 10:37
Co-authored-by: Jenna Tomkinson <107513215+jenna-tomkinson@users.noreply.github.com>
@d33bs
Copy link
Member Author

d33bs commented May 29, 2024

Thanks @jenna-tomkinson for the review! I've replied to your comments and also applied the suggestion (thanks!). When you're able, could I ask for a revisit on the open comments to ensure everything's good (I'll also re-request review)? Thanks!

@d33bs d33bs requested a review from jenna-tomkinson May 29, 2024 17:00
Copy link
Member

@jenna-tomkinson jenna-tomkinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for providing clarification on my questions! This PR LGTM and is ready to merge when you are! 🎉

src/cosmicqc/analyze.py Show resolved Hide resolved
@d33bs
Copy link
Member Author

d33bs commented May 31, 2024

Thanks @jenna-tomkinson ! I made some changes to reflect the default you mentioned and noticed a minor need to reconfigure label_outliers. I also noticed Python 3.8 was included with tests, but not in the minimum version specification so I removed this from tests. Merging this in.

@d33bs
Copy link
Member Author

d33bs commented May 31, 2024

Another note: Python 3.9 on Ubuntu 22.04 appeared to have issues in GitHub Actions so I cancelled this. All other tests succeeded, and all tests (including 3.9 on Ubuntu 22.04) succeeded in a prior commit. I expect these all to pass on merge to main.

@d33bs d33bs merged commit 7447eac into WayScience:main May 31, 2024
8 of 9 checks passed
@d33bs d33bs deleted the dataframe-output branch May 31, 2024 02:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add capability to output source dataframe with boolean columns per threshold set
2 participants