Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapt pysteps to allow for postprocessing plugins #405

Open
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

joeycasey87
Copy link

Added an infrastructure.py file to the postprocessing folder which should operate in the same way as the interface file does for the importers in the io folder. The postprocessor file is currently effectively empty as no postprocessor plugins have been added yet. A line has been added to the init file so that it should act similar to the init file in the io folder.

@@ -2,3 +2,4 @@
"""Methods for post-processing of forecasts."""

from . import ensemblestats
from postprocessors import *
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
from postprocessors import *
from .postprocessors import *

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just wondering if "postprocessors" is a good name? sounds a bit too vague, particularly as a submodule of "postprocessing" ... can we try to be a bit more specific? perhaps use "diagnostic" instead? what do you think? @ladc ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes good point - "diagnostic" makes sense if we also want to add other postprocessors in the future which are not purely diagnostic (such as bias correction methods).

Copy link

codecov bot commented Jul 25, 2024

Codecov Report

Attention: Patch coverage is 85.71429% with 16 lines in your changes missing coverage. Please review.

Project coverage is 84.01%. Comparing base (d77fe73) to head (d9b593d).
Report is 11 commits behind head on master.

Files Patch % Lines
pysteps/postprocessing/interface.py 78.87% 15 Missing ⚠️
pysteps/tests/test_interfaces.py 96.55% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #405      +/-   ##
==========================================
+ Coverage   83.67%   84.01%   +0.34%     
==========================================
  Files         159      162       +3     
  Lines       12649    12994     +345     
==========================================
+ Hits        10584    10917     +333     
- Misses       2065     2077      +12     
Flag Coverage Δ
unit_tests 84.01% <85.71%> (+0.34%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dnerini dnerini changed the title Added structure for the creation of postprocessing plugins Add diagnostic plugins Aug 5, 2024
Also reformatted with black. Tests were added for the get_method and diagnostics_info functions in the postprocessing interface. Tests for the discover_diagnostics function will be written once these changes have been merged as then the cookiecutter plugins can then be properly tested.
@joeycasey87
Copy link
Author

Hi @dnerini, could you please review this pull request when you have the time? The extended version of the cookiecutter plugin template which can be used to create importer, diagnostic postprocessing, and nowcast plugins is nearly finished but I require the postprocessing interface to be merged to ensure that everything is behaving correctly when tested. Cheers

@ladc
Copy link
Contributor

ladc commented Aug 20, 2024

Thanks for this contribution! We discussed how to make it more generic and @joeycasey87 will try to refactor it tomorrow to allow for various kinds of post-processors including new ensemblestats and bias correction methods, for example. To be continued.

joeycasey87 and others added 7 commits August 26, 2024 10:22
Changed the test file to match the test updated pysteps test file
These were necessary for the IO plugins, but less so for the diagnostics.
Avoid duplicate code, refactor into functions.
Also fix a small typo causing a bug: postprocessor_s_
@ladc ladc changed the title Add diagnostic plugins Adapt pysteps to allow for postprocessing plugins (diagnostics and ensemblestats for now) Aug 27, 2024
@ladc ladc changed the title Adapt pysteps to allow for postprocessing plugins (diagnostics and ensemblestats for now) Adapt pysteps to allow for postprocessing plugins Aug 27, 2024
@dnerini
Copy link
Member

dnerini commented Sep 8, 2024

what's the status of this PR? should we mark it as a draft?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In review
Development

Successfully merging this pull request may close these issues.

3 participants