Torch Dataset

[kalebphipps]

Description

This PR includes a torch.Datset for Calibration Items from the PAINT database. This includes:

• A dataset that can be initialized with a root_dir containing calibration items.
• The possibility to load different calibration items, i.e. raw images, cropped images, flux images, flux_centered images, or calibration properties.
• The possibility to initialize a dataset from a benchmark dataset csv file containing information on the splits.
• The possibility to initialize a dataset using all calibration items from one or more heliostats.

To enable minimal tests, some data files (images and JSONs) have been uploaded into Git.

This PR therefore, closes Issue #10.

In addition, unused variables were renamed to clear up any confusion from Issue #59.

NOTE: Due to download functions in the dataset that are not tested, this PR will decrease code coverage!

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[github-actions]

Name	Stmts	Miss	Cover	Missing
paint/__init__.py	4	0	100%
paint/data/dataset.py	72	1	99%	18
paint/data/dataset_splits.py	83	1	99%	51
paint/preprocessing/__init__.py	15	0	100%
paint/preprocessing/binary_extractor.py	59	2	97%	81-90
paint/preprocessing/calibration_stac.py	10	0	100%
paint/preprocessing/catalog_stac.py	4	0	100%
paint/preprocessing/deflectometry_stac.py	10	0	100%
paint/preprocessing/dwd_mappings.py	2	0	100%
paint/preprocessing/dwd_stac_item.py	5	0	100%
paint/preprocessing/dwd_weather.py	40	1	98%	100
paint/preprocessing/focal_spot_extractor.py	75	4	95%	86, 93, 117, 274
paint/preprocessing/heliostat_catalog_stac.py	19	2	89%	40, 48
paint/preprocessing/juelich_weather_converter.py	51	3	94%	59, 111-114
paint/preprocessing/juelich_weather_mappings.py	20	0	100%
paint/preprocessing/juelich_weather_stac_item.py	6	0	100%
paint/preprocessing/properties_stac.py	11	0	100%
paint/preprocessing/target_cropper.py	55	1	98%	206
paint/preprocessing/tower_measurements.py	47	0	100%
paint/preprocessing/tower_stac.py	5	0	100%
paint/preprocessing/weather_collection_stac.py	5	0	100%
paint/util/__init__.py	29	0	100%
paint/util/gauss_kruger_converter.py	7	0	100%
paint/util/paint_mappings.py	277	0	100%
paint/util/preprocessing.py	31	0	100%
paint/util/utils.py	49	2	96%	113-114
TOTAL	991	17	98%

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.38%. Comparing base (e7d6d81) to head (50cae29).
Report is 46 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #62      +/-   ##
==========================================
+ Coverage   97.71%   98.38%   +0.67%     
==========================================
  Files          26       26              
  Lines         919      991      +72     
==========================================
+ Hits          898      975      +77     
+ Misses         21       16       -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Markus-Goetz]

The PR reduces the test coverage. Is it possible to test the three missing lines?

I think it might be meaningful to showcase the examples in the README.md or at least mention the, so that a new user can find them easily

[kalebphipps]

I have addressed all comments and either made comments explaining why I don't plan on making the change or made the change directly and resolved the conversation.

The coverage issue has been resolved by excluding certain lines (i.e. download functions) from the coverage and including an additional test to verify the string representation. I have also completely reworked the README.md to highlight the new repository structure and example scripts.