Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix flux unit conversions in plugins #3228

Merged
merged 3 commits into from
Nov 28, 2024

Conversation

cshanahan1
Copy link
Contributor

@cshanahan1 cshanahan1 commented Oct 18, 2024

Flux unit conversions done in plugins go through the 'flux_conversion_general' function that handles all equivalencies and special cases where the angle needs to be multiplied out before conversion. plugins that listen to unit conversion are tested with every combination of units to ensure all units that the spectral y axis can be translated to work in the plugin. Test coverage was added to each plugin that listens to flux unit changed messages from the Unit Conversion plugin to make sure the plugin in question actually supports all advertised flux units.

@github-actions github-actions bot added cubeviz testing imviz plugin Label for plugins common to multiple configurations labels Oct 18, 2024
@cshanahan1 cshanahan1 force-pushed the new_fix_plugin_equivs branch from dc6fc9c to 12c5b13 Compare November 1, 2024 13:50
@cshanahan1 cshanahan1 marked this pull request as ready for review November 1, 2024 13:51
@cshanahan1 cshanahan1 added this to the 4.1 milestone Nov 1, 2024
@pllim

This comment was marked as resolved.

@pllim

This comment was marked as resolved.

jdaviz/utils.py Outdated Show resolved Hide resolved
jdaviz/utils.py Outdated Show resolved Hide resolved
jdaviz/core/marks.py Outdated Show resolved Hide resolved
jdaviz/configs/imviz/plugins/coords_info/coords_info.py Outdated Show resolved Hide resolved
jdaviz/configs/imviz/plugins/coords_info/coords_info.py Outdated Show resolved Hide resolved
equivs = _eqv_flux_to_sb_pixel() + u.spectral_density(init_x)
init_y = init_y.to(self._units['y'], equivs)

init_y = flux_conversion_general([init_y.value], init_y.unit, self._units['y'], equivs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving another note here to retest model fitting in Specviz2d following merging this PR (#3253). Bug was preexisting unit conversion plugin but still worth retesting.

jdaviz/utils.py Outdated
Comment on lines 347 to 348
def flux_conversion_general(values, original_unit, target_unit,
equivalencies=None, with_unit=True):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we keep the parameters the same here as in flux_conversion? Originally adding spec and slice had to do with the image-viewers (the one case I explicitly remember testing for is working with the current state here) and eventually image data. I'll need to test this out a bit more to, might not be necessary but I will add a note about this on the Imviz implementation ticket.

@cshanahan1 cshanahan1 force-pushed the new_fix_plugin_equivs branch from 6c6a7e7 to 7f9e561 Compare November 20, 2024 13:11
Copy link

codecov bot commented Nov 20, 2024

Codecov Report

Attention: Patch coverage is 92.69231% with 19 lines in your changes missing coverage. Please review.

Project coverage is 88.80%. Comparing base (3dc6573) to head (ba347c7).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
jdaviz/core/unit_conversion_utils.py 93.33% 8 Missing ⚠️
...igs/default/plugins/model_fitting/model_fitting.py 64.28% 5 Missing ⚠️
...imviz/plugins/aper_phot_simple/aper_phot_simple.py 86.11% 5 Missing ⚠️
...configs/cubeviz/plugins/moment_maps/moment_maps.py 92.85% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3228      +/-   ##
==========================================
- Coverage   88.81%   88.80%   -0.02%     
==========================================
  Files         125      125              
  Lines       19041    19118      +77     
==========================================
+ Hits        16912    16977      +65     
- Misses       2129     2141      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

# only flux<> flux conversions will occur, so we only
# need the spectral_density equivalency. should I just
# be using the mean wavelength here?
eqv = u.spectral_density(np.mean(slab.spectral_axis))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would spectral axis ever have NaN (e.g., from masking)?

Suggested change
eqv = u.spectral_density(np.mean(slab.spectral_axis))
eqv = u.spectral_density(np.nanmean(slab.spectral_axis))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an assumption that mean wavelength represents the effective wavelength of a collapsed cube. Does this assumption always hold? If not, the converted value might be misleading. If we cannot promise correctness, maybe better to disallow.

Copy link
Contributor Author

@cshanahan1 cshanahan1 Nov 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that was the assumption i made, but I'm not sure it's correct. is there a better way to calculate an 'effective wavelength' for a cube? i don't think it makes sense to convert the unit of the whole cube before calculating the moment, even though that would solve the issue (cc @camipacifici)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my INS days, we grabbed the effective wavelength from the header based on the filter used. But that is hard to generalize for arbitrary telescope/instrument. If we must allow unit conversion, we might have to give user the option to manually enter the correct effective wavelength. But I say just disallow it if straight conversion is not possible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should leave the decision to the user and not assume a conversion that might be inaccurate. For narrow subsets, using the median of the wavelength range of the subset should be fine, but for, e.g., collapsing the full wavelength extent of the cube, the two procedures (convert then collapse vs collapse then convert) could return different results. If there is a default and a warning is returned, the user might miss it. Could the offended dataset be grayed out if a conversion of this kind is needed? The user could then decide whether to keep it with the approximate conversion or discard it and create the collapsed image again in the new units.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the offended dataset be grayed out if a conversion of this kind is needed?

Grayed out where? We could (without too much effort) create some sort of infrastructure for warnings/alerts about specific data layers where the legend and data menu get a ‼️ symbol with text in the tooltip, or we could maybe (I'd need to think through the consequences more first) migrate some of these to be auto-updating like spectral extraction and react to a change in units 🤔

@@ -350,3 +352,75 @@ def test_correct_output_spectral_y_units(cubeviz_helper, spectrum1d_cube_custom_

mm.calculate_moment()
assert mm.moment.unit == moment_unit.replace('m', 'um')


@pytest.mark.parametrize("flux_angle_unit", [(u.Unit(x), u.sr) for x in SPEC_PHOTON_FLUX_DENSITY_UNITS] + [(u.Unit(x), PIX2) for x in SPEC_PHOTON_FLUX_DENSITY_UNITS]) # noqa
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe less confusing to use nested parametrization here? This is a bit hard to read.

https://stackoverflow.com/questions/40164910/nested-parametrized-tests-pytest

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also wonder if this is more effective as a test class where the moment map is pre-calculated and re-used in a test method via parametrization, instead of recalculating for each combo. Does that make sense?

"""Make sure outputs of the aperture photometry plugin in Cubeviz
reflect the correct choice of display units from the Unit
Conversion plugin.
@pytest.mark.parametrize("flux_angle_unit", [(u.Unit(x), u.sr) for x in SPEC_PHOTON_FLUX_DENSITY_UNITS] + [(u.Unit(x), PIX2) for x in SPEC_PHOTON_FLUX_DENSITY_UNITS]) # noqa
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cshanahan1 cshanahan1 requested a review from pllim November 21, 2024 19:49
Copy link
Contributor

@pllim pllim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! A few more minor things.

jdaviz/configs/cubeviz/plugins/moment_maps/moment_maps.py Outdated Show resolved Hide resolved
jdaviz/configs/cubeviz/plugins/moment_maps/moment_maps.py Outdated Show resolved Hide resolved
jdaviz/configs/cubeviz/plugins/moment_maps/moment_maps.py Outdated Show resolved Hide resolved
jdaviz/configs/cubeviz/plugins/moment_maps/moment_maps.py Outdated Show resolved Hide resolved
Copy link
Contributor

@pllim pllim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the huge clean up and adding tests!

Let's get this in soon before more conflicts occur.

jdaviz/configs/cubeviz/plugins/moment_maps/moment_maps.py Outdated Show resolved Hide resolved
spectral_axis_unit = u.Unit(self.spectrum_viewer.state.x_display_unit)
print(f'converting {self.moment} to {self.moment_zero_unit}. spectral_axis_unit = {spectral_axis_unit}')

# if the flux unit is a per-frequency unit but the spectral axis unit
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... Isn't it more natural for specutils to do this upstream, @rosteen ?

jdaviz/configs/cubeviz/plugins/moment_maps/moment_maps.py Outdated Show resolved Hide resolved
@spacetelescope spacetelescope deleted a comment from pllim Nov 25, 2024
@pllim
Copy link
Contributor

pllim commented Nov 25, 2024

Does this mean we can archive the ticket to think about wavelength input for moment map (someone created a ticket, right?)?

If CI stays green, my approval stands. Thanks!

Copy link
Contributor

@gibsongreen gibsongreen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was an awesome effort, great work! I created the two new bug tickets that I had mentioned in Slack, all the remaining plugin behavior went smoothly with many rounds of testing and recent code changes all look good to me.

@cshanahan1 cshanahan1 force-pushed the new_fix_plugin_equivs branch from 26bc6a9 to 622e5eb Compare November 28, 2024 03:16
Clare Shanahan added 2 commits November 27, 2024 22:33
@cshanahan1 cshanahan1 merged commit 9a90da0 into spacetelescope:main Nov 28, 2024
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cubeviz imviz plugin Label for plugins common to multiple configurations Ready for final review testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants