Respect loaded mask cube properly in spectral extraction.

[rosteen]

Fixes #3312.

Close #2718

[pllim]

Maybe give @javerbukh co-author credit given #2718 ?

[kecnry]

The code itself looks good to me - just a few workflow related questions to consider:

Do we want (now or as a a follow-up ticket) to have a dropdown to select the mask cube? Should the default be to apply the mask or to not apply it?

What happens if the intersection between an aperture and the mask results in no pixels?

Should the mask be used in other plugins as well (aperture photometry, moment map, collapse, etc)?

[camipacifici]

I think that if it is used, it should be used everywhere (it can be a follow up issue). Can there maybe be an option at load_data to load AND use the mask?

I would not set it to use by default, but maybe there could be a warning at loading like "there is a mask extension, if you want to load it do blah"

[camipacifici]

Visualize the mask cube would be very useful!

[gibsongreen]

Code seems straightforward, testing this on top of #3307 and the non-finite warning above the Fit Model button goes away with this PR.

In 3307, I was still seeing a model in the spectrum-viewer/table after model fitting, and there wasn't a traceback (but there was a warning was in the log).

I'll hold off approving for now and re-review if/once the above comments are decided on and if they go into this PR or follow-ups.

[rosteen]

Visualize the mask cube would be very useful!

Hm, is the mask a subset of the DQ flags we're showing? Or is it the set of all the flagged pixels? I'm not completely sure what the difference is between the two things or whether we're losing anything by not showing the mask vs DQ.

[rosteen]

@kecnry I just tested, if you define an aperture that is totally covered by the mask, it extracts a flat spectrum at 0 flux.

[rosteen]

I think I figured out the test failure - the science validation cube has NaNs in the mask/DQ array where the data isn't masked rather than 0s (btw, I did confirm that what we're storing from reading the mask array is what we're showing as the DQ data entry). This is awful, but easy to handle.

[codecov]

Codecov Report

Attention: Patch coverage is 73.91304% with 6 lines in your changes missing coverage. Please review.

Project coverage is 88.80%. Comparing base (6c946d3) to head (6caa76a).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
jdaviz/configs/cubeviz/plugins/parsers.py	37.50%	5 Missing ⚠️
...plugins/spectral_extraction/spectral_extraction.py	92.85%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3319      +/-   ##
==========================================
- Coverage   88.80%   88.80%   -0.01%     
==========================================
  Files         125      125              
  Lines       19137    19155      +18     
==========================================
+ Hits        16995    17010      +15     
- Misses       2142     2145       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rosteen]

I removed the lines storing the JWST DQ array as the original mask, and added a note in the docs as well as a Snackbar message indicating the original mask was applied during extraction, to address @camipacifici's concerns.

[kecnry]

Do we have any existing test data that we can send through spectral extraction to add coverage to these logic paths? Otherwise looks good to me until we can get the full support in to allow mask selection across all relevant plugins.

[gibsongreen]

The recent code updates look good to me and behavior has been maintained since my last review!

[lumberbot-app]

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

git checkout v4.0.x
git pull

2. Cherry pick the first parent branch of the this PR on top of the older branch:

git cherry-pick -x -m1 bf8cb2614ba7c823e7566b2fa8528d0de0e1c6ee

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

git commit -am 'Backport PR #3319: Respect loaded mask cube properly in spectral extraction.'

4. Push to a named branch:

git push YOURFORK v4.0.x:auto-backport-of-pr-3319-on-v4.0.x

5. Create a PR against branch v4.0.x, I would have named this PR:

"Backport PR #3319 on branch v4.0.x (Respect loaded mask cube properly in spectral extraction.)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.