auto-update support for plugin results

[kecnry]

Description

This pull request implements the infrastructure for plugin results to know their inputs and update when the input data and/or subset is changed. Currently this is only implemented for the spectral extraction (in cubeviz) plugin, but is generalized so that it could be extended to other plugins as well.

Screen.Recording.2024-01-30.at.9.27.10.AM.mov

Remaining work to generalize if continuing in this direction:

• infrastructure to know the method to call. Perhaps each plugin should implement a __call__ or similar method?
• as seen in the video above, the current implementation listening to subset updates can trigger unnecessary updates. To avoid this, we should ignore updates for style changes to subsets and could store the hash of the input subset(s)/dataset(s) to avoid reacting to duplicate messages (subset update message from multiple viewers, etc), or filter and only listen to subset updates from the default viewer.
• the current implementation specifically listens to data and spectral_subset, but this needs to be generalized to listen to any arbitrary data or subset inputs.
• hide UI behind feature flag until replacing of collapse is in place? (may not be necessary now that we're intentionally deferring this until after the 3.9 release)
• debug and fix multiple updates from same event (investigation into hash was not successful since the hash is changing between event callbacks)
• to_dict/from_dict to avoid deprecated attributes

Potential follow-ups:

• move default viewer logic when initializing plugin from looping through the registry to instead have the registry store the information in the class?
• performance would benefit strongly from improving the "overwrite" capability to actually overwrite the internal arrays instead of sending through the parser and removing/adding data entries. (migrate logic from lcviz's _set_data_component).
• warning in advance when requesting live-results and indication while the results are updating, since these operations can get expensive. If there are any internal loops which update subsets, we can defer processing the updates until that loop has finished. Maybe we should detect when an update takes longer than some threshold and display a snackbar warning telling which dataset is to blame and allow disabling.
• indication in data-menu of which data entries have this enabled, possibly with the ability to (temporarily?) disable

Change log entry

• Is a change log needed? If yes, is it added to CHANGES.rst? If you want to avoid merge conflicts,
  list the proposed change log here for review and add to CHANGES.rst before merge. If no, maintainer
  should add a no-changelog-entry-needed label.

Checklist for package maintainer(s)

This checklist is meant to remind the package maintainer(s) who will review this pull request of some common things to look for. This list is not exhaustive.

• Are two approvals required? Branch protection rule does not check for the second approval. If a second approval is not necessary, please apply the trivial label.
• Do the proposed changes actually accomplish desired goals? Also manually run the affected example notebooks, if necessary.
• Do the proposed changes follow the STScI Style Guides?
• Are tests added/updated as required? If so, do they follow the STScI Style Guides?
• Are docs added/updated as required? If so, do they follow the STScI Style Guides?
• Did the CI pass? If not, are the failures related?
• Is a milestone set? Set this to bugfix milestone if this is a bug fix and needs to be released ASAP; otherwise, set this to the next major release milestone.
• After merge, any internal documentations need updating (e.g., JIRA, Innerspace)?

[kecnry]

this for-loop is ugly. I have a local stash that attempts to replace this by having the registry itself set cls._viewer_reference_name_kwargs which shows promise but still needs some work.

[codecov]

Codecov Report

Attention: Patch coverage is 92.72727% with 8 lines in your changes are missing coverage. Please review.

Project coverage is 88.88%. Comparing base (a6de416) to head (4677435).
Report is 5 commits behind head on main.

Files	Patch %	Lines
jdaviz/app.py	87.09%	4 Missing ⚠️
jdaviz/core/template_mixin.py	92.50%	3 Missing ⚠️
jdaviz/core/user_api.py	96.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2680      +/-   ##
==========================================
+ Coverage   88.82%   88.88%   +0.06%     
==========================================
  Files         111      111              
  Lines       16879    16980     +101     
==========================================
+ Hits        14993    15093     +100     
- Misses       1886     1887       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[javerbukh]

There is some flashing that occurs when updating a subset and I do not see the second spatial subset subregion render in the viewer when creating a composite subset. Otherwise, this is looking promising! Are there other features you would like tested that are not mentioned in the description above?

[kecnry]

and I do not see the second spatial subset subregion render in the viewer when creating a composite subset.

Can you explain what you mean here? Is that the auto-collapsed spectrum you're referring to - and if so, does that happen before this PR? The plan is to use this to replace that functionality in the near future, so if that is the case, it doesn't worry me too much, but would be weird if it was introduced by this PR.

[cshanahan1]

I tried this out and it works, with one exception that I noticed.

If you create a composite subset, the sum is automatically plotted in the spectrum viewer upon creation (by glue, not going through the plugin). However, the spectral extraction doesn't support composite subsets, so the auto-update fails with a non-informative traceback, and you just can't create the composite subset. I think this can be addressed with a try-except (that would also catch other failures) that disables the auto-update but still allows you to modify the subset. As a follow up, but not now since this issue existed before this PR, be more clear in the plugin that composite subsets aren't valid selections (until that functionality is enabled).