JP-3718: Improve AMI observable averaging, error calculations #8846

rcooper295 · 2024-10-01T15:41:57Z

This PR addresses JP-3718, which changes how observables are averaged and their errors are calculated. Now calculating all "downstream" observables (i.e. calculated from fringe phases and amplitudes) from already-averaged fringe quantities, rather than calculating and then averaging. Now calculating covariances between related fringes and amplitudes (between baselines, triangles, and quads) and taking standard error of the mean (from covariance) as uncertainty.

Resolves JP-3718
Resolves #8733
Closes #8826

Tasks

news fragment change types...

changes/<PR#>.general.rst: infrastructure or miscellaneous change
changes/<PR#>.docs.rst
changes/<PR#>.stpipe.rst
changes/<PR#>.datamodels.rst
changes/<PR#>.scripts.rst
changes/<PR#>.fits_generator.rst
changes/<PR#>.set_telescope_pointing.rst
changes/<PR#>.pipeline.rst

stage 1

changes/<PR#>.group_scale.rst
changes/<PR#>.dq_init.rst
changes/<PR#>.emicorr.rst
changes/<PR#>.saturation.rst
changes/<PR#>.ipc.rst
changes/<PR#>.firstframe.rst
changes/<PR#>.lastframe.rst
changes/<PR#>.reset.rst
changes/<PR#>.superbias.rst
changes/<PR#>.refpix.rst
changes/<PR#>.linearity.rst
changes/<PR#>.rscd.rst
changes/<PR#>.persistence.rst
changes/<PR#>.dark_current.rst
changes/<PR#>.charge_migration.rst
changes/<PR#>.jump.rst
changes/<PR#>.clean_flicker_noise.rst
changes/<PR#>.ramp_fitting.rst
changes/<PR#>.gain_scale.rst

stage 2

changes/<PR#>.assign_wcs.rst
changes/<PR#>.badpix_selfcal.rst
changes/<PR#>.msaflagopen.rst
changes/<PR#>.nsclean.rst
changes/<PR#>.imprint.rst
changes/<PR#>.background.rst
changes/<PR#>.extract_2d.rst
changes/<PR#>.master_background.rst
changes/<PR#>.wavecorr.rst
changes/<PR#>.srctype.rst
changes/<PR#>.straylight.rst
changes/<PR#>.wfss_contam.rst
changes/<PR#>.flatfield.rst
changes/<PR#>.fringe.rst
changes/<PR#>.pathloss.rst
changes/<PR#>.barshadow.rst
changes/<PR#>.photom.rst
changes/<PR#>.pixel_replace.rst
changes/<PR#>.resample_spec.rst
changes/<PR#>.residual_fringe.rst
changes/<PR#>.cube_build.rst
changes/<PR#>.extract_1d.rst
changes/<PR#>.resample.rst

stage 3

changes/<PR#>.assign_mtwcs.rst
changes/<PR#>.mrs_imatch.rst
changes/<PR#>.tweakreg.rst
changes/<PR#>.skymatch.rst
changes/<PR#>.exp_to_source.rst
changes/<PR#>.outlier_detection.rst
changes/<PR#>.tso_photometry.rst
changes/<PR#>.stack_refs.rst
changes/<PR#>.align_refs.rst
changes/<PR#>.klip.rst
changes/<PR#>.spectral_leak.rst
changes/<PR#>.source_catalog.rst
changes/<PR#>.combine_1d.rst
changes/<PR#>.ami.rst

other

changes/<PR#>.wfs_combine.rst
changes/<PR#>.white_light.rst
changes/<PR#>.cube_skymatch.rst
changes/<PR#>.engdb_tools.rst
changes/<PR#>.guider_cds.rst

codecov · 2024-10-01T16:09:33Z

Codecov Report

Attention: Patch coverage is 78.94737% with 56 lines in your changes missing coverage. Please review.

Project coverage is 64.56%. Comparing base (adac245) to head (7745ad2).
Report is 21 commits behind head on main.

Files with missing lines	Patch %	Lines
jwst/ami/ami_analyze_step.py	12.72%	48 Missing ⚠️
jwst/ami/oifits.py	95.67%	8 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8846      +/-   ##
==========================================
+ Coverage   64.54%   64.56%   +0.02%     
==========================================
  Files         375      375              
  Lines       38739    38890     +151     
==========================================
+ Hits        25003    25109     +106     
- Misses      13736    13781      +45

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

rcooper295 · 2024-11-20T19:02:08Z

Note: this PR requires spacetelescope/stdatamodels#357 to be merged first, then we will update the stdatamodels requirement in pyproject.toml here.

melanieclarke · 2024-11-22T20:13:52Z

Initial regression tests started here, with stdatamodels branch from spacetelescope/stdatamodels#357:
https://github.com/spacetelescope/RegressionTests/actions/runs/11979457630

melanieclarke · 2024-12-04T22:35:21Z

Running regression tests again with stdatamodels on main:
https://github.com/spacetelescope/RegressionTests/actions/runs/12169530893

melanieclarke

I'm reviewing mostly for documentation and style, since you have confirmed that the regression test changes for AMI products are as expected. There are a few minor things to clean up, but I think this is looking good.

It looks like errors here are now standard error of the mean, addressing concerns in #8826. Can we close that PR in favor of this one?

docs/jwst/ami_analyze/description.rst

jwst/ami/ami_analyze_step.py

jwst/ami/oifits.py

emolter

Thanks for submitting this Rachel, and it looks excellent overall! In general the documentation is clear and detailed, and the code is structured well and easy to follow. Quite a few comments, but most of them are just small suggestions for clarity or docstrings that aren't quite done yet.

jwst/ami/ami_analyze_step.py

jwst/ami/instrument_data.py

jwst/ami/oifits.py

jwst/ami/tests/test_ami_interface.py

…les from already-averaged fringe phases or amplitudes, WIP

…an for uncertainty estimates

…documentation with examples.

…keywords

… calibrator_object_id in calibrated files. Use ROLL_REF instead of PA_V3, though leaving old variable names for now.

…ition angle related keywords

…@emolter

…@emolter

melanieclarke

I think all my comments have been addressed - thanks for all the documentation updates! I found two tiny remaining typos, and I'd like to re-run regression tests just in case, but I will approve now.

melanieclarke · 2024-12-06T19:35:18Z

docs/jwst/ami_analyze/description.rst

+
+.. code-block:: python
+
+   # Create a F480M filter + Vega bandpass ASDF file


Thanks for the fix! These instructions work for me now, to make the bandpass file and feed it to ami_analyze via --bandpass.

melanieclarke · 2024-12-06T19:36:20Z

docs/jwst/ami_analyze/description.rst

@@ -81,20 +144,19 @@ Interferometric observables
 :Data model: `~jwst.datamodels.AmiOIModel`
 :File suffix: _ami-oi.fits, _amimulti-oi.fits

-. The inteferometric observables are saved as OIFITS files, a registered FITS format
+The inteferometric observables are saved as OIFITS files, a registered FITS format


Suggested change

The inteferometric observables are saved as OIFITS files, a registered FITS format

The interferometric observables are saved as OIFITS files, a registered FITS format

melanieclarke · 2024-12-06T19:39:33Z

jwst/ami/instrument_data.py

+            Nov. 2024 email discussion with Tony Sohn, Paul Goudfrooij confirmed V2/V3 coordinate
+            rotation back to "North up" equatorial orientation should use ROLL_REF + V3I_YANG
+            (= PA_APER).


Super tiny, but this section still needs to be un-indented, to match the "Notes" indent level.

melanieclarke · 2024-12-06T19:47:57Z

Re-running regression tests again here:
https://github.com/spacetelescope/RegressionTests/actions/runs/12205166696

emolter

Looks good to me now too, pending regtests

melanieclarke · 2024-12-06T20:52:01Z

Regtests show some diffs in NIRISS WFSS products, as well as the AMI, but it looks like that might be due to your CRDS delivery today, @rcooper295? Can you take a look at those and see if they are as expected, too? If so, we can merge this and okify all the NIRISS changes.

rcooper295 · 2024-12-06T20:58:31Z

Yes, I think the changes in the regtest outputs are still expected from the changes so it looks ok to me!

melanieclarke · 2024-12-06T20:59:58Z

Great! Thanks for checking. I will merge and okify.

github-actions bot added the ami label Oct 1, 2024

rcooper295 mentioned this pull request Oct 1, 2024

fix standard error of the mean #8826

Closed

10 tasks

rcooper295 force-pushed the observable_update branch from 01b21a6 to efdc204 Compare October 14, 2024 18:58

github-actions bot added the documentation label Oct 14, 2024

rcooper295 force-pushed the observable_update branch 2 times, most recently from 02f0b1c to 15d38f3 Compare November 4, 2024 18:45

braingram mentioned this pull request Nov 5, 2024

Add CALIB and wcsinfo to ami schema spacetelescope/stdatamodels#357

Merged

5 tasks

rcooper295 force-pushed the observable_update branch from 15d38f3 to 12807b2 Compare November 18, 2024 16:31

github-actions bot added the installation label Nov 18, 2024

rcooper295 force-pushed the observable_update branch from f487469 to a0bd5d0 Compare November 20, 2024 19:13

rcooper295 mentioned this pull request Nov 20, 2024

JP-3739: Update NRM geometry, affine distortion use #8974

Open

10 tasks

stscijgbot-jp mentioned this pull request Nov 21, 2024

Update AMI NRM geometry #8772

Open

rcooper295 force-pushed the observable_update branch from 6c42f01 to 4176c82 Compare November 21, 2024 22:40

github-actions bot added the testing label Nov 22, 2024

rcooper295 force-pushed the observable_update branch from 7e10b41 to caea87a Compare November 22, 2024 21:21

rcooper295 force-pushed the observable_update branch 3 times, most recently from 1d84a4c to c91b184 Compare December 4, 2024 18:35

rcooper295 marked this pull request as ready for review December 4, 2024 18:52

rcooper295 requested a review from a team as a code owner December 4, 2024 18:52

rcooper295 force-pushed the observable_update branch from ebe25cd to 64b5483 Compare December 4, 2024 19:21

melanieclarke requested review from emolter and melanieclarke December 4, 2024 22:33

melanieclarke added this to the Build 11.2 milestone Dec 4, 2024

melanieclarke reviewed Dec 5, 2024

View reviewed changes

emolter reviewed Dec 5, 2024

View reviewed changes

rcooper295 added 19 commits December 6, 2024 13:27

Calculate t3 amplitudes (untested)

99d9af1

noted where t3amps mistakenly populated with closure amps (quad amps)

edc02ae

Preliminary quad phase calculation

bde3d23

docstring correction

a0e82d7

Updates to use covariances for error estimates and calculate observab…

f869bd7

…les from already-averaged fringe phases or amplitudes, WIP

Debugging fixes, now functioning

182f1d4

convert quantities to degrees and fix squared visibility uncertainties

55cef8a

Updated to use mean by default and calculate standard error of the me…

3417d8e

…an for uncertainty estimates

Fixed input args bandpass and affine2d to expect ASDF files, updated …

61d2dca

…documentation with examples.

point pyproject requirements to stdatamodels branch adding pa, calib …

2267535

…keywords

Carry wcsinfo from input through to ami-oi output, correctly populate…

cdcee17

… calibrator_object_id in calibrated files. Use ROLL_REF instead of PA_V3, though leaving old variable names for now.

update affine log message

ac86ba1

removed unused synphot import

2190158

update tests to use roll_ref, correct throughput reffile error message

9ac6b5f

changelog file added

3c3a00f

Updates to use latest changes to stdatamodels AmiOIModel to store pos…

1a806ca

…ition angle related keywords

small docs updates

99ad0f3

stdatamodels requirement back to main

b64f3a0

Resolving issues and suggestions from PR reviews by @melanieclarke and …

7745ad2

…@emolter

rcooper295 force-pushed the observable_update branch from 64b5483 to 7745ad2 Compare December 6, 2024 18:27

melanieclarke approved these changes Dec 6, 2024

View reviewed changes

emolter approved these changes Dec 6, 2024

View reviewed changes

melanieclarke merged commit 0ad9efb into spacetelescope:main Dec 6, 2024
30 of 31 checks passed

stscijgbot-jp mentioned this pull request Dec 6, 2024

Improve AMI observable averaging, error calculations #8733

Closed

rcooper295 deleted the observable_update branch December 6, 2024 21:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JP-3718: Improve AMI observable averaging, error calculations #8846

JP-3718: Improve AMI observable averaging, error calculations #8846

rcooper295 commented Oct 1, 2024 •

edited by melanieclarke

Loading

codecov bot commented Oct 1, 2024 •

edited

Loading

rcooper295 commented Nov 20, 2024

melanieclarke commented Nov 22, 2024

melanieclarke commented Dec 4, 2024

melanieclarke left a comment

emolter left a comment

melanieclarke left a comment

melanieclarke Dec 6, 2024

melanieclarke Dec 6, 2024

melanieclarke Dec 6, 2024

melanieclarke commented Dec 6, 2024

emolter left a comment

melanieclarke commented Dec 6, 2024

rcooper295 commented Dec 6, 2024

melanieclarke commented Dec 6, 2024


		.. code-block:: python

		# Create a F480M filter + Vega bandpass ASDF file

	The inteferometric observables are saved as OIFITS files, a registered FITS format
	The interferometric observables are saved as OIFITS files, a registered FITS format

JP-3718: Improve AMI observable averaging, error calculations #8846

JP-3718: Improve AMI observable averaging, error calculations #8846

Conversation

rcooper295 commented Oct 1, 2024 • edited by melanieclarke Loading

Tasks

stage 1

stage 2

stage 3

other

codecov bot commented Oct 1, 2024 • edited Loading

Codecov Report

rcooper295 commented Nov 20, 2024

melanieclarke commented Nov 22, 2024

melanieclarke commented Dec 4, 2024

melanieclarke left a comment

Choose a reason for hiding this comment

emolter left a comment

Choose a reason for hiding this comment

melanieclarke left a comment

Choose a reason for hiding this comment

melanieclarke Dec 6, 2024

Choose a reason for hiding this comment

melanieclarke Dec 6, 2024

Choose a reason for hiding this comment

melanieclarke Dec 6, 2024

Choose a reason for hiding this comment

melanieclarke commented Dec 6, 2024

emolter left a comment

Choose a reason for hiding this comment

melanieclarke commented Dec 6, 2024

rcooper295 commented Dec 6, 2024

melanieclarke commented Dec 6, 2024

rcooper295 commented Oct 1, 2024 •

edited by melanieclarke

Loading

codecov bot commented Oct 1, 2024 •

edited

Loading