JP-2259: Tweakreg source finding (#8203)

Co-authored-by: Ned Molter <emolter@stsci.edu>
Co-authored-by: Howard Bushouse <bushouse@stsci.edu>

CHANGES.rst

@@ -68,6 +68,9 @@ tweakreg
 
 - Update ``sregion`` after WCS corrections are applied. [#8158]
 
+- Added option to choose IRAFStarFinder and segmentation.SourceFinder
+  instead of DAOStarFinder and exposed star finder parameters. [#8203]
+
 
 1.13.3 (01-05-2024)
 ===================

docs/jwst/tweakreg/README.rst

@@ -13,29 +13,61 @@ image catalogs using the corrected WCS will align on the sky.
 
 Source Detection
 ----------------
-If ``meta.tweakreg_catalog`` attribute of input data models is a non-empty
-string and ``use_custom_catalogs`` is `True`, then it will be interpretted
+If the ``meta.tweakreg_catalog`` attribute of input data models is a non-empty
+string and ``use_custom_catalogs`` is `True`, then it will be interpreted
 as a file name of a user-provided source catalog. The catalog must be in a
 format automatically recognized by :py:meth:`~astropy.table.Table.read`.
 
-When ``meta.tweakreg_catalog`` attribute of input data models is `None` or
-an empty string, then ``tweakreg`` step will attempt to detect sources in the
-input images. Stars are detected in the image using the Photutils "daofind"
-function. Photutils.daofind is an implementation of the `DAOFIND`_ algorithm
+When the ``meta.tweakreg_catalog`` attribute of input data models is `None` or
+an empty string, then the ``tweakreg`` step will attempt to detect sources in the
+input images. Stars are detected in the image with one of the following source
+detection algorithms: ``photutils.detection.DAOStarFinder`` (default), 
+``photutils.detection.IRAFStarFinder``, or ``photutils.segmentation.SourceFinder`` 
+in conjunction with ``photutils.segmentation.SourceCatalog``.
+
+DAOStarFinder is an implementation of the `DAOFIND`_ algorithm
 (`Stetson 1987, PASP 99, 191
 <http://adsabs.harvard.edu/abs/1987PASP...99..191S>`_).  It searches
 images for local density maxima that have a peak amplitude greater
 than a specified threshold (the threshold is applied to a convolved
 image) and have a size and shape similar to a defined 2D Gaussian
-kernel.  ``photutils.daofind`` also provides an estimate of the objects
+kernel.  DAOFind also provides an estimate of the object's
 roundness and sharpness, whose lower and upper bounds can be
 specified.
 
+IRAFStarFinder is a Python implementation of the IRAF star finding algorithm,
+which also calculates the objects' centroids, roundness, and sharpness.
+However, IRAFStarFinder uses image moments
+instead of 1-D Gaussian fits to projected light distributions like 
+DAOStarFinder.
+
+SourceFinder implements a segmentation algorithm that identifies
+sources in an image based on a number of connected pixels above a 
+specified threshold value.  The sources are deblended using a 
+combination of multi-thresholding and watershed segmentation. 
+SourceCatalog finds the centroids of these sources, which are used
+as the retrieved star positions. 
+
+.. warning::
+    It has been shown (`STScI Technical Report JWST-STScI-008116, SM-12 
+    <https://www.stsci.edu/~goudfroo/NIRISSdoc/Centroid_Accuracies_Precisions_NIRISS_v2.pdf>`_) 
+    that for undersampled PSFs, e.g. for short-wavelength NIRISS
+    imaging data, ``DAOStarFinder`` gives bad results no matter the input parameters
+    due to its use of 1-D Gaussian fits.
+    ``IRAFStarFinder`` or ``SourceFinder`` should be used instead.
+
+.. note:: 
+    ``SourceFinder`` is likely to detect non-stellar sources 
+    such as galaxies because sources are not assumed to be
+    point-source-like. This may lead to mismatches between the
+    derived source catalog and the reference catalog during the
+    alignment step.
+
 .. _DAOFIND: http://stsdas.stsci.edu/cgi-bin/gethelp.cgi?daofind
 
 Custom Source Catalogs
 ----------------------
-Source detection built-in into the ``tweakreg`` step can be disabled by
+Source detection built into the ``tweakreg`` step can be disabled by
 providing a file name to a custom source catalog in the
 ``meta.tweakreg_catalog`` attribute of input data models.
 The catalog must be in a format automatically recognized by
@@ -63,7 +95,7 @@ other ways of supplying custom source catalogs to the step:
    Catalog file names are relative to ``catfile`` file path.
 
 Specifying custom source catalogs via either the input ASN file or
-``catfile``, will update input data models' ``meta.tweakreg_catalog``
+``catfile`` will update input data models' ``meta.tweakreg_catalog``
 attributes to the catalog file names provided in either in the ASN file or
 ``catfile``.
 
@@ -76,12 +108,12 @@ attributes to the catalog file names provided in either in the ASN file or
     1. Providing a data model file name in the ``catfile`` and leaving
        the corresponding source catalog file name empty -- same as setting
        ``'tweakreg_catalog'`` in the ASN file to an empty string ``""`` --
-       would set corresponding input data model's ``meta.tweakreg_catalog``
+       would set the corresponding input data model's ``meta.tweakreg_catalog``
        attribute to `None`. In this case, ``tweakreg_step`` will automatically
        generate a source catalog for that data model.
 
     2. If an input data model is not listed in the ``catfile`` or does not
-       have ``'tweakreg_catalog'`` attribute provided in the ASN file,
+       have the ``'tweakreg_catalog'`` attribute provided in the ASN file,
        then the catalog file name in that model's ``meta.tweakreg_catalog``
        attribute will be used. If ``model.meta.tweakreg_catalog`` is `None`,
        ``tweakreg_step`` will automatically generate a source catalog for
@@ -96,7 +128,7 @@ are aligned relative to each other.  This step produces a combined
 source catalog for the entire set of input images as if they were
 combined into a single mosaic.
 
-If the step parameter ``abs_refcat`` is set to 'GAIADR3', 'GAIADR2' or 'GAIADR1',
+If the step parameter ``abs_refcat`` is set to 'GAIADR3', 'GAIADR2', or 'GAIADR1',
 an astrometric reference catalog then gets generated by querying
 a GAIA-based astrometric catalog web service for all astrometrically
 measured sources in the combined field-of-view of the set of input
@@ -117,8 +149,8 @@ maintaining the relative alignment between the images.
 
 For this part of alignment, instead of 'GAIADR1', 'GAIADR2', or 'GAIADR3', users can
 supply an external reference catalog by providing a path to an existing
-file. User-supplied catalog must contain ``'RA'`` and ``'DEC'`` columns which
-indicate reference source world coordinates (in degrees). An optional column
+file. A user-supplied catalog must contain ``'RA'`` and ``'DEC'`` columns
+indicating reference source world coordinates (in degrees). An optional column
 in the catalog is the ``'weight'`` column, which when present, will be used
 in fitting. The catalog must be in a format automatically recognized by
 :py:meth:`~astropy.table.Table.read`.
@@ -127,7 +159,7 @@ Grouping
 --------
 
 Images taken at the same time (e.g., NIRCam images from all short-wave
-detectors) can be aligned together, that is, a single correction
+detectors) can be aligned together; that is, a single correction
 can be computed and applied to all these images because any error in
 telescope pointing will be identical in all these images and it is assumed
 that the relative positions of (e.g., NIRCam) detectors do not change.
@@ -169,24 +201,38 @@ The ``tweakreg`` step has the following optional arguments:
 * ``save_catalogs``: A boolean indicating whether or not the catalogs should
   be written out. This parameter is ignored for input data models whose
   ``meta.tweakreg_catalog`` is a non-empty string pointing to a user-supplied
-  source catalog. (Default=`False`)
+  source catalog. (Default=False)
 
 * ``use_custom_catalogs``: A boolean that indicates whether
   to ignore source catalog in the input data model's ``meta.tweakreg_catalog``
   attribute. If `False`, new catalogs will be generated by the ``tweakreg``
-  step. (Default=`False`)
+  step. (Default=False)
 
 * ``catalog_format``: A `str` indicating catalog output file format.
-  (Default='ecsv')
+  (Default= `'ecsv'`)
 
 * ``catfile``: Name of the file with a list of custom user-provided catalogs.
-  (Default='')
+  (Default= `''`)
+
+* ``bkg_boxsize``: A positive `int` indicating the background mesh box size
+  in pixels. (Default=400)
+
+* ``starfinder``: A `str` indicating the source detection algorithm to use.
+  Allowed values: `'iraf'`, `'dao'`, `'segmentation'`. (Default= `'dao'`)
+
+* ``snr_threshold``: A `float` value indicating SNR threshold above the
+  background. Required for all star finders. (Default=10.0)
+
+**Additional source finding parameters for DAO and IRAF:**
 
 * ``kernel_fwhm``: A `float` value indicating the Gaussian kernel FWHM in
   pixels. (Default=2.5)
 
-* ``snr_threshold``: A `float` value indicating SNR threshold above the
-  background. (Default=5.0)
+* ``minsep_fwhm``: A `float` value indicating the minimum separation between
+  detected objects in units of number of FWHMs. (Default=0.0)
+
+* ``sigma_radius``: A `float` value indicating the truncation radius of the
+  Gaussian kernel in units of number of FWHMs. (Default=2.5)
 
 * ``sharplo``: A `float` value indicating The lower bound on sharpness
   for object detection. (Default=0.2)
@@ -197,24 +243,52 @@ The ``tweakreg`` step has the following optional arguments:
 * ``roundlo``: A `float` value indicating the lower bound on roundness
   for object detection. (Default=-1.0)
 
-* ``roundhi``: `float` value indicating the upper bound on roundness
+* ``roundhi``: A `float` value indicating the upper bound on roundness
   for object detection. (Default=1.0)
 
 * ``brightest``: A positive `int` value indicating the number of brightest
-  objects to keep. (Default=200)
+  objects to keep. If None, keep all objects above the threshold. (Default=200)
 
 * ``peakmax``: A `float` value used to filter out objects with pixel values
   >= ``peakmax``. (Default=None)
 
-* ``bkg_boxsize``: A positive `int` indicating the background mesh box size
-  in pixels. (Default=400)
+**Additional source finding parameters for segmentation:**
+
+* ``npixels``: An `int` value indicating the minimum number of 
+  connected pixels that comprises a segment (Default=10)
+
+* ``connectivity``: An `int` value indicating the connectivity defining the
+  neighborhood of a pixel. Options are `4`, i.e., connected pixels touch along edges,
+  or `8`, i.e, connected pixels touch along edges or corners (Default=8)
+
+* ``nlevels``: An `int` value indicating the number of multi-thresholding
+  levels for deblending (Default=32)
+
+* ``contrast``: A `float` value indicating the fraction of total source flux
+  an object must have to be deblended (Default=0.001)
+
+* ``multithresh_mode``: A `str` indicating the multi-thresholding mode.
+  Allowed values: `'exponential'`, `'linear'`, `'sinh'`.
+  (Default= `'exponential'`)
+
+* ``localbkg_width``: An `int` value indicating the width of rectangular
+  annulus used to compute local background around each source. If set to 0,
+  then local background will not be subtracted. (Default=0)
+
+* ``apermask_method``: A `str` indicating the method used to handle
+  neighboring sources when performing aperture photometry.
+  Allowed values: `'correct'`, `'mask'`, `'none'`. (Default= `'correct'`)
+
+* ``kron_params``: A tuple of `float` values indicating the
+  parameters defining Kron aperture. If None, 
+  the parameters `(2.5, 1.4, 0.0)` are used. (Default=None)
 
 **Optimize alignment order:**
 
 * ``enforce_user_order``: a boolean value indicating whether or not take the
   first image as a reference image and then align the rest of the images
   to that reference image in the order in which input images have been provided
-  or to optimize order in which images are aligned. (Default=`False`)
+  or to optimize order in which images are aligned. (Default=False)
 
 **Reference Catalog parameters:**
 
@@ -269,7 +343,7 @@ The ``tweakreg`` step has the following optional arguments:
       spread across/covering the entire image).
 
 * ``nclip``: A non-negative integer number of clipping iterations
-  to use in the fit. (Default = 3)
+  to use in the fit. (Default=3)
 
 * ``sigma``: A positive `float` indicating the clipping limit, in sigma units,
   used when performing fit. (Default=3.0)
@@ -280,13 +354,13 @@ Parameters used for absolute astrometry to a reference catalog.
 
 * ``abs_refcat``: String indicating what astrometric catalog should be used.
   Currently supported options: 'GAIADR1', 'GAIADR2', 'GAIADR3', a path to an existing
-  reference catalog, `None`, or ''. See
+  reference catalog, `None`, or `''`. See
   :py:data:`jwst.tweakreg.tweakreg_step.SINGLE_GROUP_REFCAT`
   for an up-to-date list of supported built-in reference catalogs.
 
   When ``abs_refcat`` is `None` or an empty string, alignment to the
   absolute astrometry catalog will be turned off.
-  (Default='')
+  (Default= `''`)
 
 * ``abs_minobj``: A positive `int` indicating minimum number of objects
   acceptable for matching. (Default=15)
@@ -335,6 +409,19 @@ in more detail at
 
 https://tweakwcs.readthedocs.io/en/latest/
 
+Further description of the input parameters and algorithms for star finding
+can be found at the following links:
+
+* `DAOStarFinder`_
+* `IRAFStarFinder`_
+* `SourceFinder`_
+* `SourceCatalog`_
+
+.. _DAOStarFinder: https://photutils.readthedocs.io/en/stable/api/photutils.detection.DAOStarFinder.html
+.. _IRAFStarFinder: https://photutils.readthedocs.io/en/stable/api/photutils.detection.IRAFStarFinder.html
+.. _SourceFinder: https://photutils.readthedocs.io/en/stable/api/photutils.segmentation.SourceFinder.html
+.. _SourceCatalog: https://photutils.readthedocs.io/en/stable/api/photutils.segmentation.SourceCatalog.html
+
 
 Reference Files
 ===============

jwst/regtest/test_nircam_align_to_gaia.py

@@ -3,7 +3,6 @@
 from numpy.testing import assert_allclose
 
 from stdatamodels.jwst import datamodels
-
 from jwst.stpipe import Step
 
 
@@ -16,7 +15,7 @@ def run_image3pipeline(rtdata_module, jail):
     args = ["calwebb_image3", rtdata.input,
             "--steps.tweakreg.abs_refcat=GAIADR2",
             "--steps.tweakreg.save_results=True",
-            "--steps.tweakreg.output_use_model=True"
+            "--steps.tweakreg.output_use_model=True",
             ]
     Step.from_cmdline(args)
 

jwst/regtest/test_niriss_sourcefind.py

@@ -0,0 +1,32 @@
+import pytest
+from astropy.io import ascii
+from numpy.testing import assert_allclose
+
+from stdatamodels.jwst import datamodels
+from jwst.tweakreg import tweakreg_catalog
+
+
+@pytest.mark.bigdata
+@pytest.mark.parametrize("starfinder", ["iraf", "segmentation"], )
+def test_tweakreg_catalog_starfinder_alternatives(rtdata, starfinder):
+    '''
+    Test that the IRAF and segmentation star finders give expected results for undersampled NIRISS data
+    It is well known that DAOStarFinder gives bad results so is not included in this test
+    '''
+
+    stem = "jw01088003001_01101_00005"
+    rtdata.get_data(f"niriss/imaging/{stem}_nis_cal.fits")
+    model = datamodels.ImageModel(rtdata.input)
+    catalog = tweakreg_catalog.make_tweakreg_catalog(
+        model, 2.5, 10.0, starfinder=starfinder, starfinder_kwargs={
+            'brightest': None,
+            'sharphi': 3.0,
+            'minsep_fwhm': 2.5,
+            'sigma_radius': 2.5,
+        })
+    rtdata.get_truth(f"truth/test_niriss_sourcefind/{stem}_{starfinder}_cat.ecsv")
+    catalog_truth = ascii.read(rtdata.truth)
+
+    # rtol is larger than default because of numerical differences on Linux vs MacOS
+    assert_allclose(catalog['xcentroid'], catalog_truth['xcentroid'], rtol=1e-3)
+    assert_allclose(catalog['ycentroid'], catalog_truth['ycentroid'], rtol=1e-3)

jwst/tweakreg/tests/test_tweakreg.py

@@ -6,6 +6,7 @@
 import pytest
 
 from jwst.tweakreg import tweakreg_step
+from jwst.tweakreg import tweakreg_catalog
 from stdatamodels.jwst.datamodels import ImageModel
 
 
@@ -55,3 +56,10 @@ def test_common_name(groups, all_group_names, common_name):
 
         cn = tweakreg_step._common_name(group, all_group_names)
         assert cn == cn_truth
+
+
+def test_expected_failure_bad_starfinder():
+
+    model = ImageModel()
+    with pytest.raises(ValueError):
+        tweakreg_catalog.make_tweakreg_catalog(model, 5.0, bkg_boxsize=400, starfinder='bad_value')