NCSS NetcdfSubset not working with v5.5

[tlvu]

To report a non-security related issue, please provide:

• the version of the software with which you are encountering an issue: unidata/thredds-docker:5.5 (https://hub.docker.com/r/unidata/thredds-docker/tags)
• environmental information (i.e. Operating System, compiler info, java version, python version, etc.): versions in the docker image
• a description of the issue with the steps needed to reproduce it

When clicking on a NCSS link (https://boreas.ouranos.ca/testthredds/ncss/testdatasets/testdata/ta_Amon_MRI-CGCM3_decadal1980_r1i1p1_199101-200012.nc/dataset.html), all I have is a 404 page.

No other errors than this /usr/local/tomcat/content/thredds/logs/threddsServlet.log:

2024-11-19T20:34:07.387 +0000 [    167870][      29] INFO  - threddsServlet - Remote host: 172.19.0.41 - Request: "GET /testthredds/ncss/testdatasets/testdata/ta_Amon_MRI-CGCM3_decadal1980_r1i1p1_199101-200012.nc/dataset.html HTTP/1.0"
2024-11-19T20:34:07.412 +0000 [    167895][      29] INFO  - threddsServlet - Request Completed - 404 - -1 - 25

For comparison, this is a working NCSS link from a Thredds 4.6 version, on the exact same .nc test file: https://boreas.ouranos.ca/twitcher/ows/proxy/thredds/ncss/birdhouse/testdata/ta_Amon_MRI-CGCM3_decadal1980_r1i1p1_199101-200012.nc/dataset.html

For 5.5, NetcdfSubsetService is enabled in our threddsConfig.xml
https://github.com/Ouranosinc/birdhouse-deploy/blob/59d4344cecd0eba79f94b5919e7140c2cba8a6b4/birdhouse/optional-components/testthredds/threddsConfig.xml.template#L133-L135

  <NetcdfSubsetService>
    <allow>true</allow>
  </NetcdfSubsetService>

The service is enabled in our catalog:
https://github.com/Ouranosinc/birdhouse-deploy/blob/59d4344cecd0eba79f94b5919e7140c2cba8a6b4/birdhouse/optional-components/testthredds/catalog.xml.template#L14

    <service name="all" serviceType="Compound" base="" >
        <service name="http" serviceType="HTTPServer" base="/${TESTTHREDDS_CONTEXT_ROOT}/fileServer/" />
        <service name="odap" serviceType="OpenDAP" base="/${TESTTHREDDS_CONTEXT_ROOT}/dodsC/" />
        <service name="ncml" serviceType="NCML" base="/${TESTTHREDDS_CONTEXT_ROOT}/ncml/"/>
        <service name="uddc" serviceType="UDDC" base="/${TESTTHREDDS_CONTEXT_ROOT}/uddc/"/>
        <service name="iso" serviceType="ISO" base="/${TESTTHREDDS_CONTEXT_ROOT}/iso/"/>
        <service name="wcs" serviceType="WCS" base="/${TESTTHREDDS_CONTEXT_ROOT}/wcs/" />
        <service name="wms" serviceType="WMS" base="/${TESTTHREDDS_CONTEXT_ROOT}/wms/" />
        <service name="subsetServer" serviceType="NetcdfSubset" base="/${TESTTHREDDS_CONTEXT_ROOT}/ncss/" />
    </service>

The service is enabled in our datasets:
https://github.com/Ouranosinc/birdhouse-deploy/blob/59d4344cecd0eba79f94b5919e7140c2cba8a6b4/birdhouse/optional-components/testthredds/catalog.xml.template#L17-L21

    <datasetScan name="TestDatasets" ID="testdatasets" path="testdatasets" location="/pavics-testdata">

      <metadata inherited="true">
        <serviceName>all</serviceName>
      </metadata>

      <filter>
        <include wildcard="*.nc" />
        <include wildcard="*.ncml" />
        <include wildcard="*.txt" />
        <include wildcard="*.md" />
        <include wildcard="*.rst" />
      </filter>

    </datasetScan>

[lesserwhirls]

The base for NCSS has changed in 5.x to distinguish between NCSS for GRIDS vs Points, which is likely why you are getting the 404s. Can you try updating the subsetServer element to:

    <service name="subsetServer" serviceType="NetcdfSubset" base="/${TESTTHREDDS_CONTEXT_ROOT}/ncss/grid/" />

[tlvu]

The base for NCSS has changed in 5.x to distinguish between NCSS for GRIDS vs Points, which is likely why you are getting the 404s. Can you try updating the subsetServer element to:

    <service name="subsetServer" serviceType="NetcdfSubset" base="/${TESTTHREDDS_CONTEXT_ROOT}/ncss/grid/" />

Thanks a bunch, it works !!!

Could the documentation here https://docs.unidata.ucar.edu/tds/current/userguide/adding_ncss.html be updated? This is all we see right now and I would never have guessed it.

Also I enabled simultaneously both grid and point this way, is this the correct way?

    <service name="all" serviceType="Compound" base="" >
        <service name="http" serviceType="HTTPServer" base="/${TESTTHREDDS_CONTEXT_ROOT}/fileServer/" />
        (...)
        <service name="subsetServerGrid" serviceType="NetcdfSubset" base="/${TESTTHREDDS_CONTEXT_ROOT}/ncss/grid/" />
        <service name="subsetServerPoint" serviceType="NetcdfSubset" base="/${TESTTHREDDS_CONTEXT_ROOT}/ncss/point/" />
    </service>

It seems to work on my test instance https://boreas.ouranos.ca/testthredds/catalog/testdatasets/testdata/catalog.html?dataset=testdatasets/testdata/ta_Amon_MRI-CGCM3_decadal1980_r1i1p1_199101-200012.nc

[lesserwhirls]

I responded over on the PR (#545), but in case anyone stumbles across this issue, I'll repost the important parts:

[m]ixed Point and Grid FeatureType datasets (see these netCDF-Java docs for more info) are not supported in NCSS. NCSS still supports point requests against gridded data (e.g. extract time series data from a grid point), but that all falls under the /thredds/ncss/grid service for "gridded" data.

So you would enable one or the other based on the type of data in the dataset (a very simplified way of looking at it - model output or observational data), but not both.

[tlvu]

Thanks @lesserwhirls, closing this issue, fixed by #545.