nxdl2rst.py: document all the new anchors in one page

[prjemian]

Build a documentation page (separate? not part of the manual?), organized by keyword and context, listing all the anchors defined by #878.

For example, given the reST anchor /NXdata/VARIABLE@last_good-attribute (corresponds with HTML anchor #nxdata-variable-last-good-attribute), the keyword would be @last_good (converted to lower case) and the context would be /NXdata/VARIABLE@last_good-attribute with a link to the manual page anchor. Any keyword with multiple contexts would be listed together, such as

• data
  • /NXdata/DATA-field
  • /NXdetector/data-field
  • /NXentry/DATA-group
    ..

Consider registering this list (2,517 items now) with an external registry, such as https://www.pidconsortium.net/. More details

[prjemian]

This code:

definitions/utils/nxdl2rst.py

Lines 29 to 31 in a841cc2

	def addAnchor(anchor):
	"""Add a hypertext anchor to the list."""
	anchor_list.append(anchor)

is the place to gather the reST anchor points from a single NXDL file. The nxdl2rst.py program is run once for each NXDL file when building the documentation.

[prjemian]

As discussed in today's telco, we could create a new repository in this GitHub organization to host the script-generated ontology files. This would happen on every push to the main branch. Tags in this repo would be created on the ontology repo as well. Tags in the ontology repo could push the versioned ontology to a public service so that is findable.

The published ontology describes the NeXus standard, it does not define the standard.

[prjemian]

With #922, .owl file(s) will be created but still that does not describe how this content will be visualized. So this issue cannot be closed yet.

[prjemian]

Looks like another type of index section.

[prjemian]

Will add a page that provides this content via a download access so the full table (>2500 entries as of now) does not appear in the manual directly. Will provide content in various formats including: json, yaml, and possibly xml.

Content will organize by keys (as described above). Underneath, will provide all anchors using that key in at least two forms: full text anchor and URL to the documentation page

The files should also include some metadata about the date/time of creation and the git commit hash. Basically something to inform about which version of the definitions is described by this content.

[prjemian]

This code stores these files in the _static directory of the manual source. When Sphinx builds the documentation, it copies these files to the build directory.

        path = repo_root_path / "manual" / "source" / "_static"
        self.all_anchors = path / "all_anchors.txt"
        self.json_file = path / "anchors.json"
        self.xml_file = path / "anchors.xml"
        self.yaml_file = path / "anchors.yml"

[prjemian]

Also, the all_anchors.txt file is a complete list of all hypertext anchors:

definitions/utils/nxdl2rst.py

Line 32 in ed6f9fe

anchor_list.append(anchor)

[prjemian]

Possibly related to:

• Extend HTML documentation of base classes for overrides #1057
• Are the new doc strings extending (specialising) or overriding the inherited doc? #1059

[prjemian]

Now, my PR is collecting information about each NXDL anchors just before it writes the anchors for that page. Will we need to move this step, collecting the page anchors, before the page content is written?

[prjemian]

Collect all page anchors first, then write the rst for the page, list of anchors goes at the page bottom.

[prjemian]

Here is a snippet:

json

        "pixel_mask": {
            "/NXdetector/pixel_mask-field": {
                "anchor": "/NXdetector/pixel_mask-field",
                "html": "#nxdetector-pixel-mask-field",
                "url": "https://manual.nexusformat.org/classes/base_classes/NXdetector.html#nxdetector-pixel-mask-field"
            },
            "/NXmx/ENTRY/INSTRUMENT/DETECTOR/pixel_mask-field": {
                "anchor": "/NXmx/ENTRY/INSTRUMENT/DETECTOR/pixel_mask-field",
                "html": "#nxmx-entry-instrument-detector-pixel-mask-field",
                "url": "https://manual.nexusformat.org/classes/applications/NXmx.html#nxmx-entry-instrument-detector-pixel-mask-field"
            }
        },
        "pixel_mask_applied": {
            "/NXdetector/pixel_mask_applied-field": {
                "anchor": "/NXdetector/pixel_mask_applied-field",
                "html": "#nxdetector-pixel-mask-applied-field",
                "url": "https://manual.nexusformat.org/classes/base_classes/NXdetector.html#nxdetector-pixel-mask-applied-field"
            },
            "/NXmx/ENTRY/INSTRUMENT/DETECTOR/pixel_mask_applied-field": {
                "anchor": "/NXmx/ENTRY/INSTRUMENT/DETECTOR/pixel_mask_applied-field",
                "html": "#nxmx-entry-instrument-detector-pixel-mask-applied-field",
                "url": "https://manual.nexusformat.org/classes/applications/NXmx.html#nxmx-entry-instrument-detector-pixel-mask-applied-field"
            }
        },
        "point_group": {
            "/NXsample/point_group-field": {
                "anchor": "/NXsample/point_group-field",
                "html": "#nxsample-point-group-field",
                "url": "https://manual.nexusformat.org/classes/base_classes/NXsample.html#nxsample-point-group-field"
            },
            "/NXsample_component/point_group-field": {
                "anchor": "/NXsample_component/point_group-field",
                "html": "#nxsample-component-point-group-field",
                "url": "https://manual.nexusformat.org/classes/base_classes/NXsample_component.html#nxsample-component-point-group-field"
            }
        },

yaml

  pixel_mask:
    /NXdetector/pixel_mask-field:
      anchor: /NXdetector/pixel_mask-field
      html: '#nxdetector-pixel-mask-field'
      url: https://manual.nexusformat.org/classes/base_classes/NXdetector.html#nxdetector-pixel-mask-field
    /NXmx/ENTRY/INSTRUMENT/DETECTOR/pixel_mask-field:
      anchor: /NXmx/ENTRY/INSTRUMENT/DETECTOR/pixel_mask-field
      html: '#nxmx-entry-instrument-detector-pixel-mask-field'
      url: https://manual.nexusformat.org/classes/applications/NXmx.html#nxmx-entry-instrument-detector-pixel-mask-field
  pixel_mask_applied:
    /NXdetector/pixel_mask_applied-field:
      anchor: /NXdetector/pixel_mask_applied-field
      html: '#nxdetector-pixel-mask-applied-field'
      url: https://manual.nexusformat.org/classes/base_classes/NXdetector.html#nxdetector-pixel-mask-applied-field
    /NXmx/ENTRY/INSTRUMENT/DETECTOR/pixel_mask_applied-field:
      anchor: /NXmx/ENTRY/INSTRUMENT/DETECTOR/pixel_mask_applied-field
      html: '#nxmx-entry-instrument-detector-pixel-mask-applied-field'
      url: https://manual.nexusformat.org/classes/applications/NXmx.html#nxmx-entry-instrument-detector-pixel-mask-applied-field
  point_group:
    /NXsample/point_group-field:
      anchor: /NXsample/point_group-field
      html: '#nxsample-point-group-field'
      url: https://manual.nexusformat.org/classes/base_classes/NXsample.html#nxsample-point-group-field
    /NXsample_component/point_group-field:
      anchor: /NXsample_component/point_group-field
      html: '#nxsample-component-point-group-field'
      url: https://manual.nexusformat.org/classes/base_classes/NXsample_component.html#nxsample-component-point-group-field

[prjemian]

• change anchors file names to vocabulary
• internally, change anchors to terms
• build an html version (all in one page) try not to put this in the PDF
• XML version not necessary

[prjemian]

To avoid publishing the HTML version of the vocabulary in the PDF, the HTML will be written to the _static directory, next to the JSON, text, and YAML files.

file	description
_static/nxdl_vocabulary.html	Human-readable and clickable HTML list of anchors, by vocabulary term
_static/nxdl_vocabulary.json	vocabulary list by key in JSON format
_static/nxdl_vocabulary.txt	list of all anchors, sorted alphabetically
_static/nxdl_vocabulary.yml	vocabulary list by key in YAML format

• Need to refer to these files (as downloads) from some place(s) in the manual.

[prjemian]

On the classes page, add the table above with links to download each file.