Merge remote-tracking branch 'origin/master' into nemo_v5

.github/workflows/compilation.yml

@@ -51,13 +51,13 @@ on:
   push
 
 env:
-  CUDA_VERSION: 12.6.2
+  CUDA_VERSION: 12.6.3
   GFORTRAN_VERSION: 14.2.0
   HDF5_VERSION: 1.14.5
   NETCDF_C_VERSION: 4.9.2
   NETCDF_FORTRAN_VERSION: 4.6.1
-  NVFORTRAN_VERSION: 24.9
-  OPENMPI_VERSION: 5.0.5
+  NVFORTRAN_VERSION: 24.11
+  OPENMPI_VERSION: 5.0.6
   PYTHON_VERSION: 3.13.0
 
 jobs:

.gitignore

@@ -2,11 +2,13 @@
 *~
 *.a
 *.o
+*.psycache
 gnu_opt_report.txt
 *.mod
 *.MOD
 .cache
 .coverage
+*.swp
 doc/*/_build
 doc/*/*/_build
 doc/reference_guide/source/autogenerated/
@@ -25,3 +27,5 @@ src/*.egg-info
 .idea
 .rtx.toml
 .venv
+cov.xml
+.coverage.*

README.md

@@ -50,6 +50,10 @@ cloning this repository and using:
 
     $ pip install .
 
+or in developer (editable) mode using
+
+    $ pip install -e .
+
 For more information about the installation process see
 [this section of the User Guide](https://psyclone.readthedocs.io/en/latest/system_specific_setup.html).
 

changelog

@@ -1,3 +1,5 @@
+release 3.0.0 6th of December 2024
+
 	1) PR #2477 for #2463. Add support for Fortran Namelist statements.
 
 	2) PR #2367 for #2296. Extends ModuleInfo so that it can be queried
@@ -225,8 +227,8 @@
 
 	77) PR #2708 for #2663. Add LFRic OpenACC in the integration tests.
 
-	78) PR #2678 for #2636. Alter LFRic PSy-layer to lookup nlevels for
-	each kernel
+	78) PR #2678 for #2636. Alter LFRic PSy-layer to lookup nlayers for
+	each kernel.
 
 	79) PR #2687 for #2684. Add support for WHERE constructs that contain
 	references without range-notation (and prevent some incorrect cases).
@@ -247,7 +249,7 @@
 	Python used in the CI.
 
 	85) PR #2748. Revert Python used on RTD to 3.12 as 3.13 unsupported.
-	
+
 	86) PR #2707 for #257. Fixes bugs in the loop-fusion transformation.
 
 	87) PR #2752 towards #2717. Adds Assignment.is_literal_assignment property.
@@ -298,10 +300,43 @@
 	113) PR #2769 for #2768. Generalises Sign2CodeTrans and Abs2CodeTrans
 	to allow non-floating point scalar literals.
 
-	114) PR #2697 towards #2671. Fixes various issues relating to incorrect
+	114) PR #2461 for #2460. Renames DynamoPSy class to LFRicPSy and moves
+	into a new module lfric_psy.py in domain/lfric.
+
+	115) PR #2697 towards #2671. Fixes various issues relating to incorrect
 	parallelisation of loops in NEMO. Also adds support for array-
 	privatisation option.
 
+	116) PR #2804 towards #2737. Update PSyclone to use fparser 0.2.0 release.
+
+	117) PR #2803 for #2796. Fixes bug where PSyData nodes inserted within
+	LFRic loops did not have the associated variable declared.
+
+	118) PR #2792 for # 2766. Updates to all the examples and test
+	infrastructure now that the index ordering for operators in LFRic
+	has been changed.
+
+	119) PR #2775. Adds Call.get_callee() which matches the arguments of
+	a Call against the signature of potential callees.
+
+	120) PR #2628 for #2624. Improves debug_string output for Schedules.
+
+	121) PR #2800. Updates integration test dependencies and documentation
+	about compiling for GPUs.
+
+	122) PR #2811. Adds sphinx-autodoc-typehints package to 'doc'
+	dependencies so that Sphinx markup is able to make use of the
+	typehints that we are starting to add to the code base.
+
+	123) PR #2809 for #2737. Updates documentation on OpenMP to point to
+	examples of tranformations for CPU and GPU.
+
+	124) PR #2815 for #2636. Get nlayers from first field or operator
+	argument to each kernel in LFRic.
+
+	125) PR #2817 for #2816. Fix index type in initialisation of arrays
+	of fields in PSyAD test harness construction.
+
 release 2.5.0 14th of February 2024
 
 	1) PR #2199 for #2189. Fix bugs with missing maps in enter data

doc/developer_guide/conf.py

@@ -43,6 +43,7 @@
 extensions = ['sphinx.ext.autodoc', 'sphinx.ext.doctest',
               'sphinx.ext.intersphinx', 'sphinx.ext.coverage',
               'sphinx.ext.imgmath', 'sphinx.ext.viewcode',
+              'sphinx_autodoc_typehints',
               'sphinxcontrib.bibtex', 'apilinks']
 bibtex_bibfiles = ['../bibliography/references.bib']
 

doc/developer_guide/system_specific_setup.rst

@@ -64,11 +64,11 @@ Installing Documentation Tools
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Install Sphinx along with bibtex support for creating PSyclone documentation::
 
-   > sudo pip install sphinx sphinxcontrib.bibtex
+   > pip install .[doc]
 
-You can now build html documentation::
+You can now build html documentation. E.g., for the developer documentation::
 
-   > cd doc
+   > cd doc/developer_guide/
    > make html
 
 The latex package is required to create the pdf documentation

doc/developer_guide/working_practises.rst

@@ -216,6 +216,12 @@ Additionally html output can be created by adding the option ``--cov-report html
 The html output can be viewed with a browser at ``file:///.../tests/htmlcov/index.html``
 and it highlights all source lines in red that are not covered by at least one test.
 
+For convenience, there is also a script in ``<PSYCLONEHOME>/utils/run_pytest_cov.sh``
+running the coverage tests for all relevant files.
+This also creates a ``cov.xml`` file which can be used by, e.g., vscode.
+See the respective comments in this script for further information.
+
+
 .. _parallel_execution:
 
 Parallel execution
@@ -442,8 +448,8 @@ computational cost (so that we 'fail fast'):
  1. All examples in the Developer Guide are checked for correctness by
     running ``make doctest``.
 
- 2. The code base, examples and tutorials are lint'ed with flake8.
-    (Configuration of flake8 is performed in ``setup.cfg``.)
+ 2. The code base, examples and tutorials are Lint'ed with flake8, see
+    also :ref:`flake8`.
 
  3. All links within the Sphinx documentation (rst files) are checked (see
     note below);
@@ -599,6 +605,29 @@ the testing system, and some results may be impacted by other users using the
 system at the same time.
 
 
+.. _flake8:
+
+Flake8
+------------------
+
+All code is linted by flake8 and must be free from errors to be merged with
+master.
+It can be installed with::
+  
+  > pip install flake8
+
+and executed with::
+
+  > flake8 src/psyclone
+
+An existing configuration of flake8 is given in ``setup.cfg``.
+A check with flake8 is performed automatically by the CI and it can be useful
+to setup a git pre-push hook at ``.git/hooks/pre-push`` to do this check before
+pushing to the repository.
+For convenience, such a script where a such a hook can be linked to is available
+in ``utils/run_flake8.sh``.
+
+
 Performance
 ===========
 

doc/psyad/user_guide/conf.py

@@ -36,6 +36,7 @@
 # ones. The definition of 'apilinks' is in PSyclone/doc/_ext/apilinks.py.
 extensions = [
     'sphinx.ext.imgmath', 'sphinx.ext.autodoc', 'sphinx.ext.intersphinx',
+    'sphinx_autodoc_typehints',
     'apilinks']
 
 

doc/reference_guide/doxygen.config

@@ -38,7 +38,7 @@ PROJECT_NAME           = "Reference Guide"
 # could be handy for archiving the generated documentation or if some version
 # control system is used.
 
-PROJECT_NUMBER         = 2.5.0
+PROJECT_NUMBER         = 3.0.0
 
 # Using the PROJECT_BRIEF tag one can provide an optional one line description
 # for a project that appears at the top of each page and should give viewer a

doc/reference_guide/source/conf.py

@@ -62,6 +62,7 @@
     'sphinx.ext.githubpages',
     'sphinx.ext.autodoc',
     'sphinx.ext.inheritance_diagram',
+    'sphinx_autodoc_typehints',
     'autoapi.sphinx'
 ]
 

doc/user_guide/conf.py

@@ -40,6 +40,7 @@
               'sphinx.ext.intersphinx', 'sphinx.ext.coverage',
               'sphinx.ext.imgmath', 'sphinx.ext.viewcode',
               'sphinxcontrib.bibtex', 'sphinx_tabs.tabs',
+              'sphinx_autodoc_typehints',
               'apilinks',
               ]
 bibtex_bibfiles = ['../bibliography/references.bib']
@@ -245,9 +246,10 @@
 # (source start file, target name, title, author,
 #  documentclass [howto/manual]).
 latex_documents = [
-    ('index', 'psyclone.tex', 'PSyclone Documentation',
-     'Andrew Coughtrie, Rupert Ford, Joerg Henrichs, Iva Kavcic,\\\\ '
-     'Andrew Porter, Sergi Siso and Joseph Wallwork', 'manual'),
+    ('index', 'psyclone.tex', 'PSyclone User Guide',
+     'Oakley Brunt, Andrew Coughtrie, Rupert Ford, \\\\ '
+     'Joerg Henrichs, Iva Kavcic, Andrew Porter, Sergi Siso \\\ '
+     'and Joseph Wallwork', 'manual'),
 ]
 
 # Set maximum depth for the nested lists to prevent LaTeX

doc/user_guide/dynamo0p3.rst

@@ -1938,7 +1938,7 @@ conventions, are:
 2) Include ``nlayers``, the number of layers in a column. ``nlayers``
    is an ``integer`` of kind ``i_def`` and has intent ``in``. PSyclone
    will obtain the value of ``nlayers`` to use for a particular kernel
-   from the first field (in the argument list) that is written to.
+   from the first field or operator in the argument list.
 3) For each scalar/field/vector_field/operator in the order specified by
    the meta_args metadata:
 
@@ -1989,11 +1989,10 @@ conventions, are:
       is a rank-3, ``real`` array. Its precision (kind) depends on how
       it is defined in the algorithm layer, see the
       :ref:`lfric-mixed-precision` section for more details. The
-      extents of the first two dimensions are the local degrees of
+      extent of the first dimension is ``<operator_name>"_ncell_3d"``,
+      and of the second and third dimension are the local degrees of
       freedom for the ``to`` and ``from`` function spaces,
-      respectively, and that of the third is
-      ``<operator_name>"_ncell_3d"``. The name of the operator is
-      ``"op_"<argument_position>``. Again the intent is determined
+      respectively. Again the intent is determined
       from the metadata (see :ref:`lfric-api-meta-args`).
 
 4) For each function space in the order they appear in the metadata arguments
@@ -2255,10 +2254,10 @@ as the number of DoFs for each of the dofmaps. The full set of rules is:
 5) For each argument in the ``meta_args`` metadata array:
 
    1) If it is a LMA operator, include a ``real``, 3-dimensional
-      array. The first two dimensions are the local degrees of freedom
-      for the ``to`` and ``from`` spaces, respectively. The third
-      dimension is ``ncell_3d``. The precision of the array depends on
-      how it is defined in the algorithm layer, see the
+      array. The first dimension is ``ncell_3d``. The second and third
+      dimension are the local degrees of freedom for the ``to`` and
+      ``from`` spaces, respectively.  The precision of the array depends
+      on how it is defined in the algorithm layer, see the
       :ref:`lfric-mixed-precision` section for more details;
 
    2) If it is a CMA operator, include a ``real``, 3-dimensional array

doc/user_guide/dynamo0p3_topclasses.dot

@@ -5,7 +5,7 @@ rankdir=BT
 "20" [label="{LFRicInvoke|\l|arg_for_funcspace()\lfield_on_space()\lgen_code()\lis_coloured()\lunique_fss()\lunique_proxy_declarations()\l}", shape="record"];
 "21" [label="{LFRicInvokeSchedule|\l|view()\l}", shape="record"];
 "35" [label="{DynamoInvokes|\l|}", shape="record"];
-"36" [label="{DynamoPSy|\l|}", shape="record"];
+"36" [label="{LFRicPSy|\l|}", shape="record"];
 "45" [label="{Invoke|\l|first_access()\lgen()\lgen_code()\lunique_declarations()\lunique_declns_by_intent()\l}", shape="record", style=filled, fillcolor="antiquewhite"];
 "46" [label="{InvokeSchedule|\l|gen_code()\lview()\l}", shape="record"];
 "47" [label="{Invokes|\l|gen_code()\lgen_ocl_init()\lget()\l}", shape="record", style=filled, fillcolor="antiquewhite"];

doc/user_guide/introduction_to_psykal.rst

@@ -313,7 +313,7 @@ the Built-ins since they are provided as part of the environment.
 
   module solver_mod
     ...
-    use matrix_vector_mm_mod, only: matrix_vector_kernel_mm_type
+    use matrix_vector_mm_kernel_mod, only: matrix_vector_kernel_mm_type
     ...
 
     subroutine jacobi_solver_algorithm(lhs, rhs, mm, mesh, n_iter)

doc/user_guide/transformations.rst

@@ -148,11 +148,6 @@ can be found in the API-specific sections).
       :members: apply
       :noindex:
 
-.. warning:: This transformation assumes that the ABS Intrinsic acts on
-             PSyIR Real scalar data and does not check that this is
-             not the case. Once issue #658 is on master then this
-             limitation can be fixed.
-
 ####
 
 .. autoclass:: psyclone.transformations.ACCDataTrans
@@ -461,11 +456,6 @@ can be found in the API-specific sections).
       :members: apply
       :noindex:
 
-.. warning:: This transformation assumes that the SIGN Intrinsic acts
-             on PSyIR Real scalar data and does not check whether or not
-             this is the case. Once issue #658 is on master then this
-             limitation can be fixed.
-
 ####
 
 .. autoclass:: psyclone.psyir.transformations.Sum2LoopTrans
@@ -844,9 +834,13 @@ transformations currently supported allow the addition of:
 
 The generic versions of these transformations (i.e. ones that
 theoretically work for all APIs) were given in the
-:ref:`sec_transformations_available` section. The API-specific versions
-of these transformations are described in the API-specific sections of
-this document.
+:ref:`sec_transformations_available` section. Examples of their use,
+for both CPU and offload to GPU, may be found in the
+``PSyclone/examples/nemo/scripts/omp_?pu_trans.py`` transformation scripts.
+
+The API-specific versions of these transformations are described in the
+API-specific sections of this document. Examples for the LFRic API may
+be found in ``PSyclone/examples/lfric/scripts``.
 
 .. _openmp-reductions:
 
@@ -1039,9 +1033,9 @@ porting and/or debugging of an OpenACC application as it provides
 explicit control over what data is present on a device for a given
 (part of an) Invoke routine.
 
-The PGI compiler provides an alternative approach to controlling data
-movement through its 'unified memory' option
-(``-ta=tesla:managed``). When this is enabled the compiler itself takes
+The NVIDIA compiler compiler provides an alternative approach to controlling
+data movement through its 'managed memory' option
+(``-gpu=mem:managed``). When this is enabled the compiler itself takes
 on the task of ensuring that data is copied to/from the GPU when
 required. (Note that this approach can struggle with Fortran code
 containing derived types however.)

examples/gocean/eg2/Makefile

@@ -44,11 +44,11 @@
 #     make NVTX_DIR=/apps/packages/cuda/10.0/lib64 profile
 #
 # The compiler to use may be specified via the F90 environment
-# variable. e.g. to use PGI and OpenACC:
+# variable. e.g. to use the NVIDIA compiler and OpenACC:
 #
-# export F90=pgf90
-# export F90FLAGS="-O1 -acc -ta=tesla,cc70 -Minfo=all"
-# export LDFLAGS="-acc -ta=tesla,cc70"
+# export F90=nvfortran
+# export F90FLAGS="-O1 -acc -gpu=cc70 -Minfo=all"
+# export LDFLAGS="-acc -gpu=cc70"
 
 include ../../common.mk