(closes #1010) Generate PSy-layers and Kernel Subs using PSyIR lowering (instead of gen_code/f2pygen) #2834

sergisiso · 2024-12-29T17:53:54Z

No description provided.

…ns to PSyIR

…10_remaining_lfric_lowering

…rsor on invokes declarations and initialisations

… invoke comments

…nto 1010_remaining_lfric_lowering

…c_lowering

…ith psyir backend

…r backend

…ster

…c_lowering

src/psyclone/psyir/nodes/psy_data_node.py

src/psyclone/tests/domain/lfric/lfric_extract_driver_creator_test.py

…c_lowering

sergisiso · 2025-01-23T12:28:28Z

@arporter This is finally ready for review, it is huge because:

It updates all remaining f2py/gen_code to psyir/lower_to_language_level (I coudn't find a way to split this process)
It produced stylistic changes to the psy-layer output syntax, and therefore lots of tests had to be slightly updated

However, most of the PR is a 1-to-1 mapping to previous code, I decided to leave the:

Final removal of f2py intrastructure, and other refactoring now possible (I am still abusing find_or_create instead of following a more predictable construction/lookup pattern) to avoid complicating the PR by moving things arround.

I am happy to do this in different review steps if it becomes too much (also Github slows down a lot when reviewing long PRs all at once)

There are a couple of coverage misses that I may need help constructing the LFRic examples that should trigger them.

arporter

Great stuff Sergi, it's really good to see so much code being removed. I've made it through all the source files now and some of the tests.
Generally small things to tidy up although I'd like to get rid of cursor from all the declaration routines as it's not used. In fact, I'm not totally sure it's needed for the initialisation - did you have ordering issues?

arporter · 2025-01-23T20:58:44Z

src/psyclone/domain/lfric/kern_call_arg_list.py

@@ -877,14 +882,14 @@ def quad_rule(self, var_accesses=None):
                    self.append_integer_reference(arg)
                elif generic_name in ["weights_xy", "weights_z"]:
                    # 1d arrays:
-                    # TODO # 1910: These should be pointers


Was this TODO incorrect - i.e. it's fine that they aren't pointers? Can #1910 be closed?

Ah, as this is the kernel-call argument list I suppose we don't care whether or not they are pointers as we just need References?

arporter · 2025-01-23T21:08:26Z

src/psyclone/domain/lfric/kern_call_arg_list.py

-                                                 ScalarType.Intrinsic.INTEGER,
-                                                 tag=tag)
+            symbol = self._symtab.find_or_create(
+                self._kern.colourmap.name, symbol_type=DataSymbol,


I'm a bit confused here. self._kern.colourmap is a call to LFRicKern.colourmap and that returns a Symbol. Perhaps it's not in the right table or perhaps symbol ends up being that same symbol? While I'm here, the docstring for LFRicKern.colourmap says it returns a str but it in fact it returns a Symbol. It's not the fault of this PR but would you mind fixing it?

arporter · 2025-01-23T21:09:24Z

src/psyclone/domain/lfric/kern_call_arg_list.py

@@ -984,7 +993,7 @@ def cell_ref_name(self, var_accesses=None):
                                        AccessType.READ,
                                        self._kern, ["colour", "cell"])

-            return (self._kern.colourmap + "(colour,cell)",
+            return (self._kern.colourmap.name + "(colour,cell)",


Could this now be array_ref.debug_string() or (better) use the FortranWriter?

I guess it should also have a TODO as this will need cleaning up somehow.

arporter · 2025-01-23T21:10:51Z

src/psyclone/domain/lfric/kern_stub_arg_list.py

@@ -61,10 +60,6 @@ class KernStubArgList(ArgOrdering):
    '''
    def __init__(self, kern):
        ArgOrdering.__init__(self, kern)
-        # TODO 719 The stub_symtab is not connected to other parts of the


Whoop! Does this mean that this PR will close #719?

arporter · 2025-01-24T09:05:04Z

src/psyclone/domain/lfric/lfric_cell_iterators.py

-    def initialise(self, parent):
+            for name in self._nlayers_names:
+                sym = self.symtab.lookup(name)
+                sym.interface = ArgumentInterface(


Please add a comment here - I can't currently work out why you have to do this here rather than when the symbol is created in the symbol table of the kernel stub?

arporter · 2025-01-24T16:36:06Z

src/psyclone/tests/domain/lfric/lfric_dofmaps_test.py

-    result = generate(os.path.join(BASE_PATH,
-                                   "columnwise_op_asm_kernel_mod.F90"),
-                      api=TEST_API)
+    stub_psyir = generate(os.path.join(BASE_PATH,


It's not psyir but text at this point.

arporter · 2025-01-24T16:36:36Z

src/psyclone/tests/domain/lfric/lfric_dofmaps_test.py

-    result = generate(os.path.join(BASE_PATH,
-                                   "columnwise_op_app_kernel_mod.F90"),
-                      api=TEST_API)
+    stub_psyir = generate(os.path.join(BASE_PATH,


text again.

arporter · 2025-01-24T16:37:42Z

src/psyclone/tests/domain/lfric/lfric_domain_kernels_test.py

-        expected = "      ! call our kernels\n"
-    assert (expected + "      !\n"
-            "      call testkern_domain_code(nlayers_f1, ncell_2d_no_halos, "
+    assert ("    call testkern_domain_code(nlayers_f1, ncell_2d_no_halos, "


Please extend to check that it is not within a loop.

arporter · 2025-01-24T16:38:39Z

src/psyclone/tests/domain/lfric/lfric_domain_kernels_test.py

    # Now call the loop handling method directly.
-    out = fortran_writer.loop_node(schedule.children[0])
+    out = fortran_writer(schedule)


Possibly this whole bit (from L301) can be removed now?

arporter · 2025-01-24T16:40:21Z

src/psyclone/tests/domain/lfric/lfric_extract_driver_creator_test.py

@@ -115,6 +115,7 @@ def test_create_read_in_code_missing_symbol(capsys, monkeypatch):
    cntr = minfo.get_psyir()
    # We can't use 'remove()' with a DataSymbol.
    cntr.symbol_table._symbols.pop("module_var_b")
+    return


This needs attention.... :-)

sergisiso added 30 commits April 9, 2024 13:47

#1010 Start converting LFRic PSy-layer declarations and initialisatio…

bc49de6

…ns to PSyIR

Merge remote-tracking branch 'origin/1495_type_bound_methods' into 10…

a3e1b35

…10_remaining_lfric_lowering

#1010 Continue porting LFRic declarations and initialisations to PSyIR

e864bc2

#1010 Pass one more test using backend declarations instead of f2pygen

df68488

#1010 Bring to master

cd515be

#1010 More tests use backend declarations instead of f2pygen

0b90f7b

#1010 More tests use backend declarations instead of f2pygen

96c1aa6

#1010 More tests use backend declarations instead of f2pygen, pass cu…

5c74c42

…rsor on invokes declarations and initialisations

#1010 More tests use backend declarations instead of f2pygen, and add…

e661950

… invoke comments

#1010 More tests use backend declarations instead of f2pygen

64a6bfd

#1010 Update dynamo0p3_basis test to the fortran backend syntax

6c55316

#1010 Start updating stub generation tests for new backend

93f88f3

#1010 Bring to master

77fe7ff

Merge remote-tracking branch 'origin/1010_remaining_lfric_lowering' i…

99ffbbe

…nto 1010_remaining_lfric_lowering

#1010 Fix tests in psyir

99422e6

Merge remote-tracking branch 'origin/master' into 1010_remaining_lfri…

9be02af

…c_lowering

#1010 Fix tests in psyir introduced by lastest update to master

2481022

#1010 Merge master and refactored builtin tests

decd073

#1010 Fix output syntax in gocean opencl trans tests

0e05a0b

#1010 Start updating dynamo0p3_transformations_test to check output w…

ef34fa1

…ith psyir backend

#1010 Update more dynamo0p3_transformations_test test to use the psyi…

62e41e5

…r backend

#1010 Update more dynamo0p3_transformations_test test to use the psyi…

8908239

…r backend

#1010 Bring to master

930ddb9

#1010 Fix some issues introduced due to previous merge with master

a2942f1

#1010 Fix issues with reference_element

cf63302

#1010 Start porting reductions to generic PSyIR

3d8969c

#1010 Remove file mistakenly introduced when brining the branch to ma…

98a899c

…ster

#1010 Move LFRic reductions to OpenMP classes

ddef4de

#1010 Bring to master

b697236

#1010 Modifications to pass more LFRic OpenMP reduction tests

6ab2c8c

sergisiso added 3 commits December 30, 2024 20:18

#1010 Fix merge issue

dc1c142

#1010 Fix flake8 issues

3ab9c4d

#1010 Improve test coverage and fix integration test issue

852a826

sergisiso temporarily deployed to integration December 31, 2024 17:34 — with GitHub Actions Inactive

sergisiso added 3 commits January 6, 2025 16:24

#1010 Remove some more old code and fix tests

1dd394e

#1010 Do not generate th_idx and nthreads variables when not needed

5d7d712

#1010 Fix some xfail tests

5dc9ec0

sergisiso temporarily deployed to integration January 9, 2025 16:23 — with GitHub Actions Inactive

Merge remote-tracking branch 'origin/master' into 1010_remaining_lfri…

e90cbcf

…c_lowering

sergisiso commented Jan 20, 2025

View reviewed changes

src/psyclone/psyir/nodes/psy_data_node.py Outdated Show resolved Hide resolved

sergisiso commented Jan 20, 2025

View reviewed changes

src/psyclone/tests/domain/lfric/lfric_extract_driver_creator_test.py Outdated Show resolved Hide resolved

sergisiso added 3 commits January 22, 2025 12:37

#1010 Resolve some issues with psy_data

3a4b983

#1010 Move LFRic loop_start/stop symbols at the beginning

3a18bb3

#1010 Remove NOT_INITIALISED integer literals

cd18e86

sergisiso mentioned this pull request Jan 23, 2025

Performance improvement by using private declaration for gfortran #2850

Open

sergisiso added 4 commits January 23, 2025 10:09

Merge remote-tracking branch 'origin/master' into 1010_remaining_lfri…

6ddb65e

…c_lowering

#1010 Clean up psy_data_node

c345f0a

#1010 Convert some array types to pointers

9ec3f54

#1010 Clean up PR

57db171

sergisiso temporarily deployed to integration January 23, 2025 12:09 — with GitHub Actions Inactive

sergisiso requested a review from arporter January 23, 2025 12:19

sergisiso self-assigned this Jan 23, 2025

sergisiso added the ready for review label Jan 23, 2025

arporter added under review and removed ready for review labels Jan 23, 2025

arporter requested changes Jan 24, 2025

View reviewed changes

arporter added reviewed with actions LFRic Issue relates to the LFRic domain and removed under review labels Jan 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(closes #1010) Generate PSy-layers and Kernel Subs using PSyIR lowering (instead of gen_code/f2pygen) #2834

(closes #1010) Generate PSy-layers and Kernel Subs using PSyIR lowering (instead of gen_code/f2pygen) #2834

sergisiso commented Dec 29, 2024

sergisiso commented Jan 23, 2025

arporter left a comment

arporter Jan 23, 2025

arporter Jan 23, 2025

arporter Jan 23, 2025

arporter Jan 23, 2025

arporter Jan 23, 2025

arporter Jan 23, 2025

arporter Jan 24, 2025

arporter Jan 24, 2025

arporter Jan 24, 2025

arporter Jan 24, 2025

arporter Jan 24, 2025

arporter Jan 24, 2025

(closes #1010) Generate PSy-layers and Kernel Subs using PSyIR lowering (instead of gen_code/f2pygen) #2834

Are you sure you want to change the base?

(closes #1010) Generate PSy-layers and Kernel Subs using PSyIR lowering (instead of gen_code/f2pygen) #2834

Conversation

sergisiso commented Dec 29, 2024

sergisiso commented Jan 23, 2025

arporter left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment