Merge pull request #34 from AMReX-Astro/update_olcf_frontier

update Frontier docs for latest ROCm

sphinx_docs/source/olcf-compilers.rst

@@ -67,17 +67,18 @@ load modules:
 .. prompt:: bash
 
    module load PrgEnv-gnu
-   module load cray-mpich/8.1.27
+   module load cray-mpich/8.1.28
    module load craype-accel-amd-gfx90a
-   module load amd-mixed/6.0.0
+   module load amd-mixed/6.1.3
    module unload darshan-runtime
 
-this will load ROCm 6.0.0
+this will load ROCm 6.1.3
 
 .. note::
 
-   Tabulate rates seem to exhibit a strange slow down on Frontier, so
-   it is best to run without rate tabulation.
+   In the past, tabulated rates seem to exhibit a strange slow down on
+   Frontier, so it is best to run test with and without rate
+   tabulation to see if there is a performance issue.
 
 build via:
 

sphinx_docs/source/olcf-workflow.rst

@@ -502,6 +502,23 @@ https://github.com/AMReX-Astro/workflow/blob/main/job_scripts/frontier/frontier.
 Also see the WarpX docs: https://warpx.readthedocs.io/en/latest/install/hpc/frontier.html
 
 
+GPU-aware MPI
+^^^^^^^^^^^^^
+
+Some codes run better with GPU-aware MPI.  To enable this add the following to your
+submission script:
+
+.. code:: bash
+
+   export MPICH_GPU_SUPPORT_ENABLED=1
+   export FI_MR_CACHE_MONITOR=memhooks
+
+and set the runtime parameter:
+
+.. code::
+
+   amrex.use_gpu_aware_mpi=1
+
 Job Status
 ^^^^^^^^^^