Merge pull request #30 from aisclark91/alcf_new

Add new updates to the ALCF workflow

sphinx_docs/source/alcf.rst

@@ -27,16 +27,15 @@ Load the modules:
 
 .. prompt:: bash
 
-   module swap PrgEnv-nvhpc PrgEnv-gnu
-   # load gcc/11.2.0 version since CUDA doesn't support gcc 12 yet
-   module load gcc/11.2.0
+   module use /soft/modulefiles
+   module load PrgEnv-gnu
    module load nvhpc-mixed
 
 Then you can compile via:
 
 .. prompt:: bash
 
-   make COMP=gnu USE_CUDA=TRUE
+   make COMP=cray USE_CUDA=TRUE
 
 
 Disks
@@ -48,10 +47,11 @@ Project workspace is at: ``/lus/grand/projects/AstroExplosions/``
 Queues
 ======
 
-https://www.alcf.anl.gov/support/user-guides/polaris/queueing-and-running-jobs/job-and-queue-scheduling/index.html
-
 For production jobs, you submit to the ``prod`` queue.
 
+For debugging jobs, there are two options: the ``debug``  and ``debug-scaling`` options. More information can be found in:
+https://docs.alcf.anl.gov/polaris/running-jobs/#queues
+
 .. note::
 
    The smallest node count that seems to be allowed in production is 10 nodes.
@@ -108,3 +108,43 @@ A script that can be used to chain jobs with PBS is:
 .. literalinclude:: ../../job_scripts/polaris/chainqsub.sh
    :caption: ``chainqsub.sh``
 
+
+Installing Python
+=================
+
+The most recommended way to install python in polaris is to create a virtual environment
+on the top of the conda-based environment, provided by the module conda, and install all the extra
+required packages on this virtual environment by using ``pip``. Although is very tempting
+to clone the whole base environment and fully customize the installed conda packages, some
+modules like ``mpi4py`` may require access to the MPICH libraries that are only tailored to be
+used by the conda-based environment provided by the conda module. All these instructions
+follow the guidelines published here: https://docs.alcf.anl.gov/polaris/data-science-workflows/python/
+
+To create the virtual environment:
+
+.. prompt:: bash
+
+   module use /soft/modulefiles
+   module load conda 
+   conda activate
+   VENV_DIR="venvs/polaris"
+   mkdir -p "${VENV_DIR}"
+   python -m venv "${VENV_DIR}" --system-site-packages
+   source "${VENV_DIR}/bin/activate"
+
+To activate it in a new terminal, if the module path ``/soft/modulefiles``
+is loaded:
+
+.. prompt:: bash
+
+   module load conda 
+   conda activate
+   VENV_DIR="venvs/polaris"
+   source "${VENV_DIR}/bin/activate"
+
+Once the virtual environment is active, any extra package can be installed with
+the use of ``pip``:
+
+.. prompt:: bash
+
+   python -m pip install <package>