From 9275ec478fafc23fb9d008f5413f7e5cc1494982 Mon Sep 17 00:00:00 2001 From: Damien Irving Date: Thu, 25 Jul 2024 15:34:16 -0400 Subject: [PATCH 01/10] Fix lesson 1 link --- learners/setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/learners/setup.md b/learners/setup.md index 2e8283b..8d9da68 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -42,7 +42,7 @@ The Carpentries maintain a list of common issues on their :::::::::::::::::::::::::::::::::::::::::::::::::: Your workshop instructor may also ask that you install the python packages introduced in the -[first lesson](https://carpentries-lab.github.io/python-aos-lesson/01-conda/index.html) +[first lesson](https://carpentries-lab.github.io/python-aos-lesson/01-conda.html) ahead of time. You can do this via the command line or by using the Anaconda Navigator: From cc2e666aa8f7feb02d6cef9ef145a0324c306d7a Mon Sep 17 00:00:00 2001 From: Damien Irving Date: Thu, 25 Jul 2024 15:46:31 -0400 Subject: [PATCH 02/10] Update Anaconda link --- episodes/01-conda.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/episodes/01-conda.md b/episodes/01-conda.md index cbd32b7..ec82bd9 100644 --- a/episodes/01-conda.md +++ b/episodes/01-conda.md @@ -54,7 +54,7 @@ a number of scientific Python "distributions" have been released over the years. These come with the most popular data science libraries and their dependencies pre-installed, and some also come with a package manager to assist with installing additional libraries that weren't pre-installed. -Today the most popular distribution for data science is [Anaconda](https://www.anaconda.com/distribution/), +Today the most popular distribution for data science is [Anaconda](https://docs.anaconda.com/anaconda/), which comes with a package (and environment) manager called [conda](https://conda.io/docs/). ## Introducing conda From 954fc50e700b702abe8bfc73cc41dc1bd6eb0bb5 Mon Sep 17 00:00:00 2001 From: Damien Irving Date: Thu, 25 Jul 2024 16:04:31 -0400 Subject: [PATCH 03/10] Update links in 01-conda.md --- episodes/01-conda.md | 48 +++++++++++++------------------------------- 1 file changed, 14 insertions(+), 34 deletions(-) diff --git a/episodes/01-conda.md b/episodes/01-conda.md index ec82bd9..e173316 100644 --- a/episodes/01-conda.md +++ b/episodes/01-conda.md @@ -29,7 +29,7 @@ we need to consider what Python libraries are best suited to the task. For reading, writing and analysing data stored in the netCDF file format, atmosphere and ocean scientists will typically do most of their work with either the -[xarray](https://xarray.pydata.org/en/stable/) or [iris](https://scitools.org.uk/iris/) libraries. +[xarray](https://docs.xarray.dev) or [iris](https://scitools-iris.readthedocs.io/) libraries. These libraries are built on top of more generic data science libraries like numpy and matplotlib, to make the types of analysis we do faster and more efficient. To learn more about the PyAOS "stack" shown in the diagram below @@ -59,10 +59,10 @@ which comes with a package (and environment) manager called [conda](https://cond ## Introducing conda -According to the [latest documentation](https://docs.anaconda.com/anaconda/#anaconda-navigator-or-conda), -Anaconda comes with over 250 of the most widely used data science libraries (and their dependencies) pre-installed. -In addition, there are several thousand more libraries available via the `conda install` command, -which can be executed using the Bash Shell or Anaconda Prompt (Windows only). +According to the [latest documentation]([https://docs.anaconda.com/anaconda/), +Anaconda comes with over 300 of the most widely used data science libraries (and their dependencies) pre-installed. +In addition, there are several thousand more libraries available via the Anaconda Public Repository, +which can be installed by running the `conda install` command the Bash Shell or Anaconda Prompt (Windows only). It is also possible to install packages using the Anaconda Navigator graphical user interface. ::::::::::::::::::::::::::::::::::::::::: callout @@ -93,7 +93,7 @@ OR using Anaconda Navigator: ## Miniconda If you don't want to install the entire Anaconda distribution, -you can install [Miniconda](https://conda.pydata.org/miniconda.html) instead. +you can install [Miniconda](https://docs.anaconda.com/miniconda/) instead. It essentially comes with conda and nothing else. :::::::::::::::::::::::::::::::::::::::::::::::::: @@ -105,8 +105,7 @@ one of the most important features that Anaconda provides is the [Anaconda Cloud](https://anaconda.org) website, where the community can contribute conda installation packages. This is critical because many of our libraries have a small user base, -which means they'll never make it into the top few thousand data science libraries -supported by Anaconda. +which means they'll never make it into the Anaconda Public Repository. You can search Anaconda Cloud to find the command needed to install the package. For instance, here is the search result for the `iris` package: @@ -115,7 +114,7 @@ For instance, here is the search result for the `iris` package: As you can see, there are often multiple versions of the same package up on Anaconda Cloud. To try and address this duplication problem, -[conda-forge](https://conda-forge.github.io/) has been launched, +[conda-forge](https://conda-forge.org/) has been launched, which aims to be a central repository that contains just a single (working) version of each package on Anaconda Cloud. You can therefore expand the selection of packages available via `conda install` @@ -137,7 +136,7 @@ because mixing packages from multiple channels can cause headaches like binary i For these particular lessons we will use `xarray`, but all the same tasks could be performed with `iris`. We'll also install -[`dask`](https://dask.org/) (`xarray` uses this for parallel processing), +[`dask`](https://www.dask.org/) (`xarray` uses this for parallel processing), [`netCDF4`](https://unidata.github.io/netcdf4-python/) (`xarray` requires this to read netCDF files), [`cartopy`](https://scitools.org.uk/cartopy/) (to help with geographic plot projections), [`cmocean`](https://matplotlib.org/cmocean/) (for nice color palettes) and @@ -172,7 +171,7 @@ If you've got multiple data science projects on the go, installing all your packages in the same conda environment can get a little messy. (By default they are installed in the root/base environment.) It's therefore common practice to -[create separate conda environments](https://conda.io/docs/user-guide/tasks/manage-environments.html) +[create separate conda environments](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html) for the various projects you're working on. For instance, we could create an environment called `pyaos-lesson` for this lesson. @@ -231,20 +230,6 @@ using the YAML file: $ conda env create -f pyaos-lesson.yml ``` -For ease of sharing the YAML file, -it can be uploaded to your account at the Anaconda Cloud website, - -```bash -$ conda env upload -f pyaos-lesson.yml -``` - -so that others can re-create the environment by simply refering to your Anaconda username: - -```bash -$ conda env create damienirving/pyaos-lesson -$ conda activate pyaos-lesson -``` - The ease with which others can recreate your environment (on any operating system) is a huge breakthough for reproducible research. @@ -278,7 +263,7 @@ $ The `>>>` prompt indicates that you are now talking to the Python interpreter. A more powerful alternative to the default Python interpreter is IPython (Interactive Python). -The [online documentation](https://ipython.readthedocs.io/en/stable/) +The [online documentation](https://ipython.readthedocs.io) outlines all the special features that come with IPython, but as an example, it lets you execute bash shell commands without having to exit the IPython interpreter: @@ -342,8 +327,7 @@ Python 3 notebook: ## JupyterLab -The Jupyter team have recently launched -[JupyterLab](https://blog.jupyter.org/jupyterlab-is-ready-for-users-5a6f039b8906) +If you like Jupyter Notebooks you might want to try [JupyterLab](https://jupyterlab.readthedocs.io), which combines the Jupyter Notebook with many of the features common to an IDE. :::::::::::::::::::::::::::::::::::::::::::::::::: @@ -365,10 +349,8 @@ you'll need to install jupyter too.) ::::::::::::::: solution -The [setup menu](https://carpentries-lab.github.io/python-aos-lesson/setup.html) -at the top of the page -contains drop-down boxes explaining how to install the Python libraries -using the Bash Shell or Anaconda Navigator. +The [software installation instructions](https://carpentries-lab.github.io/python-aos-lesson/#software-installation) +explain how to install the Python libraries using the Bash Shell or Anaconda Navigator. ::::::::::::::::::::::::: @@ -412,5 +394,3 @@ import numpy as np - Use conda to install and manage your Python environments. :::::::::::::::::::::::::::::::::::::::::::::::::: - - From 5433d11aa208c920fdc06ab481cdff94f52797e3 Mon Sep 17 00:00:00 2001 From: Damien Irving Date: Thu, 25 Jul 2024 16:07:13 -0400 Subject: [PATCH 04/10] Update content in 02-visualisation.md --- episodes/02-visualisation.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/episodes/02-visualisation.md b/episodes/02-visualisation.md index 24baa81..baf09f0 100644 --- a/episodes/02-visualisation.md +++ b/episodes/02-visualisation.md @@ -261,7 +261,7 @@ plt.show() The default colorbar used by matplotlib is `viridis`. It used to be `jet`, -but that was changed a couple of years ago in response to the +but that was changed in response to the [\#endtherainbow](https://www.climate-lab-book.ac.uk/2014/end-of-the-rainbow/) campaign. Putting all the code together @@ -334,7 +334,7 @@ clim.plot.contourf( Rather than plot the annual climatology, edit the code so that it plots the June-August (JJA) season. -(Hint: the [groupby]() functionality can be used to +(Hint: the `groupby` functionality can be used to group all the data into seasons prior to averaging over the time axis) ::::::::::::::: solution From 98f631600439f9dc810fe08e63703d83f019c61b Mon Sep 17 00:00:00 2001 From: Damien Irving Date: Thu, 25 Jul 2024 16:10:14 -0400 Subject: [PATCH 05/10] Add links to 04-cmdline.md --- episodes/04-cmdline.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/episodes/04-cmdline.md b/episodes/04-cmdline.md index 8c9931c..e230831 100644 --- a/episodes/04-cmdline.md +++ b/episodes/04-cmdline.md @@ -26,7 +26,8 @@ and/or take notes on a new data analysis task. Once you've scoped out the task (as we have for plotting the precipitation climatology), that code can be transferred to a Python script so that it can be executed at the command line. It's likely that your data processing workflows will include command line utilities -from the CDO and NCO projects in addition to Python code, +from the [CDO](https://code.mpimet.mpg.de/projects/cdo) and +[NCO](https://nco.sourceforge.net/) projects in addition to Python code, so the command line is the natural place to manage your workflows (e.g. using shell scripts or make files). From 2ef3eb8481049745e35dffffa041930802a66a3e Mon Sep 17 00:00:00 2001 From: Damien Irving Date: Thu, 25 Jul 2024 16:17:11 -0400 Subject: [PATCH 06/10] Update 05-git.md --- episodes/05-git.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/episodes/05-git.md b/episodes/05-git.md index ef3ac96..345625c 100644 --- a/episodes/05-git.md +++ b/episodes/05-git.md @@ -649,7 +649,8 @@ you may see a suggestion for `git checkout` instead of `git restore`. As of Git version 2.29, `git restore` is still an experimental command and operates as a specialized form of `git checkout`. -`git checkout HEAD plot_precipitation_climatology` + +`git checkout HEAD plot_precipitation_climatology.py` is the equivalent command. From e93dc3798ce9bd15e23a98daf7ae2ea4d192c1ef Mon Sep 17 00:00:00 2001 From: Damien Irving Date: Thu, 25 Jul 2024 16:28:34 -0400 Subject: [PATCH 07/10] Update links in 08-defensive.md --- episodes/08-defensive.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/episodes/08-defensive.md b/episodes/08-defensive.md index 48ee505..cce34c1 100644 --- a/episodes/08-defensive.md +++ b/episodes/08-defensive.md @@ -47,7 +47,7 @@ write unit tests to make sure each component of our program produces expected ou and use a logging framework to report on program activity. In this lesson, we'll look at how error handling, assertions and logging can make the unit conversion in our program more reliable, -and we'll provide links to further information on unit testing. +and we'll provide links to further information on unit . ## Types of errors @@ -327,7 +327,7 @@ An assertion checks that something is true at a particular point in the program. For programs that are more complex (or research critical) than `plot_precipitation_climatology.py`, it's a good idea to take the next step and check the overall behavior of entire pieces (or units) of code. Related concepts like unit testing and continuous integration are beyond the scope of this lesson, -but *Research Software Engineering With Python* has a [chapter on testing](https://merely-useful.github.io/py-rse/testing.html) +but *Research Software Engineering With Python* has a [chapter on testing](https://third-bit.com/py-rse/testing.html) that is well worth a read. :::::::::::::::::::::::::::::::::::::::::::::::::: From 120f9992a63afa6b9f934091d79e0ca24d523e4c Mon Sep 17 00:00:00 2001 From: Damien Irving Date: Thu, 25 Jul 2024 16:31:15 -0400 Subject: [PATCH 08/10] Update links in 09-provenance.md --- episodes/09-provenance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/episodes/09-provenance.md b/episodes/09-provenance.md index 304fbbb..d4a9f4e 100644 --- a/episodes/09-provenance.md +++ b/episodes/09-provenance.md @@ -208,7 +208,7 @@ def main(inargs): ## Handling different image formats -The [`plt.savefig` documentation](https://matplotlib.org/3.1.1/api/_as_gen/matplotlib.pyplot.savefig.html) +The [`plt.savefig` documentation](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.savefig.html) provides information on the metadata keys accepted by PNG, PDF, EPS and PS image formats. From 737e130d69f96925b1ba7b1b2a7827ce81bddd84 Mon Sep 17 00:00:00 2001 From: Damien Irving Date: Thu, 25 Jul 2024 16:36:24 -0400 Subject: [PATCH 09/10] Update links in 10-large-data.md --- episodes/10-large-data.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/episodes/10-large-data.md b/episodes/10-large-data.md index 43263cf..8c8a390 100644 --- a/episodes/10-large-data.md +++ b/episodes/10-large-data.md @@ -33,7 +33,7 @@ for the high resolution CNRM-CM6-1-HR model. Instructors teaching this lesson can download the CNRM-CM6-1-HR daily precipitation data from the Earth System Grid Federation (ESGF). -See the [instructor notes](https://carpentries-lab.github.io/python-aos-lesson/guide/index.html) for details. +See the [instructor notes](https://carpentries-lab.github.io/python-aos-lesson/instructor/instructor-notes.html) for details. Since it is a very large download (45 GB), learners are not expected to download the data. (None of the exercises at the end of the lesson require downloading the data.) @@ -353,8 +353,8 @@ that aren't built into `xarray` (e.g. an interpolation routine from the SciPy library) we'd first need to use the `apply_ufunc` or `map_blocks` function to make those operations "Dask-aware". -The [xarray tutorial](https://xarray-contrib.github.io/xarray-tutorial/scipy-tutorial/06_xarray_and_dask.html#Automatic-parallelization-with-apply_ufunc-and-map_blocks) -from SciPy 2020 explains how to do this. +There's an [xarray tutorial](https://docs.xarray.dev/en/stable/examples/apply_ufunc_vectorize_1d.html) +that explains how to do this. :::::::::::::::::::::::::::::::::::::::::::::::::: From c63385fa21696c77b827393a1b4d9de4f598920e Mon Sep 17 00:00:00 2001 From: Damien Irving Date: Thu, 25 Jul 2024 16:42:16 -0400 Subject: [PATCH 10/10] Update instructor-notes.md --- instructors/instructor-notes.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/instructors/instructor-notes.md b/instructors/instructor-notes.md index a7ee435..a11deed 100644 --- a/instructors/instructor-notes.md +++ b/instructors/instructor-notes.md @@ -20,8 +20,8 @@ This is done to reinforce the repetitive "add, commit, push" workflow in git. At the beginning of the workshop, participants are required to download a number of data files -(instructions at the [setup page](https://carpentries-lab.github.io/python-aos-lesson/setup.html)). -In the [first lesson](https://carpentries-lab.github.io/python-aos-lesson/01-conda/index.html), +(instructions at the [setup page](https://carpentries-lab.github.io/python-aos-lesson/index.html#data)). +In the [first lesson](https://carpentries-lab.github.io/python-aos-lesson/01-conda.html#install-the-python-libraries-required-for-this-lesson) they are then required to install some python libraries (`jupyter`, `xarray`, `cmocean`, etc). Both these tasks can be problematic at venues with slow wifi, so it is often a good idea to ask participants to download the data @@ -43,12 +43,10 @@ Use the following search terms to locate the data at your nearest ESGF node: ## Software and code -The [setup page](https://carpentries-lab.github.io/python-aos-lesson/setup.html) +The [setup page](https://carpentries-lab.github.io/python-aos-lesson/index.html#software-installation) gives details of the software installation instructions that can provided to participants. You can also send the -[helper lesson check](https://github.com/carpentries-lab/python-aos-lesson/blob/gh-pages/helper_lesson_check.md) +[helper lesson check](https://github.com/carpentries-lab/python-aos-lesson/blob/main/helper_lesson_check.md) to helpers prior to the workshop, so that they can test that all the software and code is working correctly. - -