Skip to content

Commit

Permalink
markdown source builds
Browse files Browse the repository at this point in the history
Auto-generated via {sandpaper}
Source  : 3d83349
Branch  : main
Author  : Damien Irving <irving.damien@gmail.com>
Time    : 2024-07-25 20:43:24 +0000
Message : Merge pull request #67 from carpentries-lab/DamienIrving-patch-1

Fix broken links
  • Loading branch information
actions-user committed Jul 25, 2024
1 parent 17c9727 commit 09e85d3
Show file tree
Hide file tree
Showing 10 changed files with 41 additions and 61 deletions.
50 changes: 15 additions & 35 deletions 01-conda.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ we need to consider what Python libraries are best suited to the task.

For reading, writing and analysing data stored in the netCDF file format,
atmosphere and ocean scientists will typically do most of their work with either the
[xarray](https://xarray.pydata.org/en/stable/) or [iris](https://scitools.org.uk/iris/) libraries.
[xarray](https://docs.xarray.dev) or [iris](https://scitools-iris.readthedocs.io/) libraries.
These libraries are built on top of more generic data science libraries like numpy and matplotlib,
to make the types of analysis we do faster and more efficient.
To learn more about the PyAOS "stack" shown in the diagram below
Expand All @@ -54,15 +54,15 @@ a number of scientific Python "distributions" have been released over the years.
These come with the most popular data science libraries and their dependencies pre-installed,
and some also come with a package manager to assist with installing
additional libraries that weren't pre-installed.
Today the most popular distribution for data science is [Anaconda](https://www.anaconda.com/distribution/),
Today the most popular distribution for data science is [Anaconda](https://docs.anaconda.com/anaconda/),
which comes with a package (and environment) manager called [conda](https://conda.io/docs/).

## Introducing conda

According to the [latest documentation](https://docs.anaconda.com/anaconda/#anaconda-navigator-or-conda),
Anaconda comes with over 250 of the most widely used data science libraries (and their dependencies) pre-installed.
In addition, there are several thousand more libraries available via the `conda install` command,
which can be executed using the Bash Shell or Anaconda Prompt (Windows only).
According to the [latest documentation]([https://docs.anaconda.com/anaconda/),
Anaconda comes with over 300 of the most widely used data science libraries (and their dependencies) pre-installed.
In addition, there are several thousand more libraries available via the Anaconda Public Repository,
which can be installed by running the `conda install` command the Bash Shell or Anaconda Prompt (Windows only).
It is also possible to install packages using the Anaconda Navigator graphical user interface.

::::::::::::::::::::::::::::::::::::::::: callout
Expand Down Expand Up @@ -93,7 +93,7 @@ OR using Anaconda Navigator:
## Miniconda

If you don't want to install the entire Anaconda distribution,
you can install [Miniconda](https://conda.pydata.org/miniconda.html) instead.
you can install [Miniconda](https://docs.anaconda.com/miniconda/) instead.
It essentially comes with conda and nothing else.

::::::::::::::::::::::::::::::::::::::::::::::::::
Expand All @@ -105,8 +105,7 @@ one of the most important features that Anaconda provides is the
[Anaconda Cloud](https://anaconda.org) website,
where the community can contribute conda installation packages.
This is critical because many of our libraries have a small user base,
which means they'll never make it into the top few thousand data science libraries
supported by Anaconda.
which means they'll never make it into the Anaconda Public Repository.

You can search Anaconda Cloud to find the command needed to install the package.
For instance, here is the search result for the `iris` package:
Expand All @@ -115,7 +114,7 @@ For instance, here is the search result for the `iris` package:

As you can see, there are often multiple versions of the same package up on Anaconda Cloud.
To try and address this duplication problem,
[conda-forge](https://conda-forge.github.io/) has been launched,
[conda-forge](https://conda-forge.org/) has been launched,
which aims to be a central repository that contains just a single (working) version
of each package on Anaconda Cloud.
You can therefore expand the selection of packages available via `conda install`
Expand All @@ -137,7 +136,7 @@ because mixing packages from multiple channels can cause headaches like binary i
For these particular lessons we will use `xarray`,
but all the same tasks could be performed with `iris`.
We'll also install
[`dask`](https://dask.org/) (`xarray` uses this for parallel processing),
[`dask`](https://www.dask.org/) (`xarray` uses this for parallel processing),
[`netCDF4`](https://unidata.github.io/netcdf4-python/) (`xarray` requires this to read netCDF files),
[`cartopy`](https://scitools.org.uk/cartopy/) (to help with geographic plot projections),
[`cmocean`](https://matplotlib.org/cmocean/) (for nice color palettes) and
Expand Down Expand Up @@ -172,7 +171,7 @@ If you've got multiple data science projects on the go,
installing all your packages in the same conda environment can get a little messy.
(By default they are installed in the root/base environment.)
It's therefore common practice to
[create separate conda environments](https://conda.io/docs/user-guide/tasks/manage-environments.html)
[create separate conda environments](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html)
for the various projects you're working on.

For instance, we could create an environment called `pyaos-lesson` for this lesson.
Expand Down Expand Up @@ -231,20 +230,6 @@ using the YAML file:
$ conda env create -f pyaos-lesson.yml
```

For ease of sharing the YAML file,
it can be uploaded to your account at the Anaconda Cloud website,

```bash
$ conda env upload -f pyaos-lesson.yml
```

so that others can re-create the environment by simply refering to your Anaconda username:

```bash
$ conda env create damienirving/pyaos-lesson
$ conda activate pyaos-lesson
```

The ease with which others can recreate your environment (on any operating system)
is a huge breakthough for reproducible research.

Expand Down Expand Up @@ -278,7 +263,7 @@ $
The `>>>` prompt indicates that you are now talking to the Python interpreter.

A more powerful alternative to the default Python interpreter is IPython (Interactive Python).
The [online documentation](https://ipython.readthedocs.io/en/stable/)
The [online documentation](https://ipython.readthedocs.io)
outlines all the special features that come with IPython,
but as an example, it lets you execute bash shell commands
without having to exit the IPython interpreter:
Expand Down Expand Up @@ -342,8 +327,7 @@ Python 3 notebook:

## JupyterLab

The Jupyter team have recently launched
[JupyterLab](https://blog.jupyter.org/jupyterlab-is-ready-for-users-5a6f039b8906)
If you like Jupyter Notebooks you might want to try [JupyterLab](https://jupyterlab.readthedocs.io),
which combines the Jupyter Notebook with many of the features common to an IDE.

::::::::::::::::::::::::::::::::::::::::::::::::::
Expand All @@ -365,10 +349,8 @@ you'll need to install jupyter too.)

::::::::::::::: solution

The [setup menu](https://carpentries-lab.github.io/python-aos-lesson/setup.html)
at the top of the page
contains drop-down boxes explaining how to install the Python libraries
using the Bash Shell or Anaconda Navigator.
The [software installation instructions](https://carpentries-lab.github.io/python-aos-lesson/#software-installation)
explain how to install the Python libraries using the Bash Shell or Anaconda Navigator.

:::::::::::::::::::::::::

Expand Down Expand Up @@ -412,5 +394,3 @@ import numpy as np
- Use conda to install and manage your Python environments.

::::::::::::::::::::::::::::::::::::::::::::::::::


4 changes: 2 additions & 2 deletions 02-visualisation.md
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ plt.show()

The default colorbar used by matplotlib is `viridis`.
It used to be `jet`,
but that was changed a couple of years ago in response to the
but that was changed in response to the
[\#endtherainbow](https://www.climate-lab-book.ac.uk/2014/end-of-the-rainbow/) campaign.

Putting all the code together
Expand Down Expand Up @@ -334,7 +334,7 @@ clim.plot.contourf(
Rather than plot the annual climatology,
edit the code so that it plots the June-August (JJA) season.

(Hint: the [groupby]() functionality can be used to
(Hint: the `groupby` functionality can be used to
group all the data into seasons prior to averaging over the time axis)

::::::::::::::: solution
Expand Down
3 changes: 2 additions & 1 deletion 04-cmdline.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ and/or take notes on a new data analysis task.
Once you've scoped out the task (as we have for plotting the precipitation climatology),
that code can be transferred to a Python script so that it can be executed at the command line.
It's likely that your data processing workflows will include command line utilities
from the CDO and NCO projects in addition to Python code,
from the [CDO](https://code.mpimet.mpg.de/projects/cdo) and
[NCO](https://nco.sourceforge.net/) projects in addition to Python code,
so the command line is the natural place to manage your workflows
(e.g. using shell scripts or make files).

Expand Down
3 changes: 2 additions & 1 deletion 05-git.md
Original file line number Diff line number Diff line change
Expand Up @@ -649,7 +649,8 @@ you may see a suggestion for `git checkout` instead of `git restore`.
As of Git version 2.29,
`git restore` is still an experimental command
and operates as a specialized form of `git checkout`.
`git checkout HEAD plot_precipitation_climatology`

`git checkout HEAD plot_precipitation_climatology.py`
is the equivalent command.


Expand Down
4 changes: 2 additions & 2 deletions 08-defensive.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ write unit tests to make sure each component of our program produces expected ou
and use a logging framework to report on program activity.
In this lesson, we'll look at how error handling, assertions and logging
can make the unit conversion in our program more reliable,
and we'll provide links to further information on unit testing.
and we'll provide links to further information on unit .

## Types of errors

Expand Down Expand Up @@ -327,7 +327,7 @@ An assertion checks that something is true at a particular point in the program.
For programs that are more complex (or research critical) than `plot_precipitation_climatology.py`,
it's a good idea to take the next step and check the overall behavior of entire pieces (or units) of code.
Related concepts like unit testing and continuous integration are beyond the scope of this lesson,
but *Research Software Engineering With Python* has a [chapter on testing](https://merely-useful.github.io/py-rse/testing.html)
but *Research Software Engineering With Python* has a [chapter on testing](https://third-bit.com/py-rse/testing.html)
that is well worth a read.

::::::::::::::::::::::::::::::::::::::::::::::::::
Expand Down
2 changes: 1 addition & 1 deletion 09-provenance.md
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ def main(inargs):

## Handling different image formats

The [`plt.savefig` documentation](https://matplotlib.org/3.1.1/api/_as_gen/matplotlib.pyplot.savefig.html)
The [`plt.savefig` documentation](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.savefig.html)
provides information on the metadata keys accepted by
PNG, PDF, EPS and PS image formats.

Expand Down
6 changes: 3 additions & 3 deletions 10-large-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ for the high resolution CNRM-CM6-1-HR model.

Instructors teaching this lesson can download the CNRM-CM6-1-HR
daily precipitation data from the Earth System Grid Federation (ESGF).
See the [instructor notes](https://carpentries-lab.github.io/python-aos-lesson/guide/index.html) for details.
See the [instructor notes](https://carpentries-lab.github.io/python-aos-lesson/instructor/instructor-notes.html) for details.
Since it is a very large download (45 GB),
learners are not expected to download the data.
(None of the exercises at the end of the lesson require downloading the data.)
Expand Down Expand Up @@ -353,8 +353,8 @@ that aren't built into `xarray`
(e.g. an interpolation routine from the SciPy library)
we'd first need to use the `apply_ufunc` or `map_blocks` function
to make those operations "Dask-aware".
The [xarray tutorial](https://xarray-contrib.github.io/xarray-tutorial/scipy-tutorial/06_xarray_and_dask.html#Automatic-parallelization-with-apply_ufunc-and-map_blocks)
from SciPy 2020 explains how to do this.
There's an [xarray tutorial](https://docs.xarray.dev/en/stable/examples/apply_ufunc_vectorize_1d.html)
that explains how to do this.

::::::::::::::::::::::::::::::::::::::::::::::::::

Expand Down
10 changes: 4 additions & 6 deletions instructor-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ This is done to reinforce the repetitive "add, commit, push" workflow in git.

At the beginning of the workshop,
participants are required to download a number of data files
(instructions at the [setup page](https://carpentries-lab.github.io/python-aos-lesson/setup.html)).
In the [first lesson](https://carpentries-lab.github.io/python-aos-lesson/01-conda/index.html),
(instructions at the [setup page](https://carpentries-lab.github.io/python-aos-lesson/index.html#data)).
In the [first lesson](https://carpentries-lab.github.io/python-aos-lesson/01-conda.html#install-the-python-libraries-required-for-this-lesson)
they are then required to install some python libraries (`jupyter`, `xarray`, `cmocean`, etc).
Both these tasks can be problematic at venues with slow wifi,
so it is often a good idea to ask participants to download the data
Expand All @@ -43,12 +43,10 @@ Use the following search terms to locate the data at your nearest ESGF node:

## Software and code

The [setup page](https://carpentries-lab.github.io/python-aos-lesson/setup.html)
The [setup page](https://carpentries-lab.github.io/python-aos-lesson/index.html#software-installation)
gives details of the software installation instructions that can provided to participants.

You can also send the
[helper lesson check](https://github.com/carpentries-lab/python-aos-lesson/blob/gh-pages/helper_lesson_check.md)
[helper lesson check](https://github.com/carpentries-lab/python-aos-lesson/blob/main/helper_lesson_check.md)
to helpers prior to the workshop,
so that they can test that all the software and code is working correctly.


18 changes: 9 additions & 9 deletions md5sum.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,17 @@
"helper_lesson_check.md" "1d1b53176140a057b588969f9f60fa61" "site/built/helper_lesson_check.md" "2024-07-25"
"index.md" "0e62af27fdd16f8c6f2c9e1fef67e750" "site/built/index.md" "2024-07-25"
"paper.md" "32ace9442c642b5e25ce659e7ae9df72" "site/built/paper.md" "2024-07-25"
"episodes/01-conda.md" "a6686033d7b73c398bf6f559e29b0065" "site/built/01-conda.md" "2024-07-25"
"episodes/02-visualisation.md" "1e0da181aa29d53a614a7358d5379434" "site/built/02-visualisation.md" "2024-07-25"
"episodes/01-conda.md" "6145214a159aca5d93d996ce060b6fc2" "site/built/01-conda.md" "2024-07-25"
"episodes/02-visualisation.md" "a12f74e55346ec75d7fc4a945ab028c5" "site/built/02-visualisation.md" "2024-07-25"
"episodes/03-functions.md" "5f76d28ee961529d9fdf812a01cc5e81" "site/built/03-functions.md" "2024-07-25"
"episodes/04-cmdline.md" "ba7a163f8c93824a88a42045723108c6" "site/built/04-cmdline.md" "2024-07-25"
"episodes/05-git.md" "6b10244dccd7f145ba1ad6c915ac06b8" "site/built/05-git.md" "2024-07-25"
"episodes/04-cmdline.md" "b4b598fae925dc8a9ec46a8cafae2048" "site/built/04-cmdline.md" "2024-07-25"
"episodes/05-git.md" "eb8f096013c9345e773be2150f3a83c2" "site/built/05-git.md" "2024-07-25"
"episodes/06-github.md" "3ae16cc2084d7dca01805a73eabc556a" "site/built/06-github.md" "2024-07-25"
"episodes/07-vectorisation.md" "04d52cf7cd3a849c406c5c96cca4671c" "site/built/07-vectorisation.md" "2024-07-25"
"episodes/08-defensive.md" "4e709227290c2bffb21d737b55cb67f9" "site/built/08-defensive.md" "2024-07-25"
"episodes/09-provenance.md" "91a9a5b5e34b594ca49d77c319264eb8" "site/built/09-provenance.md" "2024-07-25"
"episodes/10-large-data.md" "909dba426cb9409b6124181662c31af4" "site/built/10-large-data.md" "2024-07-25"
"instructors/instructor-notes.md" "858a8aea24f020d8732ac71b2601f2e6" "site/built/instructor-notes.md" "2024-07-25"
"episodes/08-defensive.md" "ffb30e3794f41e1ccabd50fee6c93550" "site/built/08-defensive.md" "2024-07-25"
"episodes/09-provenance.md" "c7e54cd796b333c02084d0b774bae734" "site/built/09-provenance.md" "2024-07-25"
"episodes/10-large-data.md" "25882b71d089110a254d7265cbd1ae62" "site/built/10-large-data.md" "2024-07-25"
"instructors/instructor-notes.md" "1ce510d0cb6a52e613cda477b6fad382" "site/built/instructor-notes.md" "2024-07-25"
"learners/reference.md" "4e0dcbc7892af6f9610d44d356e66617" "site/built/reference.md" "2024-07-25"
"learners/setup.md" "d86e41df032ef642f234e5014b28178a" "site/built/setup.md" "2024-07-25"
"learners/setup.md" "67749e8099ace68108310de95ec0a636" "site/built/setup.md" "2024-07-25"
"profiles/learner-profiles.md" "60b93493cf1da06dfd63255d73854461" "site/built/learner-profiles.md" "2024-07-25"
2 changes: 1 addition & 1 deletion setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ The Carpentries maintain a list of common issues on their
::::::::::::::::::::::::::::::::::::::::::::::::::

Your workshop instructor may also ask that you install the python packages introduced in the
[first lesson](https://carpentries-lab.github.io/python-aos-lesson/01-conda/index.html)
[first lesson](https://carpentries-lab.github.io/python-aos-lesson/01-conda.html)
ahead of time.
You can do this via the command line or by using the Anaconda Navigator:

Expand Down

0 comments on commit 09e85d3

Please sign in to comment.