Repository containing scaffolding for a Python 3-based data science project with GPU acceleration using the Fast.ai ecosystem.
Simply follow the instructions to create a new project repository from this template.
Project organization is based on ideas from Good Enough Practices for Scientific Computing.
- Put each project in its own directory, which is named after the project.
- Put external scripts or compiled programs in the
bin
directory. - Put raw data and metadata in a
data
directory. - Put text documents associated with the project in the
doc
directory. - Put all Docker related files in the
docker
directory. - Install the Conda environment into an
env
directory. - Put all notebooks in the
notebooks
directory. - Put files generated during cleanup and analysis in a
results
directory. - Put project source code in the
src
directory. - Name all files to reflect their content or function.
After adding any necessary dependencies to the Conda environment.yml
file you can create the
environment in a sub-directory of your project directory by running the following command.
$ conda env create --prefix ./env --file environment.yml
Once the new environment has been created you can activate the environment with the following command.
$ conda activate ./env
Note that the env
directory is not under version control as it can always be re-created from
the environment.yml
file as necessary.
If you add (remove) dependencies to (from) the environment.yml
file after the environment has
already been created, then you can update the environment with the following command.
$ conda env update --prefix ./env --file environment.yml --prune
The list of explicit dependencies for the project are listed in the environment.yml
file. Too see the full lost of packages installed into the environment run the following command.
conda list --prefix ./env
If you wish to make use of the JupyterLab extensions included in the environment.yml
file, then you will need to run the postBuild
script after activating the environment to rebuild the client-side components of the extensions. Note that this step only needs to be done once (unless you add additional JupyterLab extensions).
$ conda activate ./env
(/path/to/project-dir/env)$ . postBuild
In order to build Docker images for your project and run containers with GPU acceleration you will need to install Docker, Docker Compose and the NVIDIA Docker runtime.
Detailed instructions for using Docker to build and image and launch containers can be found in
the docker/README.md
.