Table of Contents
1. Develop readable, reusable, & extendable codebases; 2. Build technical frameworks that improve productivity & reproducibility; 3. Easily leverage previous work to accelerate data-driven discovery.
The template has been loosely based on the Model-View-Controller (MVC) design pattern widely used in building web applications, but adapted here to suit scientific software development. The MVC approach allows us to separate out the various parts of a given software project according to their function (i.e. data processing, input handling, output generating, etc), organising the project logically so that it's easier to resuse, extend and maintain.
As a quick primer on how MVC has been interepreted in the context of a scientific research project:
MODEL - The model handles application data, and the rules for manipulating and transforming that data. You could think of the model as e.g. a series of functions that operate on data objects. For example, say you have a series of functions contained within the file diversityindex.jl. This file would be created inside the model directory, along with e.g. size_distribution.jl, both of which may be considered as separate logical modules within your project.
VIEW - The view, broadly speaking, is a visualisation of the state of the model. For example, you might have developed a block of code within a file called heatmap.jl that takes the output from diversity_index.jl and produces a heatmap. This file would be contained within the view directory.
CONTROLLER - I like to think of the controller as procedural mediator between model and view functionality. For clarity, consider the following context-specific workflow:
- DiversityIndexController.jl takes a dataset from the data/raw directory in the root of the project and passes it to a series of funtions in diversity_index.jl;
- diversity_index.jl passes a transformed dataset back to DiversityIndexController.jl, which proceeds to save the output into the processed data directory, before passing it to heatmap.jl;
- heatmap.jl generates the plot, and saves it to results/figures directory
This project was built using Cookiecutter: a command-line utility that allows us to develop custom templates for quickly bootstrapping the set-up of software projects while avoiding common mistakes.
First, satisfy Cookiecutter's Python requirements, then follow the Cookiecutter installation instructions according to your own system and preferences (apt-get, brew, pip, conda, etc):
- Python 2.7 or 3.5+
- Cookiecutter package
Please install a system appropriate version of Miniconda the package manager before proceeding.
Then, navigate to the python-specific README for further instructions.
Julia uses its own native system to manage virtual environments and dependancies (which has been implemented into the Makefile included with the template). Once you've downloaded Cookiecutter and its Python requirements, please navigate to the Julia-specific README for further set-up instructions.
├── LICENSE
├── Makefile <- Great alternative to e.g. bash scripts for automating stuff!
├── README.md <- Quick guide for other users/reviewers/future-you to learn about your work
│
├── data
│ ├── interim <- For storing intermediate, transformed data
│ ├── processed <- Final data ready to be used by models or in analyses
│ ├── raw <- The original, immutable data dump
│ └── imported <- Pre-processed data from another project/external source
│
│
├── notebooks <- Jupyter/Pluto notebooks for data exploration & method development
│ Naming convention: `1.0-initial-data-exploration`.
│
├── results
│ ├── papers <- Any completed/in-progress papers and notes
│ └── figures <- Final figures for reporting or publishing
│
├── requirements <- Package & dependency environment for the project
│ (requirements.txt in Python, Manifest.toml in Julia)
│
│
├── src <- Main body of code lives in here
│ ├── __init__.py <- Python-only - to treat src directory as a callable module
│ │
│ ├── controllers <- Scripts for mediating data & passing to models or views
│ │ │
│ │ ├── example_controller_1.py/.jl
│ │ └── example_controller_2.py/.jl
│ │
│ ├── models <- Scripts that process & manipulate data (algorithms, simulation, ML models)
│ │ │
│ │ ├── example_model_1.py/.jl
│ │ └── example_model_2.py/.jl
│ │
│ └── views <- Scripts for visualising outputs (e.g. figs, diagrams, tables)
│ │
│ ├── example_view_1.py/.jl
│ └── example_view_2.py/.jl
│
└── git-info.pdf <- Cheatsheet for common git commands - super important!
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement".
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Added amazing feature!') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Distributed under the MIT License. See License.txt for more information.
Lee Bardon - @teatauri - leerbardon@gmail.com
Project Link: https://github.com/leebardon/cookiecutter-for-scientists
- This cookiecutter was built using the excellent cookiecutter project
- It was influenced by the data science project template developed by Data Driven