From ee4b5bfa5b348b3e5b244847509a75978648a932 Mon Sep 17 00:00:00 2001 From: Nick Moore Date: Wed, 1 Nov 2023 10:51:51 +1100 Subject: [PATCH] update docs with contribution instructions --- docs/config-file-format/index.md | 68 ++++++++++++++++++++++++------- docs/contributing/index.md | 49 ++++++++++++++++++++++ docs/index.md | 3 +- docs/installing-countess/index.md | 30 ++++++++++---- 4 files changed, 126 insertions(+), 24 deletions(-) create mode 100644 docs/contributing/index.md diff --git a/docs/config-file-format/index.md b/docs/config-file-format/index.md index 9eabc20..3615dee 100644 --- a/docs/config-file-format/index.md +++ b/docs/config-file-format/index.md @@ -8,41 +8,49 @@ While CountESS is intended to be used via the GUI, the config file is designed to be editable if necessary and to be compatible with revision control systems. -# Sections & Modules +## Sections & Modules -Each section of the file has the label of a node of the data flow graph. +.INI files are grouped into sections. +Each section corresponds to a node of the data flow graph, and +has the label of a node of the data flow graph. All nodes must +have a distinct name. [Load FASTQ] -The section contains key/value pairs which configure that node. +Each section contains key/value pairs which configure that node. Keys are hierarchical, with subkeys separated by '.' Keys and subkeys starting with `_` are reserved for CountESS. -Several key/value pairs exist to define the plugin module being +Several reserved key/value pairs exist to define the plugin module being used for this node: _module = countess.plugins.fastq _class = LoadFastqPlugin _version = 0.0.37 -CountESS also stores graphical layout information in the .INI file. -Currently this is stored as a major (flow-wards) and minor (sideways) -coordinate between 0 and 1000: +CountESS also uses a reserved key/value pair to store graphical layout +information. Currently this is stored as a major (flow-wards) and minor (sideways) +coordinate, each between 0 and 1000: _position = 55 500 -The connetions between nodes are stored as key/value pairs too: +## Connections + +The connections between nodes are stored as reserved key/value pairs too: [Join on "Run"] _parent.0 = Read File Descriptions _parent.1 = Break out "Run" -# Plugin Configuration +Plugins are written to the file "top down" so that a node never appears in +the config file before all of its parents do. + +## Plugin Configuration All other keys are defined by the plugin modules. Keys may not contain -`.` or `=` or probably some other values used in the .INI format. +`.` or `=` or other values used in the .INI format. -For now, values are stored in +Values are stored in Python [`repr()`](https://docs.python.org/3/library/functions.html#repr) format and interpreted using [`ast.literal_eval`](https://docs.python.org/3/library/ast.html#ast.literal_eval) @@ -59,15 +67,45 @@ Arrays of values are stored using numeric subkeys: When loading, subkeys are reinstated in numeric order but any missing values are skipped, so if you remove `files.1.filename` above everything will just work. +When re-writing config files from the GUI the array elements will be given +contiguous numbers again. -# Per-Column Arrays +## Per-Column Arrays -Where an array of values represents columns of a datatable, an additional -`_label` subkey records this mapping so it can be restored: +Where an array of values represents columns of a datatable, an additional `_label` +subkey records this mapping so it can be restored: columns.5._label = 'count' - columsn.5.sum = True + columns.5.sum = True + columns.5.count = True columns.7._label = 'variant' columns.7.index = True +## Working with Revision Control + +This file format should be well-behaved with revision control systems such as Git, +with some small caveats: + +* Duplicate section names are not allowed: + + [New Counter 1] + config = True + + [New Counter 1] + config = False + + To fix this, manually rename the sections. + +* Duplicate keys are not allowed, so revision control operations may end up with an + invalid state such as: + + files.0.filename = 'Alice\'s File' + files.0.filename = 'Bob\'s File' + + To fix this, manually renumber the files. + +* The order of nodes is generally preserved, but parents are always written before + their children, which means that changes to the topology may lead to large changes + in revision control. +These issues are all potentially addressable in future versions of the software. diff --git a/docs/contributing/index.md b/docs/contributing/index.md new file mode 100644 index 0000000..d7f00e2 --- /dev/null +++ b/docs/contributing/index.md @@ -0,0 +1,49 @@ +# Contributing to CountESS + +CountESS is an open source project and we welcome contributions. + +## Writing your own plugins + +You don't need to change anything in CountESS to write your own +plugins, or to coordinate with or ask permission from the core development +team! + +See [Writing CountESS Plugins](../writing-plugins/) for details. + +## Raising an issue + +If you've found a problem or have an enhancement you're interested in working on, +It's a good idea to check the open issues and raise a new issue if +your concerns aren't already addressed. This opens discussion with the core +development team and lets other people who might be interested know what you're +working on. + +## Installing for development + +First make your own fork in github. + +Then clone your fork locally and install for development: + + git clone $YOUR_GIT_REPOSITORY countess + cd countess + pip install -e .[dev] + +Edit files locally and test using: + + script/code-check + tests/test_gui.py + tests/cmd_tests.sh + +Please use code formatting tools to tidy your code up as this makes merging +changes easier: + + script/code-clean + +Upload your changes and raise a pull request through github so that +the CountESS core developers can test and review it. + +## Contributors Agreement + +TBA + + diff --git a/docs/index.md b/docs/index.md index d8f7502..c5d8ce5 100644 --- a/docs/index.md +++ b/docs/index.md @@ -13,8 +13,9 @@ under a [BSD 3-clause license](license/). * [Installing CountESS](installing-countess/) * [Running CountESS](running-countess/) -* [Writing Plugins](writing-plugins/) * [Configuration File Format](config-file-format/) +* [Writing Plugins](writing-plugins/) +* [Contributing to CountESS](contributing/) ## Presentations diff --git a/docs/installing-countess/index.md b/docs/installing-countess/index.md index cdef48e..adfdc25 100644 --- a/docs/installing-countess/index.md +++ b/docs/installing-countess/index.md @@ -18,9 +18,12 @@ Clone the repository and [install for development](https://pip.pypa.io/en/stable cd CountESS pip install -e .[dev] +See [Contributing to CountESS](../contributing/) for more information on development. + ## ... using nix -CountESS can be installed and run with nix: +CountESS can be installed and run in one command with +[nix](https://nixos.org/): nix run github:CountESS-Project/CountESS @@ -31,24 +34,35 @@ we use [homebrew](https://brew.sh/) to install more recent versions. 1. download the homebrew tarball & make its binaries available from your shell. - These instructions are for installing on a Mac without needing Administrator access. - If you already have homebrew installed you can skip this step: + *These instructions are for installing on a Mac + [without administrative privileges](https://docs.brew.sh/Installation#untar-anywhere-unsupported) + If you already have homebrew installed you can skip this step.* - mkdir bin homebrew + cd + mkdir homebrew curl -L https://github.com/Homebrew/brew/tarball/master | tar xz --strip 1 -C homebrew - echo 'export PATH=$HOME/bin:$HOME/homebrew/bin:$PATH' >> .bash_profile + echo 'export PATH=$HOME/homebrew/bin:$PATH' >> .bash_profile source .bash_profile -2. Use homebrew to install a recent Tk and Python 3.10 into your `$HOME/homebrew/bin` - directory. This step is pretty slow as it has to compile Tk and Python. +2. Use homebrew to install a recent Tk library and Python 3.10 into your `$HOME/homebrew/bin` + directory. This step is pretty slow: brew install tcl-tk python@3.10 python-tk@3.10 + + Optionally, add symlinks to your upgraded software for `python` and `pip`: + + cd + mkdir bin ln -s $HOME/homebrew/bin/python3.10 bin/python ln -s $HOME/homebrew/bin/pip3.10 bin/pip + echo 'export PATH=$HOME/bin:$PATH' >> .bash_profile + source .bash_profile 3. Install countess using pip: - pip install countess + pip3.10 install countess + + The CountESS binaries `countess_gui` and `countess_cli` should now be available from your shell. # NEXT