Skip to content

Commit

Permalink
update docs with contribution instructions
Browse files Browse the repository at this point in the history
  • Loading branch information
nickzoic committed Oct 31, 2023
1 parent d2fcd74 commit ee4b5bf
Show file tree
Hide file tree
Showing 4 changed files with 126 additions and 24 deletions.
68 changes: 53 additions & 15 deletions docs/config-file-format/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,41 +8,49 @@ While CountESS is intended to be used via the GUI, the config file
is designed to be editable if necessary and to be compatible with
revision control systems.

# Sections & Modules
## Sections & Modules

Each section of the file has the label of a node of the data flow graph.
.INI files are grouped into sections.
Each section corresponds to a node of the data flow graph, and
has the label of a node of the data flow graph. All nodes must
have a distinct name.

[Load FASTQ]

The section contains key/value pairs which configure that node.
Each section contains key/value pairs which configure that node.
Keys are hierarchical, with subkeys separated by '.'

Keys and subkeys starting with `_` are reserved for CountESS.
Several key/value pairs exist to define the plugin module being
Several reserved key/value pairs exist to define the plugin module being
used for this node:

_module = countess.plugins.fastq
_class = LoadFastqPlugin
_version = 0.0.37

CountESS also stores graphical layout information in the .INI file.
Currently this is stored as a major (flow-wards) and minor (sideways)
coordinate between 0 and 1000:
CountESS also uses a reserved key/value pair to store graphical layout
information. Currently this is stored as a major (flow-wards) and minor (sideways)
coordinate, each between 0 and 1000:

_position = 55 500

The connetions between nodes are stored as key/value pairs too:
## Connections

The connections between nodes are stored as reserved key/value pairs too:

[Join on "Run"]
_parent.0 = Read File Descriptions
_parent.1 = Break out "Run"

# Plugin Configuration
Plugins are written to the file "top down" so that a node never appears in
the config file before all of its parents do.

## Plugin Configuration

All other keys are defined by the plugin modules. Keys may not contain
`.` or `=` or probably some other values used in the .INI format.
`.` or `=` or other values used in the .INI format.

For now, values are stored in
Values are stored in
Python [`repr()`](https://docs.python.org/3/library/functions.html#repr)
format and interpreted using
[`ast.literal_eval`](https://docs.python.org/3/library/ast.html#ast.literal_eval)
Expand All @@ -59,15 +67,45 @@ Arrays of values are stored using numeric subkeys:

When loading, subkeys are reinstated in numeric order but any missing values
are skipped, so if you remove `files.1.filename` above everything will just work.
When re-writing config files from the GUI the array elements will be given
contiguous numbers again.

# Per-Column Arrays
## Per-Column Arrays

Where an array of values represents columns of a datatable, an additional
`_label` subkey records this mapping so it can be restored:
Where an array of values represents columns of a datatable, an additional `_label`
subkey records this mapping so it can be restored:

columns.5._label = 'count'
columsn.5.sum = True
columns.5.sum = True
columns.5.count = True
columns.7._label = 'variant'
columns.7.index = True

## Working with Revision Control

This file format should be well-behaved with revision control systems such as Git,
with some small caveats:

* Duplicate section names are not allowed:

[New Counter 1]
config = True

[New Counter 1]
config = False

To fix this, manually rename the sections.

* Duplicate keys are not allowed, so revision control operations may end up with an
invalid state such as:

files.0.filename = 'Alice\'s File'
files.0.filename = 'Bob\'s File'

To fix this, manually renumber the files.

* The order of nodes is generally preserved, but parents are always written before
their children, which means that changes to the topology may lead to large changes
in revision control.

These issues are all potentially addressable in future versions of the software.
49 changes: 49 additions & 0 deletions docs/contributing/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Contributing to CountESS

CountESS is an open source project and we welcome contributions.

## Writing your own plugins

You don't need to change anything in CountESS to write your own
plugins, or to coordinate with or ask permission from the core development
team!

See [Writing CountESS Plugins](../writing-plugins/) for details.

## Raising an issue

If you've found a problem or have an enhancement you're interested in working on,
It's a good idea to check the open issues and raise a new issue if
your concerns aren't already addressed. This opens discussion with the core
development team and lets other people who might be interested know what you're
working on.

## Installing for development

First make your own fork in github.

Then clone your fork locally and install for development:

git clone $YOUR_GIT_REPOSITORY countess
cd countess
pip install -e .[dev]

Edit files locally and test using:

script/code-check
tests/test_gui.py
tests/cmd_tests.sh

Please use code formatting tools to tidy your code up as this makes merging
changes easier:

script/code-clean

Upload your changes and raise a pull request through github so that
the CountESS core developers can test and review it.

## Contributors Agreement

TBA


3 changes: 2 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@ under a [BSD 3-clause license](license/).

* [Installing CountESS](installing-countess/)
* [Running CountESS](running-countess/)
* [Writing Plugins](writing-plugins/)
* [Configuration File Format](config-file-format/)
* [Writing Plugins](writing-plugins/)
* [Contributing to CountESS](contributing/)

## Presentations

Expand Down
30 changes: 22 additions & 8 deletions docs/installing-countess/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,12 @@ Clone the repository and [install for development](https://pip.pypa.io/en/stable
cd CountESS
pip install -e .[dev]

See [Contributing to CountESS](../contributing/) for more information on development.

## ... using nix

CountESS can be installed and run with nix:
CountESS can be installed and run in one command with
[nix](https://nixos.org/):

nix run github:CountESS-Project/CountESS

Expand All @@ -31,24 +34,35 @@ we use [homebrew](https://brew.sh/) to install more recent versions.

1. download the homebrew tarball & make its binaries available from your shell.

These instructions are for installing on a Mac without needing Administrator access.
If you already have homebrew installed you can skip this step:
*These instructions are for installing on a Mac
[without administrative privileges](https://docs.brew.sh/Installation#untar-anywhere-unsupported)
If you already have homebrew installed you can skip this step.*

mkdir bin homebrew
cd
mkdir homebrew
curl -L https://github.com/Homebrew/brew/tarball/master | tar xz --strip 1 -C homebrew
echo 'export PATH=$HOME/bin:$HOME/homebrew/bin:$PATH' >> .bash_profile
echo 'export PATH=$HOME/homebrew/bin:$PATH' >> .bash_profile
source .bash_profile

2. Use homebrew to install a recent Tk and Python 3.10 into your `$HOME/homebrew/bin`
directory. This step is pretty slow as it has to compile Tk and Python.
2. Use homebrew to install a recent Tk library and Python 3.10 into your `$HOME/homebrew/bin`
directory. This step is pretty slow:

brew install tcl-tk python@3.10 python-tk@3.10

Optionally, add symlinks to your upgraded software for `python` and `pip`:

cd
mkdir bin
ln -s $HOME/homebrew/bin/python3.10 bin/python
ln -s $HOME/homebrew/bin/pip3.10 bin/pip
echo 'export PATH=$HOME/bin:$PATH' >> .bash_profile
source .bash_profile

3. Install countess using pip:

pip install countess
pip3.10 install countess

The CountESS binaries `countess_gui` and `countess_cli` should now be available from your shell.

# NEXT

Expand Down

0 comments on commit ee4b5bf

Please sign in to comment.