cwl-tes submits your tasks to a TES server. Task submission is parallelized when possible.
Funnel is an implementation of the GA4GH task execution API. It runs your dockerized tasks on slurm, htcondor, google compute engine, aws batch, etc.
- Start the task server
funnel server run
- Run your CWL tool/workflow
cwl-tes --tes http://localhost:8000 tests/hashsplitter-workflow.cwl.yml --input tests/resources/test.txt
I strongly recommend using a virtualenv for installation since cwl-tes depends on a specific version of cwltool.
Install from pip:
pip install cwl-tes
Install from source:
python setup.py install
To start a funnel server instance automatically and run all of the tests, install tox and run it
$ pip install tox
$ tox
For running only the conformance tests in python 2.7:
$ tox -e py27-unit
In a similar way they can be run on any supported python interpreter.
A more manual approach:
Download the conformance tests:
git submodule update --init --recursive
Start the funnel server.
funnel server --config /path/to/config.yaml
Make sure that TMPDIR is specified in the AllowedDirs of your Local storage configuration.
To run all the tests:
./tests/run_conformance.sh
To run a specifc test:
./tests/run_conformance.sh 10