Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add dorado #256

Closed
wants to merge 24 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
533a7e7
Template update for nf-core/tools version 2.9
nf-core-bot Jun 30, 2023
8e16866
Template update for nf-core/tools version 2.10
nf-core-bot Sep 25, 2023
1867595
fast5_to_pod5 and dorado (not demultiplexed)
yuukiiwa Oct 17, 2023
b664520
update
yuukiiwa Oct 17, 2023
b6309f2
update ci test pointers
yuukiiwa Oct 18, 2023
9ab2f0f
fixes
yuukiiwa Oct 18, 2023
7a8cce9
fix
yuukiiwa Oct 18, 2023
510adfb
editor config linting fix
yuukiiwa Oct 18, 2023
db1eaaf
linting fix
yuukiiwa Oct 18, 2023
183fd5c
Template update for nf-core/tools version 2.11
nf-core-bot Dec 19, 2023
b3a51bb
Template update for nf-core/tools version 2.11.1
nf-core-bot Dec 20, 2023
16dc59e
Template update for nf-core/tools version 2.12
nf-core-bot Jan 29, 2024
04614ea
Template update for nf-core/tools version 2.13
nf-core-bot Feb 20, 2024
b884a45
Template update for nf-core/tools version 2.13.1
nf-core-bot Feb 29, 2024
2d70e1a
Update main.nf
LaurenceKuhl Apr 22, 2024
352d2f9
remove checker
yuukiiwa Apr 26, 2024
d3d089d
clean up
yuukiiwa Apr 26, 2024
5738424
clean up
yuukiiwa Apr 26, 2024
510ce56
Template update for nf-core/tools version 2.14.0
nf-core-bot May 8, 2024
d2a0e22
Template update for nf-core/tools version 2.14.1
nf-core-bot May 9, 2024
5e61982
Merge pull request #266 from LaurenceKuhl/master
yuukiiwa Aug 26, 2024
7922b5f
fix merge conflicts
yuukiiwa Sep 17, 2024
fa14487
Merge pull request #268 from nf-core/nf-core-template-merge-2.14.1
yuukiiwa Sep 17, 2024
a92abac
nextflow config unparsable
yuukiiwa Sep 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 2 additions & 9 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,15 @@
"name": "nfcore",
"image": "nfcore/gitpod:latest",
"remoteUser": "gitpod",
"runArgs": ["--privileged"],

// Configure tool-specific properties.
"customizations": {
// Configure properties specific to VS Code.
"vscode": {
// Set *default* container specific settings.json values on container create.
"settings": {
"python.defaultInterpreterPath": "/opt/conda/bin/python",
"python.linting.enabled": true,
"python.linting.pylintEnabled": true,
"python.formatting.autopep8Path": "/opt/conda/bin/autopep8",
"python.formatting.yapfPath": "/opt/conda/bin/yapf",
"python.linting.flake8Path": "/opt/conda/bin/flake8",
"python.linting.pycodestylePath": "/opt/conda/bin/pycodestyle",
"python.linting.pydocstylePath": "/opt/conda/bin/pydocstyle",
"python.linting.pylintPath": "/opt/conda/bin/pylint"
"python.defaultInterpreterPath": "/opt/conda/bin/python"
},

// Add the IDs of extensions you want installed when the container is created.
Expand Down
11 changes: 10 additions & 1 deletion .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,16 @@ end_of_line = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
indent_style = unset
indent_size = unset
[/subworkflows/nf-core/**]
charset = unset
end_of_line = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
indent_style = unset

[/assets/email*]
indent_size = unset

# ignore python and markdown
[*.{py,md}]
indent_style = unset
13 changes: 9 additions & 4 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,8 @@ Please use the pre-filled template to save time.
However, don't be put off by this template - other more general issues and suggestions are welcome!
Contributions to the code are even more welcome ;)

:::info
If you need help using or modifying nf-core/nanoseq then the best place to ask is on the nf-core Slack [#nanoseq](https://nfcore.slack.com/channels/nanoseq) channel ([join our Slack here](https://nf-co.re/join/slack)).
:::
> [!NOTE]
> If you need help using or modifying nf-core/nanoseq then the best place to ask is on the nf-core Slack [#nanoseq](https://nfcore.slack.com/channels/nanoseq) channel ([join our Slack here](https://nf-co.re/join/slack)).

## Contribution workflow

Expand All @@ -27,6 +26,12 @@ If you're not used to this workflow with git, you can start with some [docs from

## Tests

You have the option to test your changes locally by running the pipeline. For receiving warnings about process selectors and other `debug` information, it is recommended to use the debug profile. Execute all the tests with the following command:

```bash
nf-test test --profile debug,test,docker --verbose
```

When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests.
Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.

Expand Down Expand Up @@ -87,7 +92,7 @@ Once there, use `nf-core schema build` to add to `nextflow_schema.json`.

Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.

The process resources can be passed on to the tool dynamically within the process with the `${task.cpu}` and `${task.memory}` variables in the `script:` block.
The process resources can be passed on to the tool dynamically within the process with the `${task.cpus}` and `${task.memory}` variables in the `script:` block.

### Naming schemes

Expand Down
1 change: 1 addition & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/nano
- [ ] If necessary, also make a PR on the nf-core/nanoseq _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] Make sure your code lints (`nf-core lint`).
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
- [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`).
- [ ] Usage Documentation in `docs/usage.md` is updated.
- [ ] Output Documentation in `docs/output.md` is updated.
- [ ] `CHANGELOG.md` is updated.
Expand Down
19 changes: 12 additions & 7 deletions .github/workflows/awsfulltest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,27 +8,32 @@ on:
types: [published]
workflow_dispatch:
jobs:
run-tower:
run-platform:
name: Run AWS full tests
if: github.repository == 'nf-core/nanoseq'
runs-on: ubuntu-latest
steps:
- name: Launch workflow via tower
uses: seqeralabs/action-tower-launch@v1
- name: Launch workflow via Seqera Platform
uses: seqeralabs/action-tower-launch@v2
# TODO nf-core: You can customise AWS full pipeline tests as required
# Add full size test data (but still relatively small datasets for few samples)
# on the `test_full.config` test runs with only one set of parameters
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
revision: ${{ github.sha }}
workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/nanoseq/work-${{ github.sha }}
parameters: |
{
"hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/nanoseq/results-${{ github.sha }}"
}
profiles: test_full,aws_tower
- uses: actions/upload-artifact@v3
profiles: test_full

- uses: actions/upload-artifact@v4
with:
name: Tower debug log file
path: tower_action_*.log
name: Seqera Platform debug log file
path: |
seqera_platform_action_*.log
seqera_platform_action_*.json
20 changes: 12 additions & 8 deletions .github/workflows/awstest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,29 @@ name: nf-core AWS test
on:
workflow_dispatch:
jobs:
run-tower:
run-platform:
name: Run AWS tests
if: github.repository == 'nf-core/nanoseq'
runs-on: ubuntu-latest
steps:
# Launch workflow using Tower CLI tool action
- name: Launch workflow via tower
uses: seqeralabs/action-tower-launch@v1
# Launch workflow using Seqera Platform CLI tool action
- name: Launch workflow via Seqera Platform
uses: seqeralabs/action-tower-launch@v2
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
revision: ${{ github.sha }}
workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/nanoseq/work-${{ github.sha }}
parameters: |
{
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/nanoseq/results-test-${{ github.sha }}"
}
profiles: test,aws_tower
- uses: actions/upload-artifact@v3
profiles: test

- uses: actions/upload-artifact@v4
with:
name: Tower debug log file
path: tower_action_*.log
name: Seqera Platform debug log file
path: |
seqera_platform_action_*.log
seqera_platform_action_*.json
2 changes: 1 addition & 1 deletion .github/workflows/branch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
# NOTE - this doesn't currently work if the PR is coming from a fork, due to limitations in GitHub actions secrets
- name: Post PR comment
if: failure()
uses: mshick/add-pr-comment@v1
uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2
with:
message: |
## This PR is against the `master` branch :x:
Expand Down
19 changes: 12 additions & 7 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,17 +24,20 @@ jobs:
strategy:
matrix:
NXF_VER:
- "22.10.1"
- "23.04.0"
- "latest-everything"
steps:
- name: Check out pipeline code
uses: actions/checkout@v3
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4

- name: Install Nextflow
uses: nf-core/setup-nextflow@v1
uses: nf-core/setup-nextflow@v2
with:
version: "${{ matrix.NXF_VER }}"

- name: Disk space cleanup
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1

- name: Run pipeline with test data
run: |
nextflow run ${GITHUB_WORKSPACE} -profile test,docker --outdir ./results
Expand All @@ -49,10 +52,12 @@ jobs:
strategy:
matrix:
profiles:
- "test_nodx_vc"
- "test_nodx_stringtie"
- "test_nodx_noaln"
- "test_nodx_rnamod"
- "test_bc_nodx"
- "test_nobc_dx"
- "test_nobc_nodx_vc"
- "test_nobc_nodx_stringtie"
- "test_nobc_nodx_noaln"
- "test_nobc_nodx_rnamod"
steps:
- name: Check out pipeline code
uses: actions/checkout@v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/clean-up.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v7
- uses: actions/stale@28ca1036281a5e5922ead5184a1bbf96e5fc984e # v9
with:
stale-issue-message: "This issue has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment otherwise this issue will be closed in 20 days."
stale-pr-message: "This PR has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment if it is still useful."
Expand Down
86 changes: 86 additions & 0 deletions .github/workflows/download_pipeline.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
name: Test successful pipeline download with 'nf-core download'

# Run the workflow when:
# - dispatched manually
# - when a PR is opened or reopened to master branch
# - the head branch of the pull request is updated, i.e. if fixes for a release are pushed last minute to dev.
on:
workflow_dispatch:
inputs:
testbranch:
description: "The specific branch you wish to utilize for the test execution of nf-core download."
required: true
default: "dev"
pull_request:
types:
- opened
- edited
- synchronize
branches:
- master
pull_request_target:
branches:
- master

env:
NXF_ANSI_LOG: false

jobs:
download:
runs-on: ubuntu-latest
steps:
- name: Install Nextflow
uses: nf-core/setup-nextflow@v2

- name: Disk space cleanup
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1

- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5
with:
python-version: "3.12"
architecture: "x64"
- uses: eWaterCycle/setup-singularity@931d4e31109e875b13309ae1d07c70ca8fbc8537 # v7
with:
singularity-version: 3.8.3

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install git+https://github.com/nf-core/tools.git@dev

- name: Get the repository name and current branch set as environment variable
run: |
echo "REPO_LOWERCASE=${GITHUB_REPOSITORY,,}" >> ${GITHUB_ENV}
echo "REPOTITLE_LOWERCASE=$(basename ${GITHUB_REPOSITORY,,})" >> ${GITHUB_ENV}
echo "REPO_BRANCH=${{ github.event.inputs.testbranch || 'dev' }}" >> ${GITHUB_ENV}

- name: Download the pipeline
env:
NXF_SINGULARITY_CACHEDIR: ./
run: |
nf-core download ${{ env.REPO_LOWERCASE }} \
--revision ${{ env.REPO_BRANCH }} \
--outdir ./${{ env.REPOTITLE_LOWERCASE }} \
--compress "none" \
--container-system 'singularity' \
--container-library "quay.io" -l "docker.io" -l "ghcr.io" \
--container-cache-utilisation 'amend' \
--download-configuration

- name: Inspect download
run: tree ./${{ env.REPOTITLE_LOWERCASE }}

- name: Run the downloaded pipeline (stub)
id: stub_run_pipeline
continue-on-error: true
env:
NXF_SINGULARITY_CACHEDIR: ./
NXF_SINGULARITY_HOME_MOUNT: true
run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -stub -profile test,singularity --outdir ./results
- name: Run the downloaded pipeline (stub run not supported)
id: run_pipeline
if: ${{ job.steps.stub_run_pipeline.status == failure() }}
env:
NXF_SINGULARITY_CACHEDIR: ./
NXF_SINGULARITY_HOME_MOUNT: true
run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -profile test,singularity --outdir ./results
Loading
Loading