Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add minimal QC for NEO #90

Draft
wants to merge 12 commits into
base: master
Choose a base branch
from
Draft

Add minimal QC for NEO #90

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
a05d911
update odkfull to latest; work on #89
kltm Apr 11, 2022
363e128
add a gitignore from pipeline; nominally for cleaning for #89
kltm Apr 11, 2022
cf271ea
update on upstream data available; work on #89
kltm Apr 11, 2022
22758e3
see if fast enough for an obo-only build in GHA; work on #89
kltm Apr 11, 2022
4ad5ca0
see if fast enough for an obo-only build in GHA; work on #89
kltm Apr 11, 2022
405a493
make sure target is there; #89
kltm Apr 11, 2022
4327e65
less verbose output; #89
kltm Apr 11, 2022
b6c4d75
adjust memory; comment; for #89
kltm Apr 11, 2022
c03b714
adjust memory again; for #89
kltm Apr 11, 2022
dc61981
adjust memory again, lower default considering GHA mem availability; …
kltm Apr 11, 2022
9d93075
testing conditional for runoak/oaklib; for #89
kltm Apr 12, 2022
11558ad
target neo.owl at default location; for #89
kltm Apr 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

name: CI

# Controls when the action will run.
# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the master branch
push:
Expand All @@ -19,11 +19,12 @@ jobs:
test:
# The type of runner that the job will run on
runs-on: ubuntu-latest
container: obolibrary/odkfull:v1.2.23
container: obolibrary/odkfull:v1.3.0

# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2

- run: make test
- name: Create data environment
run: make test-gha
27 changes: 27 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
## NPM.
node_modules

## Emacs things (from
## https://github.com/github/gitignore/blob/master/Global/Emacs.gitignore).
*~
\#*\#
/.emacs.desktop
/.emacs.desktop.lock
.elc
auto-save-list
tramp
.\#*

## Org-mode
.org-id-locations
*_archive

# Pipeline artifacts
rules.log
pipeline/target/
pipeline/env
pipeline/*.log

*.egg-info
env
__pycache__
19 changes: 16 additions & 3 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
OBO = http://purl.obolibrary.org/obo

OWLTOOLS_MEMORY ?= 6G

all: target all_obo neo.obo neo.owl

clean:
Expand All @@ -12,9 +14,20 @@ OBO_SRCS = $(patsubst %,target/neo-%.obo,$(SRCS))
all_obo: $(OBO_SRCS)
test_obo: target $(patsubst %,target/neo-%.obo,$(TEST_SRCS))

## For future tests in a glorious future.
#test-gha: touch_trigger test_obo
#test-gha: target all_obo neo.obo
test-gha:
echo "test disabled until GH is fast enough to allow data build"

## "Local" tests for full (expensive) data tests.
#test: touch_trigger test_obo
test:
echo "tests disabled until its easier to run perl on travis"
ifeq (, $(shell which runoak))
echo "test disabled until we have runoak in our environment"
else
runoak --input neo.owl info GO:0022008
endif

touch_trigger:
touch trigger
Expand All @@ -24,7 +37,7 @@ trigger:

IMPORTS = imports/pr_import.obo
neo.obo: $(OBO_SRCS) $(IMPORTS)
owltools --create-ontology http://purl.obolibrary.org/obo/go/noctua/neo.owl $^ --merge-support-ontologies -o -f obo $@.tmp && grep -v ^owl-axioms $@.tmp > $@
OWLTOOLS_MEMORY=$(OWLTOOLS_MEMORY) owltools --create-ontology http://purl.obolibrary.org/obo/go/noctua/neo.owl $^ --merge-support-ontologies -o -f obo $@.tmp && grep -v ^owl-axioms $@.tmp > $@

## datasets.json is created as a throwaway in the NEO versions of the
## pipeline and is based on the go-site master data.
Expand Down Expand Up @@ -61,7 +74,7 @@ target/neo-goa_sars-cov-2.obo: mirror/goa_sars-cov-2.gpi.gz
mirror/uniprot_reviewed.gpi.gz: datasets.json
wget --no-check-certificate http://ftp.ebi.ac.uk/pub/contrib/goa/uniprot_reviewed.gpi.gz -O mirror/uniprot_reviewed.gpi.gz.tmp
gzip -dc mirror/uniprot_reviewed.gpi.gz.tmp > mirror/uniprot_reviewed.gpi.tmp
perl filter.pl -v --metadata datasets.json --filter filter_list.txt --input mirror/uniprot_reviewed.gpi.tmp > mirror/filtered_uniprot_reviewed.gpi.tmp
perl filter.pl --metadata datasets.json --filter filter_list.txt --input mirror/uniprot_reviewed.gpi.tmp > mirror/filtered_uniprot_reviewed.gpi.tmp
gzip -c mirror/filtered_uniprot_reviewed.gpi.tmp > mirror/filtered_uniprot_reviewed.gpi.gz.tmp
mv mirror/filtered_uniprot_reviewed.gpi.gz.tmp mirror/uniprot_reviewed.gpi.gz
target/neo-uniprot_reviewed.obo: mirror/uniprot_reviewed.gpi.gz
Expand Down
8 changes: 0 additions & 8 deletions Makefile-gafs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,3 @@
mirror/aspgd-20211215.gaf.gz:
wget --no-check-certificate https://go-public.s3.amazonaws.com/files/aspgd-20211215.gaf.gz -O $@.tmp && mv $@.tmp $@


target/neo-aspgd.obo: mirror/aspgd-20211215.gaf.gz
gzip -dc mirror/aspgd-20211215.gaf.gz | ./gaf2obo.pl -s Aspergillus -n aspgd > $@.tmp && mv $@.tmp $@


mirror/gene_association.cgd.gz:
wget --no-check-certificate http://www.candidagenome.org/download/go/gene_association.cgd.gz -O $@.tmp && mv $@.tmp $@

Expand Down