Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build the sv-base docker using GATK released images instead of compiling from the given commit #487

Open
VJalili opened this issue Jan 23, 2023 · 0 comments
Labels

Comments

@VJalili
Copy link
Member

VJalili commented Jan 23, 2023

Affected module(s) or script(s)

  • sv-base docker image

Description

The sv-base docker image is built by compiling GATK from a particular git commit, which is motivated by the need for the un-merged/un-released tool PrintSVEvidence. This tool is used in multiple workflows.

RUN git clone https://github.com/broadinstitute/gatk.git && \
cd gatk && \
git lfs install && \
git checkout ${GATK_COMMIT}
# build GATK_JAR
RUN cd gatk && \
./gradlew localJar && \
mv $(readlink -f build/libs/gatk.jar) ${GATK_JAR}

Building the sv-base docker image as above has a few drawbacks that impede the image build on the GitHub actions.

  • It requires a full git history checkout at cloning to check out the particular commit. The full git history checkout of GATK is a slow operation on Github runners
  • Pulling LFS-tracked files results in downloading ~14GB data, which is (a) a very slow operation on Github runners, and (b) leads to out-of-disk issues.

Downloading LFS-tracked files, full history cloning together with compiling/building of GATK, adds to the total runtime of building docker images, resulting in over 6h of runtime on Github actions for building all the Docker images, which is force-killed by Github.

Building sv-base leveraging released GATK docker images can potentially alleviate the building challenges, which will require updating workflows and docker images where PrintSVEvidence is needed. Maybe the latter can be addressed as part of #324.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants