HCA Staging Import Validator

Runs a pre-check of a staging area to identify issues that might cause the snapshot or indexing processes to fail.

Note to develop on this repo, you will need to Setup Git Secrets.

Testing

NB - this can take over 10 hours to run, depending on the size of the staging area. In the future, we may want to add a fake test staging area to speed up testing.

python validate_staging_area.py --staging-area <gs_path> --ignore-dangling-inputs

PRs

Once you have tested locally, built and pushed to TestPyPi successfully, you can submit a PR. Current reviewers are aherbst-broad and danielsotirhos.

Building and publishing

We follow the basic python packaging and distribution guide. These instructions assume you are working from the repository root, and that you have appropriate permissions to the corresponding pypi project. It's encouraged that any changes to this package be tested via testpypi first. (NB you will need accounts on both PyPI and test.pypi.org, as well as permission to upload to this project in both.)

⚠ WARNING: Be sure to update the version number in the pyproject.toml file before building and uploading to (test)PyPI!

• Install build:

python -m pip install --upgrade build

• Build packages:

python -m build

• Install twine:

python -m pip install --upgrade twine

• Test the upload:

python -m twine upload --repository testpypi dist/*

• Upload the package to pypi (ideally using an API token)

python -m twine upload dist/*