Skip to content

Commit

Permalink
Merge branch 'master' into arc_corr_and_glob_check_fix
Browse files Browse the repository at this point in the history
  • Loading branch information
JoschD committed Oct 28, 2024
2 parents 51c2400 + feea136 commit 67570e0
Show file tree
Hide file tree
Showing 108 changed files with 3,424 additions and 4,252 deletions.
7 changes: 7 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners

# GLOBAL CODEOWNERS ---
# These owners will be the default owners for everything in
# the repo. Unless a later match takes precedence,
# they will be requested for review when someone opens a pull request.
* @pylhc/approved-reviewers
26 changes: 12 additions & 14 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,28 @@
# Continuous Integration Workflows
# Continous Integration Workflows

This package implements different workflows for CI.
They are organized as follows.
This package implements different workflows for CI, based on our organisation's common workflows.
They are organised as follows.

### Documentation
## Documentation

The `documentation` workflow triggers on any push to master, builds the documentation and pushes it to the `gh-pages` branch (if the build is successful).
It runs on `ubuntu-latest` and one of our supported version, currently `Python 3.9`.

### Testing Suite
## Testing Suite

Tests are ensured in the `tests` workflow, in two stages.
The first stage runs our simple tests on all push events (except to `master`), and the second one runs the rest of the testing suite (the `extended` tests).
Tests run on a matrix of all supported operating systems (`ubuntu-20.04`, `ubuntu-22.04`, `windows-latest` and `macos-latest`) for all supported Python versions (currently `3.8` to `3.11`).
Tests are ensured in the `tests` workflow, which triggers on all pushes.
It runs on a matrix of all supported operating systems for all supported Python versions.

### Test Coverage
## Test Coverage

Test coverage is calculated in the `coverage` workflow, which triggers on pushes to `master` and any push to a `pull request`.
It runs on `ubuntu-latest` and `Python 3.9`, and reports the coverage results of the test suite to `CodeClimate`.
Test coverage is calculated in the `coverage` wokflow, which triggers on pushes to `master` and any push to a `pull request`.
It reports the coverage results of the test suite to `CodeClimate`.

### Regular Testing
## Regular Testing

A `cron` workflow triggers every Monday at 3am (UTC time) and runs the full testing suite, on all available operating systems and supported Python versions.
It also runs on `Python 3.x` so that newly released Python versions that would break tests are automatically included.

### Publishing
## Publishing

Publishing to `PyPI` is done through the `publish` workflow, which triggers anytime a `release` is made of the GitHub repository.
It builds a `wheel`, checks it, and pushes to `PyPI` if checks are successful.
62 changes: 5 additions & 57 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
# Runs all tests and pushes coverage report to codeclimate
name: Coverage

defaults:
run:
shell: bash

on: # Runs on all push events to master branch and any push related to a pull request
push:
branches:
Expand All @@ -13,56 +9,8 @@ on: # Runs on all push events to master branch and any push related to a pull r

jobs:
coverage:
name: ${{ matrix.os }} / ${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
strategy:
matrix: # only a single supported Python on latest ubuntu
os: [ubuntu-latest]
python-version: [3.9]

steps:
- uses: actions/checkout@v3

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: '**/setup.py'

- name: Upgrade pip, setuptools and wheel
run: python -m pip install --upgrade pip setuptools wheel

- name: Install package
run: python -m pip install '.[test]'

- name: Set up env for CodeClimate (push)
run: |
echo "GIT_BRANCH=${GITHUB_REF/refs\/heads\//}" >> $GITHUB_ENV
echo "GIT_COMMIT_SHA=$GITHUB_SHA" >> $GITHUB_ENV
if: github.event_name == 'push'

- name: Set up env for CodeClimate (pull_request)
env:
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
echo "GIT_BRANCH=$GITHUB_HEAD_REF" >> $GITHUB_ENV
echo "GIT_COMMIT_SHA=$PR_HEAD_SHA" >> $GITHUB_ENV
if: github.event_name == 'pull_request'

- name: Prepare CodeClimate binary
env:
CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}
run: |
curl -LSs 'https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64' >./cc-test-reporter;
chmod +x ./cc-test-reporter
./cc-test-reporter before-build
- name: Run all tests
run: python -m pytest -m "not cern_network" --cov-report xml --cov=omc3

- name: Push Coverage to CodeClimate
if: ${{ success() }} # only if tests were successful
env:
CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}
run: ./cc-test-reporter after-build
uses: pylhc/.github/.github/workflows/coverage.yml@master
with:
src-dir: omc3
pytest-options: -m "not cern_network" --cov-report term-missing
secrets: inherit
38 changes: 6 additions & 32 deletions .github/workflows/cron.yml
Original file line number Diff line number Diff line change
@@ -1,39 +1,13 @@
# Runs all tests on master everyday at 10 am (UTC time)
# Runs all tests on master on Mondays at 3 am (UTC time)
name: Cron Testing

defaults:
run:
shell: bash

on: # Runs on master branch on Mondays at 3am UTC time
on:
schedule:
- cron: '* 3 * * mon'

jobs:
tests:
name: ${{ matrix.os }} / ${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04, ubuntu-22.04, macos-latest, windows-latest]
# Make sure to escape 3.10 with quotes so it doesn't get interpreted as float 3.1 by GA's parser
python-version: [3.8, 3.9, "3.10", "3.11", 3.x] # crons should always run latest python hence 3.x

steps:
- uses: actions/checkout@v3

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: '**/setup.py'

- name: Upgrade pip, setuptools and wheel
run: python -m pip install --upgrade pip setuptools wheel

- name: Install package
run: python -m pip install '.[test]'

- name: Run all tests
run: python -m pytest -m "not cern_network"
tests:
uses: pylhc/.github/.github/workflows/cron.yml@master
with:
pytest-options: -m "not cern_network"
46 changes: 1 addition & 45 deletions .github/workflows/documentation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,6 @@
# The build is published to github pages if the triggering event is a push to the master branch (PR merge)
name: Build and upload documentation

defaults:
run:
shell: bash

on: # Runs on any push event in a PR or any push event to master
pull_request:
push:
Expand All @@ -15,44 +11,4 @@ on: # Runs on any push event in a PR or any push event to master

jobs:
documentation:
name: ${{ matrix.os }} / ${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
strategy:
matrix: # only a single supported Python on latest ubuntu
os: [ubuntu-latest]
python-version: [3.9]

steps:
- uses: actions/checkout@v3

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: '**/setup.py'

- name: Upgrade pip, setuptools and wheel
run: python -m pip install --upgrade pip setuptools wheel

- name: Install package
run: python -m pip install '.[doc]'

- name: Build documentation
run: python -m sphinx -b html doc ./doc_build -d ./doc_build

- name: Upload build artifacts # upload artifacts so reviewers can have a quick look without building documentation from the branch locally
uses: actions/upload-artifact@v2
if: success() && github.event_name == 'pull_request' # only for pushes in PR
with:
name: site-build
path: doc_build
retention-days: 5

- name: Upload documentation to gh-pages
if: success() && github.ref == 'refs/heads/master' # only for pushes to master
uses: JamesIves/github-pages-deploy-action@3.6.2
with:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
BRANCH: gh-pages
FOLDER: doc_build
uses: pylhc/.github/.github/workflows/documentation.yml@master
39 changes: 2 additions & 37 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -1,46 +1,11 @@
# Publishes to PyPI upon creation of a release
name: Upload Package to PyPI

defaults:
run:
shell: bash

on: # Runs everytime a release is added to the repository
release:
types: [created]

jobs:
deploy:
name: ${{ matrix.os }} / ${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
strategy:
matrix: # only a single supported Python on latest ubuntu
os: [ubuntu-latest]
python-version: [3.9]


steps:
- uses: actions/checkout@v3

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: '**/setup.py'

- name: Upgrade pip, setuptools, wheel, build and twine
run: python -m pip install --upgrade pip setuptools wheel build twine

- name: Build and check build
run: |
python -m build
twine check dist/*
- name: Publish
if: ${{ success() }}
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
twine upload dist/*
uses: pylhc/.github/.github/workflows/publish.yml@master
secrets: inherit
75 changes: 12 additions & 63 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Runs all tests in two steps: the basic tests and the "extended" tests
name: Tests
# Runs all tests
name: All Tests

defaults:
run:
Expand All @@ -10,65 +10,14 @@ on: # Runs on any push event to any branch except master (the coverage workflow
branches-ignore:
- 'master'


jobs:
basic_tests: # Runs the basic tests, aka all tests not marked with "extended", on all push events
name: basic / ${{ matrix.os }} / ${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04, ubuntu-22.04, macos-latest, windows-latest]
# Make sure to escape 3.10 with quotes so it doesn't get interpreted as float 3.1 by GA's parser
python-version: [3.9, "3.10", "3.11"]

steps:
- uses: actions/checkout@v3

- if: matrix.os == 'macos-latest'
name: Set up hdf5
run: brew install hdf5

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: '**/setup.py'

- name: Upgrade pip, setuptools and wheel
run: python -m pip install --upgrade pip setuptools wheel

- name: Install package
run: python -m pip install '.[test]'

- name: Run Basic Tests
run: python -m pytest -m "not extended and not cern_network"


extended_tests: # Runs tests marked as "extended", after the previous step
needs: basic_tests # only here for aesthetics purpose
name: extended / ${{ matrix.os }} / ${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04, ubuntu-22.04, macos-latest, windows-latest]
python-version: [3.9, "3.10", "3.11"]

steps:
- uses: actions/checkout@v3

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache-dependency-path: '**/setup.py'

- name: Upgrade pip, setuptools and wheel
run: python -m pip install --upgrade pip setuptools wheel

- name: Install package
run: python -m pip install '.[test]'

- name: Run Extended Tests
run: python -m pytest -m "extended and not cern_network"
basic_tests:
uses: pylhc/.github/.github/workflows/tests.yml@master
with:
pytest-options: -m "not extended and not cern_network"

extended_tests:
needs: basic_tests
uses: pylhc/.github/.github/workflows/tests.yml@master
with:
pytest-options: -m "extended and not cern_network"
Loading

0 comments on commit 67570e0

Please sign in to comment.