Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

19373 Update CI for queue services #2424

Merged
merged 26 commits into from
Feb 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
6fe54c9
Add workflow_dispatch to ci jobs
leodube-aot Jan 25, 2024
9f94d17
Update queue-services CI jobs
leodube-aot Jan 26, 2024
fa5d777
Update queue-services pyproject.toml files
leodube-aot Jan 26, 2024
27188fd
Update queue-services poetry.lock files
leodube-aot Jan 26, 2024
636ac84
Run black formatter on queue-services
leodube-aot Jan 30, 2024
0f9e3ff
Fix pylint errors on queue-services
leodube-aot Jan 30, 2024
a59197f
Update CI yaml files
leodube-aot Jan 30, 2024
42325d1
Run isort on queue-services
leodube-aot Jan 30, 2024
2b1a68b
Run black again on entity-auth
leodube-aot Jan 30, 2024
b51e026
Run black again on entity-filer
leodube-aot Jan 30, 2024
23eaeaa
Fix pylint issues on entity_auth
leodube-aot Jan 31, 2024
7ceda93
Fix pylint issues on entity_bn src
leodube-aot Jan 31, 2024
46a02bd
Fix pylint issues on entity_emailer src
leodube-aot Jan 31, 2024
9303d91
Fix pylint issues on entity_pay src
leodube-aot Jan 31, 2024
cfb28cb
Fix remaining lint src issues (except for filer)
leodube-aot Jan 31, 2024
88f3352
Fix remaining flake8 src issues (except for filer)
leodube-aot Jan 31, 2024
0c84953
Update yaml and toml files to use Backend CI
leodube-aot Jan 31, 2024
8a0f7aa
Run isort again
leodube-aot Jan 31, 2024
394ff05
Fix new flake8 issues
leodube-aot Jan 31, 2024
5e40e08
Add empty init files
leodube-aot Jan 31, 2024
bfa7a26
Run black on init files
leodube-aot Jan 31, 2024
493d264
Fix linting issues entity_filer
leodube-aot Feb 1, 2024
bdc390c
Update noqa comment placement in filer
leodube-aot Feb 1, 2024
e567322
Add pytest coverage and configuration
leodube-aot Feb 1, 2024
3993d53
Update pylint coverage config
leodube-aot Feb 1, 2024
23e7195
Update coverage.run config
leodube-aot Feb 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 7 additions & 110 deletions .github/workflows/business-auth-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,120 +6,17 @@ on:
paths:
- "queue_services/entity-auth/**"
- "queue_services/common/**"
workflow_dispatch:

defaults:
run:
shell: bash
working-directory: ./queue_services/entity-auth

jobs:
setup-job:
runs-on: ubuntu-20.04

if: github.repository == 'bcgov/lear'

steps:
- uses: actions/checkout@v3
- run: "true"

linting:
needs: setup-job
runs-on: ubuntu-20.04

strategy:
matrix:
python-version: [3.8]

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
make setup
- name: Lint with pylint
id: pylint
run: |
make pylint
- name: Lint with flake8
id: flake8
run: |
make flake8

testing:
needs: setup-job
env:
DATABASE_TEST_USERNAME: postgres
DATABASE_TEST_PASSWORD: postgres
DATABASE_TEST_NAME: postgres
DATABASE_TEST_HOST: localhost
DATABASE_HOST: localhost
DATABASE_PASSWORD: postgres
NATS_SERVERS: "nats://nats:4222"
NATS_CLIENT_NAME: entity.legal_api
NATS_CLUSTER_ID: test-cluster
NATS_FILER_SUBJECT: entity.filing.filer
NATS_QUEUE: entity-auth-worker
TEST_NATS_DOCKER: True
STAN_CLUSTER_NAME: test-cluster
JWT_OIDC_JWKS_CACHE_TIMEOUT: 300
GO_LIVE_DATE: 2019-08-12
LEGAL_API_URL: https://mock_legal_api_url
ACCOUNT_SVC_ENTITY_URL: https://mock_account_svc_entity_url
COLIN_API: https://mock_colin_api_url
ACCOUNT_SVC_AUTH_URL: https://mock_account_svc_auth_url
ACCOUNT_SVC_CLIENT_ID: account_svc_client_id
ACCOUNT_SVC_CLIENT_SECRET: account_svc_client_secret
BUSINESS_SCHEMA_ID: test_business_schema_id
BUSINESS_CRED_DEF_ID: test_credential_definition_id
BUSINESS_SCHEMA_NAME: digital_business_card
BUSINESS_SCHEMA_VERSION: "1.0.0"


runs-on: ubuntu-20.04

services:
postgres:
image: postgres:12
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
make setup
- name: Test with pytest
id: test
run: |
make test
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
with:
file: ./queue_services/entity-auth/coverage.xml
flags: entityfiler
name: codecov-entity-auth
fail_ci_if_error: true

build-check:
needs: setup-job
runs-on: ubuntu-20.04

steps:
- uses: actions/checkout@v3
- name: build to check strictness
id: build
run: |
make build-nc
business-auth-ci:
uses: bcgov/bcregistry-sre/.github/workflows/backend-ci.yaml@main
with:
app_name: "business-auth"
working_directory: "./queue_services/entity-auth"
codecov_flag: "entityauth"
106 changes: 7 additions & 99 deletions .github/workflows/business-bn-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,109 +6,17 @@ on:
paths:
- "queue_services/entity-bn/**"
- "queue_services/common/**"
workflow_dispatch:

defaults:
run:
shell: bash
working-directory: ./queue_services/entity-bn

jobs:
setup-job:
runs-on: ubuntu-20.04

if: github.repository == 'bcgov/lear'

steps:
- uses: actions/checkout@v3
- run: "true"

linting:
needs: setup-job
runs-on: ubuntu-20.04

strategy:
matrix:
python-version: [3.8]

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
make setup
- name: Lint with pylint
id: pylint
run: |
make pylint
- name: Lint with flake8
id: flake8
run: |
make flake8

testing:
needs: setup-job
env:
DATABASE_TEST_USERNAME: postgres
DATABASE_TEST_PASSWORD: postgres
DATABASE_TEST_NAME: postgres
DATABASE_TEST_HOST: localhost
NATS_SERVERS: "nats://nats:4222"
NATS_CLIENT_NAME: entity.bn.tester
NATS_CLUSTER_ID: test-cluster
NATS_ENTITY_EVENT_SUBJECT: entity.events
NATS_QUEUE: entity-bn-worker
TEMPLATE_PATH: /home/runner/work/lear/lear/queue_services/entity-bn/src/entity_bn/bn_templates
TEST_NATS_DOCKER: True
STAN_CLUSTER_NAME: test-cluster
BN_HUB_API_URL: https://sometest:4443/rest/REST/BCPartner
BN_HUB_CLIENT_ID: id
BN_HUB_CLIENT_SECRET: secret

runs-on: ubuntu-20.04

services:
postgres:
image: postgres:12
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
make setup
- name: Test with pytest
id: test
run: |
make test
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
with:
file: ./queue_services/entity-bn/coverage.xml
flags: entitybn
name: codecov-entity-bn
fail_ci_if_error: true

build-check:
needs: setup-job
runs-on: ubuntu-20.04

steps:
- uses: actions/checkout@v3
- name: build to check strictness
id: build
run: |
make build-nc
business-bn-ci:
uses: bcgov/bcregistry-sre/.github/workflows/backend-ci.yaml@main
with:
app_name: "business-bn"
working_directory: "./queue_services/entity-bn"
codecov_flag: "entitybn"
122 changes: 7 additions & 115 deletions .github/workflows/business-emailer-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,125 +6,17 @@ on:
paths:
- "queue_services/entity-emailer/**"
- "queue_services/common/**"
workflow_dispatch:

defaults:
run:
shell: bash
working-directory: ./queue_services/entity-emailer

jobs:
setup-job:
runs-on: ubuntu-20.04

if: github.repository == 'bcgov/lear'

steps:
- uses: actions/checkout@v3
- run: "true"

linting:
needs: setup-job
runs-on: ubuntu-20.04

strategy:
matrix:
python-version: [3.8]

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
make setup
- name: Lint with pylint
id: pylint
run: |
make pylint
- name: Lint with flake8
id: flake8
run: |
make flake8

testing:
needs: setup-job
env:
DATABASE_TEST_USERNAME: postgres
DATABASE_TEST_PASSWORD: postgres
DATABASE_TEST_NAME: postgres
DATABASE_TEST_HOST: localhost
DATABASE_HOST: localhost
DATABASE_PASSWORD: postgres
TRACKER_DATABASE_TEST_USERNAME: postgres
TRACKER_DATABASE_TEST_PASSWORD: postgres
TRACKER_DATABASE_TEST_NAME: postgres
TRACKER_DATABASE_TEST_HOST: localhost
TRACKER_DATABASE_TEST_PORT: 5433
NATS_SERVERS: "nats://nats:4222"
NATS_CLIENT_NAME: entity.email.tester
NATS_CLUSTER_ID: test-cluster
NATS_FILER_SUBJECT: entity.email
NATS_QUEUE: entity-email-worker
JWT_OIDC_JWKS_CACHE_TIMEOUT: 300
TEMPLATE_PATH: /home/runner/work/lear/lear/queue_services/entity-emailer/src/entity_emailer/email_templates
DASHBOARD_URL: https://dev.bcregistry.ca/businesses/
TEST_NATS_DOCKER: True
STAN_CLUSTER_NAME: test-cluster

runs-on: ubuntu-20.04

services:
postgres:
image: postgres:12
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
postgres-tracker:
image: postgres:12
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
ports:
- 5433:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
make setup
- name: Test with pytest
id: test
run: |
make test
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
with:
file: ./queue_services/entity-emailer/coverage.xml
flags: entityemailer
name: codecov-entity-emailer
fail_ci_if_error: true

build-check:
needs: setup-job
runs-on: ubuntu-20.04

steps:
- uses: actions/checkout@v3
- name: build to check strictness
id: build
run: |
make build-nc
business-emailer-ci:
uses: bcgov/bcregistry-sre/.github/workflows/backend-ci.yaml@main
with:
app_name: "business-emailer"
working_directory: "./queue_services/entity-emailer"
codecov_flag: "entityemailer"
Loading
Loading