diff --git a/.github/E2E_TESTS_ON_GITHUB_CI.md b/.github/E2E_TESTS_ON_GITHUB_CI.md index 572e0070a05..02144eff64f 100644 --- a/.github/E2E_TESTS_ON_GITHUB_CI.md +++ b/.github/E2E_TESTS_ON_GITHUB_CI.md @@ -53,87 +53,6 @@ For security reasons, test secrets and sensitive information are not stored dire If you need to run a GitHub workflow using custom secrets, please refer to the [guide on running GitHub workflows with your test secrets](https://github.com/smartcontractkit/chainlink-testing-framework/blob/main/lib/config/README.md#run-github-workflow-with-your-test-secrets). -## About the Reusable Workflow +## About the E2E Test Reusable Workflow -The [E2E Tests Reusable Workflow](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/run-e2e-tests-reusable-workflow.yml) is designed to run any type of E2E test on GitHub CI, including docker/testcontainers, old K8s tests, or tests in CRIB in the future. - -Our goal is to migrate all workflows to use this reusable workflow for executing E2E tests. This approach will streamline our CI and allow for the automatic execution of tests at different stages of the software development process. Learn more about the advantages of using reusable workflows [here](https://smartcontract-it.atlassian.net/wiki/spaces/TT/pages/815497220/CI+Workflows+for+E2E+Tests). - -**Examples of Workflows Utilizing the Reusable Workflow:** - -- [Integration Tests](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/integration-tests.yml) -- [Nightly E2E Tests](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/run-nightly-e2e-tests.yml) -- [Selected E2E Tests](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/run-selected-e2e-tests.yml) -- [On-Demand Automation Tests](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/automation-ondemand-tests.yml) -- [CCIP Chaos Tests](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/ccip-chaos-tests.yml) - -### E2E Test Configuration on GitHub CI - -The [e2e-tests.yml](https://github.com/smartcontractkit/chainlink/blob/develop/.github/e2e-tests.yml) file lists all E2E tests configured for execution on CI. Each entry specifies the type of GitHub Runner needed and the workflows in which the test is included (for example, `smoke/ocr_test.go:*` is executed both in PRs and nightly). - -### Slack Notification After Tests - -To configure Slack notifications after tests executed via the reusable workflow, follow these steps: - -- Set `slack_notification_after_tests` to either `always` or `on_failure` depending on when you want notifications to be sent. -- Assign `slack_notification_after_tests_channel_id` to the ID of the Slack channel where notifications should be sent. -- Provide a title for the notification by setting `slack_notification_after_tests_name`. -- Optionally use `slack_notification_after_tests_notify_user_id_on_failure` to reply in the thread and notify a user about the failed workflow - -**Example:** - -```yml -jobs: - call-run-e2e-tests-workflow: - name: Run E2E Tests - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml - with: - chainlink_version: develop - test_workflow: Nightly E2E Tests - slack_notification_after_tests: true - slack_notification_after_tests_channel_id: "#team-test-tooling-internal" - slack_notification_after_tests_name: Nightly E2E Tests - slack_notification_after_tests_notify_user_id_on_failure: U0XXXXXXX -``` - -## Guides - -### How to Run Selected Tests on GitHub CI - -The [Run Selected E2E Tests Workflow](https://github.com/smartcontractkit/chainlink/actions/workflows/run-selected-e2e-tests.yml) allows you to execute specific E2E tests as defined in the [e2e-tests.yml](https://github.com/smartcontractkit/chainlink/blob/develop/.github/e2e-tests.yml). This is useful for various purposes such as testing specific features on a particular branch or verifying that modifications to a test have not introduced any issues. - -For details on all available inputs that the workflow supports, refer to the [workflow definition](https://github.com/smartcontractkit/chainlink/actions/workflows/run-selected-e2e-tests.yml). - -**Example Usage:** - -To run a set of VRFv2Plus tests from a custom branch, use the following command: - -```bash -gh workflow run run-selected-e2e-tests.yml \ --f test_ids="TestVRFv2Plus_LinkBilling,TestVRFv2Plus_NativeBilling,TestVRFv2Plus_DirectFunding,TestVRFV2PlusWithBHS" \ --f chainlink_version=develop \ ---ref TT-1550-Provide-PoC-for-keeping-test-configs-in-git -``` - -### How to Run Custom Tests with Reusable Workflow - -To run a specific list of tests, utilize the `custom_test_list_json` input. This allows you to provide a customized list of tests. If your test list is dynamic, you can generate it during a preceding job and then reference it using: `custom_test_list_json: ${{ needs.gen_test_list.outputs.test_list }}`. - -```yml -run-e2e-tests-workflow: - name: Run E2E Tests - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml - with: - custom_test_list_json: > - { - "tests": [ - { - "id": "TestVRFv2Plus", - "path": "integration-tests/smoke/vrfv2plus_test.go", - "runs_on": "ubuntu-latest", - "test_env_type": "docker", - "test_cmd": "cd integration-tests/smoke && go test vrfv2plus_test.go" - } - ] - } -``` \ No newline at end of file +For information on the E2E Test Reusable Workflow, visit the documentation in the [smartcontractkit/.github repository](https://github.com/smartcontractkit/.github/blob/main/.github/workflows/README.md). diff --git a/.github/e2e-tests.yml b/.github/e2e-tests.yml index c3678837db6..4b2e8fe8309 100644 --- a/.github/e2e-tests.yml +++ b/.github/e2e-tests.yml @@ -3,7 +3,7 @@ # Each entry in this file includes the following: # - The GitHub runner (runs_on field) that will execute tests. # - The tests that will be run by the runner. -# - The workflows (e.g., Run PR E2E Tests, Nightly E2E Tests) that should trigger these tests. +# - The triggers (e.g., Run PR E2E Tests, Nightly E2E Tests) that should trigger these tests. # runner-test-matrix: @@ -14,7 +14,7 @@ runner-test-matrix: path: integration-tests/smoke/ocr_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -127,7 +127,7 @@ runner-test-matrix: path: integration-tests/smoke/forwarder_ocr_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -138,7 +138,7 @@ runner-test-matrix: path: integration-tests/smoke/forwarders_ocr2_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -149,7 +149,7 @@ runner-test-matrix: path: integration-tests/smoke/ocr2_test.go test_env_type: docker runs_on: ubuntu22.04-16cores-64GB - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -162,7 +162,7 @@ runner-test-matrix: path: integration-tests/smoke/ocr2_test.go test_env_type: docker runs_on: ubuntu22.04-16cores-64GB - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -176,7 +176,7 @@ runner-test-matrix: path: integration-tests/chaos/ocr_chaos_test.go test_env_type: k8s-remote-runner runs_on: ubuntu-latest - workflows: + triggers: - Automation On Demand Tests - E2E Chaos Tests test_cmd: cd integration-tests/chaos && DETACH_RUNNER=false go test -test.run "^TestOCRChaos$" -v -test.parallel=10 -timeout 60m -count=1 -json @@ -191,7 +191,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -202,7 +202,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -213,7 +213,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -224,7 +224,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -235,7 +235,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -246,7 +246,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -257,7 +257,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -268,7 +268,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -279,7 +279,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -290,7 +290,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -301,7 +301,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -312,7 +312,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -323,7 +323,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -334,7 +334,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -345,7 +345,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -356,7 +356,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -367,7 +367,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -378,7 +378,7 @@ runner-test-matrix: path: integration-tests/smoke/keeper_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Merge Queue E2E Core Tests - Nightly E2E Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestKeeperBasicSmoke$ -test.parallel=3 -timeout 30m -count=1 -json @@ -388,7 +388,7 @@ runner-test-matrix: path: integration-tests/smoke/keeper_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Merge Queue E2E Core Tests - Nightly E2E Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestKeeperBlockCountPerTurn$ -test.parallel=3 -timeout 30m -count=1 -json @@ -398,7 +398,7 @@ runner-test-matrix: path: integration-tests/smoke/keeper_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Merge Queue E2E Core Tests - Nightly E2E Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestKeeperSimulation$ -test.parallel=2 -timeout 30m -count=1 -json @@ -408,7 +408,7 @@ runner-test-matrix: path: integration-tests/smoke/keeper_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Merge Queue E2E Core Tests - Nightly E2E Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestKeeperCheckPerformGasLimit$ -test.parallel=2 -timeout 30m -count=1 -json @@ -418,7 +418,7 @@ runner-test-matrix: path: integration-tests/smoke/keeper_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Merge Queue E2E Core Tests - Nightly E2E Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestKeeperRegisterUpkeep$ -test.parallel=3 -timeout 30m -count=1 -json @@ -428,7 +428,7 @@ runner-test-matrix: path: integration-tests/smoke/keeper_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Merge Queue E2E Core Tests - Nightly E2E Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestKeeperAddFunds$ -test.parallel=3 -timeout 30m -count=1 -json @@ -438,7 +438,7 @@ runner-test-matrix: path: integration-tests/smoke/keeper_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Merge Queue E2E Core Tests - Nightly E2E Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestKeeperRemove$ -test.parallel=3 -timeout 30m -count=1 -json @@ -448,7 +448,7 @@ runner-test-matrix: path: integration-tests/smoke/keeper_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Merge Queue E2E Core Tests - Nightly E2E Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestKeeperPauseRegistry$ -test.parallel=2 -timeout 30m -count=1 -json @@ -458,7 +458,7 @@ runner-test-matrix: path: integration-tests/smoke/keeper_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Merge Queue E2E Core Tests - Nightly E2E Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestKeeperMigrateRegistry$ -test.parallel=1 -timeout 30m -count=1 -json @@ -468,7 +468,7 @@ runner-test-matrix: path: integration-tests/smoke/keeper_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Merge Queue E2E Core Tests - Nightly E2E Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestKeeperNodeDown$ -test.parallel=3 -timeout 30m -count=1 -json @@ -478,7 +478,7 @@ runner-test-matrix: path: integration-tests/smoke/keeper_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Merge Queue E2E Core Tests - Nightly E2E Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestKeeperPauseUnPauseUpkeep$ -test.parallel=1 -timeout 30m -count=1 -json @@ -488,7 +488,7 @@ runner-test-matrix: path: integration-tests/smoke/keeper_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Merge Queue E2E Core Tests - Nightly E2E Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestKeeperUpdateCheckData$ -test.parallel=1 -timeout 30m -count=1 -json @@ -498,7 +498,7 @@ runner-test-matrix: path: integration-tests/smoke/keeper_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Merge Queue E2E Core Tests - Nightly E2E Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestKeeperJobReplacement$ -test.parallel=1 -timeout 30m -count=1 -json @@ -520,7 +520,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_upgrade_test.go test_env_type: docker runs_on: ubuntu22.04-8cores-32GB - workflows: + triggers: - Automation Nightly Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestAutomationNodeUpgrade/registry_2_0 -test.parallel=1 -timeout 60m -count=1 -json test_env_vars: @@ -534,7 +534,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_upgrade_test.go test_env_type: docker runs_on: ubuntu22.04-8cores-32GB - workflows: + triggers: - Automation Nightly Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestAutomationNodeUpgrade/registry_2_1 -test.parallel=5 -timeout 60m -count=1 -json test_env_vars: @@ -548,7 +548,7 @@ runner-test-matrix: path: integration-tests/smoke/automation_upgrade_test.go test_env_type: docker runs_on: ubuntu22.04-8cores-32GB - workflows: + triggers: - Automation Nightly Tests test_cmd: cd integration-tests/smoke && go test -test.run ^TestAutomationNodeUpgrade/registry_2_2 -test.parallel=5 -timeout 60m -count=1 -json test_env_vars: @@ -564,7 +564,7 @@ runner-test-matrix: test_env_type: docker test_env_vars: TEST_SUITE: reorg - workflows: + triggers: - Automation On Demand Tests test_cmd: cd integration-tests/reorg && DETACH_RUNNER=false go test -v -test.run ^TestAutomationReorg/registry_2_0 -test.parallel=1 -timeout 30m -count=1 -json pyroscope_env: ci-automation-on-demand-reorg @@ -575,7 +575,7 @@ runner-test-matrix: test_env_type: docker test_env_vars: TEST_SUITE: reorg - workflows: + triggers: - Automation On Demand Tests test_cmd: cd integration-tests/reorg && DETACH_RUNNER=false go test -v -test.run ^TestAutomationReorg/registry_2_1 -test.parallel=2 -timeout 30m -count=1 -json pyroscope_env: ci-automation-on-demand-reorg @@ -586,7 +586,7 @@ runner-test-matrix: test_env_type: docker test_env_vars: TEST_SUITE: reorg - workflows: + triggers: - Automation On Demand Tests test_cmd: cd integration-tests/reorg && DETACH_RUNNER=false go test -v -test.run ^TestAutomationReorg/registry_2_2 -test.parallel=2 -timeout 30m -count=1 -json pyroscope_env: ci-automation-on-demand-reorg @@ -597,7 +597,7 @@ runner-test-matrix: test_env_type: docker test_env_vars: TEST_SUITE: reorg - workflows: + triggers: - Automation On Demand Tests test_cmd: cd integration-tests/reorg && DETACH_RUNNER=false go test -v -test.run ^TestAutomationReorg/registry_2_3 -test.parallel=2 -timeout 30m -count=1 -json pyroscope_env: ci-automation-on-demand-reorg @@ -606,7 +606,7 @@ runner-test-matrix: path: integration-tests/chaos/automation_chaos_test.go test_env_type: k8s-remote-runner runs_on: ubuntu-latest - workflows: + triggers: - Automation On Demand Tests - E2E Chaos Tests test_cmd: cd integration-tests/chaos && DETACH_RUNNER=false go test -v -test.run ^TestAutomationChaos$ -test.parallel=20 -timeout 60m -count=1 -json @@ -619,7 +619,7 @@ runner-test-matrix: test_env_type: k8s-remote-runner remote_runner_memory: 4Gi runs_on: ubuntu-latest - # workflows: + # triggers: # - Nightly E2E Tests test_cmd: cd integration-tests/benchmark && go test -v -test.run ^TestAutomationBenchmark$ -test.parallel=1 -timeout 30m -count=1 -json pyroscope_env: ci-benchmark-automation-nightly @@ -633,7 +633,7 @@ runner-test-matrix: test_env_type: k8s-remote-runner remote_runner_memory: 4Gi runs_on: ubuntu-latest - # workflows: + # triggers: # - Nightly E2E Tests test_cmd: cd integration-tests/benchmark && go test -v -test.run ^TestAutomationBenchmark$ -test.parallel=1 -timeout 30m -count=1 -json pyroscope_env: ci-benchmark-automation-nightly @@ -652,7 +652,7 @@ runner-test-matrix: test_env_type: docker test_cmd: cd integration-tests/smoke && go test -v -test.run TestVRFv2Basic -test.parallel=1 -timeout 30m -count=1 -json test_secrets_required: true - workflows: + triggers: - On Demand VRFV2 Smoke Test (Ethereum clients) - id: load/vrfv2plus/vrfv2plus_test.go:^TestVRFV2PlusPerformance$Smoke @@ -664,7 +664,7 @@ runner-test-matrix: test_secrets_required: true test_env_vars: TEST_TYPE: Smoke - workflows: + triggers: - On Demand VRFV2 Plus Performance Test - id: load/vrfv2plus/vrfv2plus_test.go:^TestVRFV2PlusBHSPerformance$Smoke @@ -676,7 +676,7 @@ runner-test-matrix: test_secrets_required: true test_env_vars: TEST_TYPE: Smoke - workflows: + triggers: - On Demand VRFV2 Plus Performance Test - id: load/vrfv2/vrfv2_test.go:^TestVRFV2Performance$Smoke @@ -688,7 +688,7 @@ runner-test-matrix: test_secrets_required: true test_env_vars: TEST_TYPE: Smoke - workflows: + triggers: - On Demand VRFV2 Performance Test - id: load/vrfv2/vrfv2_test.go:^TestVRFV2PlusBHSPerformance$Smoke @@ -700,14 +700,14 @@ runner-test-matrix: test_secrets_required: true test_env_vars: TEST_TYPE: Smoke - workflows: + triggers: - On Demand VRFV2 Performance Test - id: smoke/vrf_test.go:* path: integration-tests/smoke/vrf_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -718,7 +718,7 @@ runner-test-matrix: path: integration-tests/smoke/vrfv2_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -729,7 +729,7 @@ runner-test-matrix: path: integration-tests/smoke/vrfv2plus_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -751,7 +751,7 @@ runner-test-matrix: test_env_type: docker test_cmd: cd integration-tests/smoke && go test -v -test.run "TestVRFv2Plus$/(Link_Billing|Native_Billing|Direct_Funding)|TestVRFV2PlusWithBHS" -test.parallel=1 -timeout 2h -count=1 -json test_config_override_path: integration-tests/testconfig/vrfv2plus/overrides/new_env/sepolia_new_env_test_config.toml - workflows: + triggers: - VRF E2E Release Tests # END: VRF tests @@ -762,7 +762,7 @@ runner-test-matrix: path: integration-tests/smoke/log_poller_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -773,7 +773,7 @@ runner-test-matrix: path: integration-tests/smoke/log_poller_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -784,7 +784,7 @@ runner-test-matrix: path: integration-tests/smoke/log_poller_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -795,7 +795,7 @@ runner-test-matrix: path: integration-tests/smoke/log_poller_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -806,7 +806,7 @@ runner-test-matrix: path: integration-tests/smoke/log_poller_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -817,7 +817,7 @@ runner-test-matrix: path: integration-tests/smoke/log_poller_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -828,7 +828,7 @@ runner-test-matrix: path: integration-tests/smoke/log_poller_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -839,7 +839,7 @@ runner-test-matrix: path: integration-tests/smoke/log_poller_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -854,7 +854,7 @@ runner-test-matrix: path: integration-tests/smoke/runlog_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -865,7 +865,7 @@ runner-test-matrix: path: integration-tests/smoke/cron_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -876,7 +876,7 @@ runner-test-matrix: path: integration-tests/smoke/flux_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -887,7 +887,7 @@ runner-test-matrix: path: integration-tests/smoke/reorg_above_finality_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -898,7 +898,7 @@ runner-test-matrix: path: integration-tests/migration/upgrade_version_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -913,7 +913,7 @@ runner-test-matrix: path: integration-tests/smoke/job_distributor_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E Core Tests - Merge Queue E2E Core Tests - Nightly E2E Tests @@ -928,7 +928,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/smoke/ccip_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E CCIP Tests - Merge Queue E2E CCIP Tests - Nightly E2E Tests @@ -940,7 +940,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/smoke/ccip_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E CCIP Tests - Merge Queue E2E CCIP Tests - Nightly E2E Tests @@ -953,7 +953,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/smoke/ccip_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E CCIP Tests - Merge Queue E2E CCIP Tests - Nightly E2E Tests @@ -966,7 +966,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/smoke/ccip_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E CCIP Tests - Merge Queue E2E CCIP Tests - Nightly E2E Tests @@ -980,7 +980,7 @@ runner-test-matrix: test_env_type: docker runs_on: ubuntu-latest # Leader lane test is flakey in Core repo - Need to fix CCIP-3074 to enable it. - workflows: + triggers: # - PR E2E CCIP Tests # - Nightly E2E Tests test_cmd: cd integration-tests/ccip-tests/smoke && go test ccip_test.go -test.run ^TestSmokeCCIPForBidirectionalLane$ -timeout 30m -count=1 -test.parallel=1 -json @@ -992,7 +992,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/smoke/ccip_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E CCIP Tests - Merge Queue E2E CCIP Tests - Nightly E2E Tests @@ -1004,7 +1004,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/smoke/ccip_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E CCIP Tests - Merge Queue E2E CCIP Tests - Nightly E2E Tests @@ -1016,7 +1016,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/smoke/ccip_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E CCIP Tests - Merge Queue E2E CCIP Tests - Nightly E2E Tests @@ -1028,7 +1028,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/smoke/ccip_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E CCIP Tests - Merge Queue E2E CCIP Tests - Nightly E2E Tests @@ -1040,7 +1040,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/smoke/ccip_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Nightly E2E Tests test_cmd: cd integration-tests/ccip-tests/smoke && go test ccip_test.go -test.run ^TestSmokeCCIPOffRampCapacityLimit$ -timeout 30m -count=1 -test.parallel=1 -json test_env_vars: @@ -1050,7 +1050,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/smoke/ccip_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - Nightly E2E Tests test_cmd: cd integration-tests/ccip-tests/smoke && go test ccip_test.go -test.run ^TestSmokeCCIPOffRampAggRateLimit$ -timeout 30m -count=1 -test.parallel=1 -json test_env_vars: @@ -1060,7 +1060,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/smoke/ccip_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E CCIP Tests - Merge Queue E2E CCIP Tests - Nightly E2E Tests @@ -1073,7 +1073,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/smoke/ccip_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E CCIP Tests - Merge Queue E2E CCIP Tests - Nightly E2E Tests @@ -1086,7 +1086,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/smoke/ccip_test.go test_env_type: docker runs_on: ubuntu-latest - workflows: + triggers: - PR E2E CCIP Tests - Merge Queue E2E CCIP Tests - Nightly E2E Tests @@ -1103,7 +1103,7 @@ runner-test-matrix: test_env_vars: TEST_SUITE: ccip-load E2E_TEST_GRAFANA_DASHBOARD_URL: "/d/6vjVx-1V8/ccip-long-running-tests" - workflows: + triggers: - E2E CCIP Load Tests test_artifacts_on_failure: - ./integration-tests/load/logs/payload_ccip.json @@ -1118,7 +1118,7 @@ runner-test-matrix: # test_config_override_path: integration-tests/ccip-tests/testconfig/tomls/load-with-arm-curse-uncurse.toml # test_env_vars: # E2E_TEST_GRAFANA_DASHBOARD_URL: "/d/6vjVx-1V8/ccip-long-running-tests" - # workflows: + # triggers: # - E2E CCIP Load Tests # test_artifacts_on_failure: # - ./integration-tests/load/logs/payload_ccip.json @@ -1127,7 +1127,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/chaos/ccip_test.go test_env_type: k8s-remote-runner runs_on: ubuntu-latest - workflows: + triggers: - E2E CCIP Chaos Tests test_cmd: cd integration-tests/ccip-tests/chaos && DETACH_RUNNER=false go test ccip_test.go -v -test.parallel=11 -timeout 60m -count=1 -json test_env_vars: @@ -1139,7 +1139,7 @@ runner-test-matrix: path: integration-tests/ccip-tests/load/ccip_test.go test_env_type: k8s-remote-runner runs_on: ubuntu-latest - workflows: + triggers: # Disabled until CCIP-2555 is resolved # - E2E CCIP Chaos Tests test_cmd: cd integration-tests/ccip-tests/load && DETACH_RUNNER=false go test -run '^TestLoadCCIPStableWithPodChaosDiffCommitAndExec' -v -test.parallel=4 -timeout 120m -count=1 -json diff --git a/.github/workflows/automation-benchmark-tests.yml b/.github/workflows/automation-benchmark-tests.yml index d1af80fb91e..7d46b8e0c2b 100644 --- a/.github/workflows/automation-benchmark-tests.yml +++ b/.github/workflows/automation-benchmark-tests.yml @@ -24,8 +24,9 @@ on: jobs: run-e2e-tests-workflow: name: Run E2E Tests - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: + test_path: .github/e2e-tests.yml test_ids: '${{ inputs.testType }}/automation_test.go:TestAutomationBenchmark' test_config_override_path: ${{ inputs.test_config_override_path }} SLACK_USER: ${{ inputs.slackMemberID }} diff --git a/.github/workflows/automation-load-tests.yml b/.github/workflows/automation-load-tests.yml index 23c053203a1..5d37e81c143 100644 --- a/.github/workflows/automation-load-tests.yml +++ b/.github/workflows/automation-load-tests.yml @@ -19,8 +19,9 @@ on: jobs: run-e2e-tests-workflow: name: Run E2E Tests - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: + test_path: .github/e2e-tests.yml test_ids: 'load/automationv2_1/automationv2_1_test.go:TestLogTrigger' test_config_override_path: ${{ inputs.test_config_override_path }} SLACK_USER: ${{ inputs.slackMemberID }} diff --git a/.github/workflows/automation-nightly-tests.yml b/.github/workflows/automation-nightly-tests.yml index 06dc00b3a14..1ff80cff3cd 100644 --- a/.github/workflows/automation-nightly-tests.yml +++ b/.github/workflows/automation-nightly-tests.yml @@ -10,9 +10,10 @@ on: jobs: run-e2e-tests-workflow: name: Run E2E Tests - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: - test_workflow: Automation Nightly Tests + test_path: .github/e2e-tests.yml + test_trigger: Automation Nightly Tests chainlink_version: ${{ github.sha }} slack_notification_after_tests: true slack_notification_after_tests_channel_id: "#automation-test-notifications" diff --git a/.github/workflows/automation-ondemand-tests.yml b/.github/workflows/automation-ondemand-tests.yml index a5b26744253..053a0e91472 100644 --- a/.github/workflows/automation-ondemand-tests.yml +++ b/.github/workflows/automation-ondemand-tests.yml @@ -153,8 +153,9 @@ jobs: call-run-e2e-tests-workflow: name: Run E2E Tests needs: set-tests-to-run - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: + test_path: .github/e2e-tests.yml test_list: ${{ needs.set-tests-to-run.outputs.test_list }} require_chainlink_image_versions_in_qa_ecr: ${{ needs.set-tests-to-run.outputs.require_chainlink_image_versions_in_qa_ecr }} with_existing_remote_runner_version: ${{ github.event.inputs.with_existing_remote_runner_version }} diff --git a/.github/workflows/ccip-chaos-tests.yml b/.github/workflows/ccip-chaos-tests.yml index 3a6cae796d2..034f0e2cb34 100644 --- a/.github/workflows/ccip-chaos-tests.yml +++ b/.github/workflows/ccip-chaos-tests.yml @@ -14,11 +14,12 @@ concurrency: jobs: run-e2e-tests-workflow: name: Run E2E Tests - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: + test_path: .github/e2e-tests.yml chainlink_version: ${{ github.sha }} require_chainlink_image_versions_in_qa_ecr: ${{ github.sha }} - test_workflow: E2E CCIP Chaos Tests + test_trigger: E2E CCIP Chaos Tests test_log_level: debug slack_notification_after_tests: on_failure slack_notification_after_tests_channel_id: '#ccip-testing' diff --git a/.github/workflows/ccip-load-tests.yml b/.github/workflows/ccip-load-tests.yml index 235b9b0f67e..6464cccc739 100644 --- a/.github/workflows/ccip-load-tests.yml +++ b/.github/workflows/ccip-load-tests.yml @@ -31,9 +31,10 @@ concurrency: jobs: run-e2e-tests-workflow: name: Run E2E Tests - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: - test_workflow: E2E CCIP Load Tests + test_path: .github/e2e-tests.yml + test_trigger: E2E CCIP Load Tests test_config_override_path: ${{ inputs.test_config_override_path }} chainlink_version: ${{ inputs.chainlink_version || github.sha }} slack_notification_after_tests: always diff --git a/.github/workflows/integration-chaos-tests.yml b/.github/workflows/integration-chaos-tests.yml index c9da7d84381..3198320f6cf 100644 --- a/.github/workflows/integration-chaos-tests.yml +++ b/.github/workflows/integration-chaos-tests.yml @@ -10,11 +10,12 @@ on: jobs: run-e2e-tests-workflow: name: Run E2E Tests - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: + test_path: .github/e2e-tests.yml chainlink_version: ${{ github.sha }} require_chainlink_image_versions_in_qa_ecr: ${{ github.sha }} - test_workflow: E2E Chaos Tests + test_trigger: E2E Chaos Tests test_log_level: debug secrets: QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }} diff --git a/.github/workflows/integration-tests.yml b/.github/workflows/integration-tests.yml index 63893867993..79db8c2859d 100644 --- a/.github/workflows/integration-tests.yml +++ b/.github/workflows/integration-tests.yml @@ -242,12 +242,13 @@ jobs: contents: read needs: [build-chainlink, changes] if: github.event_name == 'pull_request' && ( needs.changes.outputs.core_changes == 'true' || needs.changes.outputs.github_ci_changes == 'true') - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: workflow_name: Run Core E2E Tests For PR chainlink_version: ${{ inputs.evm-ref || github.sha }} chainlink_upgrade_version: ${{ github.sha }} - test_workflow: PR E2E Core Tests + test_path: .github/e2e-tests.yml + test_trigger: PR E2E Core Tests upload_cl_node_coverage_artifact: true upload_cl_node_coverage_artifact_prefix: cl_node_coverage_data_ enable_otel_traces_for_ocr2_plugins: ${{ contains(join(github.event.pull_request.labels.*.name, ' '), 'enable tracing') }} @@ -281,12 +282,13 @@ jobs: contents: read needs: [build-chainlink, changes] if: github.event_name == 'merge_group' && ( needs.changes.outputs.core_changes == 'true' || needs.changes.outputs.github_ci_changes == 'true') - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: workflow_name: Run Core E2E Tests For Merge Queue chainlink_version: ${{ inputs.evm-ref || github.sha }} chainlink_upgrade_version: ${{ github.sha }} - test_workflow: Merge Queue E2E Core Tests + test_path: .github/e2e-tests.yml + test_trigger: Merge Queue E2E Core Tests upload_cl_node_coverage_artifact: true upload_cl_node_coverage_artifact_prefix: cl_node_coverage_data_ enable_otel_traces_for_ocr2_plugins: ${{ contains(join(github.event.pull_request.labels.*.name, ' '), 'enable tracing') }} @@ -324,12 +326,13 @@ jobs: contents: read needs: [build-chainlink, changes] if: github.event_name == 'pull_request' && (needs.changes.outputs.ccip_changes == 'true' || needs.changes.outputs.github_ci_changes == 'true') - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: workflow_name: Run CCIP E2E Tests For PR chainlink_version: ${{ inputs.evm-ref || github.sha }} chainlink_upgrade_version: ${{ github.sha }} - test_workflow: PR E2E CCIP Tests + test_path: .github/e2e-tests.yml + test_trigger: PR E2E CCIP Tests upload_cl_node_coverage_artifact: true upload_cl_node_coverage_artifact_prefix: cl_node_coverage_data_ enable_otel_traces_for_ocr2_plugins: ${{ contains(join(github.event.pull_request.labels.*.name, ' '), 'enable tracing') }} @@ -363,12 +366,13 @@ jobs: contents: read needs: [build-chainlink, changes] if: github.event_name == 'merge_group' && (needs.changes.outputs.ccip_changes == 'true' || needs.changes.outputs.github_ci_changes == 'true') - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: workflow_name: Run CCIP E2E Tests For Merge Queue chainlink_version: ${{ inputs.evm-ref || github.sha }} chainlink_upgrade_version: ${{ github.sha }} - test_workflow: Merge Queue E2E CCIP Tests + test_path: .github/e2e-tests.yml + test_trigger: Merge Queue E2E CCIP Tests upload_cl_node_coverage_artifact: true upload_cl_node_coverage_artifact_prefix: cl_node_coverage_data_ enable_otel_traces_for_ocr2_plugins: ${{ contains(join(github.event.pull_request.labels.*.name, ' '), 'enable tracing') }} diff --git a/.github/workflows/on-demand-ocr-soak-test.yml b/.github/workflows/on-demand-ocr-soak-test.yml index cdb0b5da01a..978c1eb67d0 100644 --- a/.github/workflows/on-demand-ocr-soak-test.yml +++ b/.github/workflows/on-demand-ocr-soak-test.yml @@ -39,8 +39,9 @@ on: jobs: run-e2e-tests-workflow: name: Run E2E Tests - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: + test_path: .github/e2e-tests.yml test_ids: ${{ inputs.testToRun}} test_config_override_path: ${{ inputs.test_config_override_path }} chainlink_version: ${{ inputs.chainlink_version }} diff --git a/.github/workflows/on-demand-vrfv2-performance-test.yml b/.github/workflows/on-demand-vrfv2-performance-test.yml index d5eefcc348f..1f1a847d82c 100644 --- a/.github/workflows/on-demand-vrfv2-performance-test.yml +++ b/.github/workflows/on-demand-vrfv2-performance-test.yml @@ -67,7 +67,7 @@ jobs: run-e2e-tests-workflow: name: Run E2E Tests needs: set-tests-to-run - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: custom_test_list_json: ${{ needs.set-tests-to-run.outputs.test_list }} chainlink_version: ${{ inputs.chainlink_version }} diff --git a/.github/workflows/on-demand-vrfv2-smoke-tests.yml b/.github/workflows/on-demand-vrfv2-smoke-tests.yml index ea42a9014da..9f77c7ab53f 100644 --- a/.github/workflows/on-demand-vrfv2-smoke-tests.yml +++ b/.github/workflows/on-demand-vrfv2-smoke-tests.yml @@ -70,7 +70,7 @@ jobs: run-e2e-tests-workflow: name: Run E2E Tests needs: set-tests-to-run - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: custom_test_list_json: ${{ needs.set-tests-to-run.outputs.test_list }} chainlink_version: ${{ inputs.chainlink_version }} diff --git a/.github/workflows/on-demand-vrfv2plus-performance-test.yml b/.github/workflows/on-demand-vrfv2plus-performance-test.yml index f026086fff7..ae42c329459 100644 --- a/.github/workflows/on-demand-vrfv2plus-performance-test.yml +++ b/.github/workflows/on-demand-vrfv2plus-performance-test.yml @@ -67,7 +67,7 @@ jobs: run-e2e-tests-workflow: name: Run E2E Tests needs: set-tests-to-run - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: custom_test_list_json: ${{ needs.set-tests-to-run.outputs.test_list }} chainlink_version: ${{ inputs.chainlink_version }} diff --git a/.github/workflows/on-demand-vrfv2plus-smoke-tests.yml b/.github/workflows/on-demand-vrfv2plus-smoke-tests.yml index e1821336c63..9cd863eb7e3 100644 --- a/.github/workflows/on-demand-vrfv2plus-smoke-tests.yml +++ b/.github/workflows/on-demand-vrfv2plus-smoke-tests.yml @@ -70,7 +70,7 @@ jobs: run-e2e-tests-workflow: name: Run E2E Tests needs: set-tests-to-run - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: custom_test_list_json: ${{ needs.set-tests-to-run.outputs.test_list }} chainlink_version: ${{ inputs.chainlink_version }} diff --git a/.github/workflows/run-e2e-tests-reusable-workflow.yml b/.github/workflows/run-e2e-tests-reusable-workflow.yml deleted file mode 100644 index 8f0fd54e37f..00000000000 --- a/.github/workflows/run-e2e-tests-reusable-workflow.yml +++ /dev/null @@ -1,998 +0,0 @@ -# This is a reusable workflow that runs E2E tests for Chainlink. -# It is not meant to be run on its own. -# -# IMPORTANT NOTE: All workflow_call inputs appear as plain text in GitHub Logs (see https://github.com/actions/runner/issues/2988). -# Do not include any sensitive information in these inputs. Instead, for handling test secrets, refer to https://github.com/smartcontractkit/chainlink-testing-framework/blob/main/config/README.md#test-secrets -# -name: Run E2E Tests -on: - workflow_call: - inputs: - workflow_name: - description: 'Custom name for the workflow run' - required: false - type: string - default: 'Run E2E Tests' - chainlink_version: - description: 'Enter Chainlink version to use for the tests. Example: v2.10.0, develop or commit sha' - required: false - type: string - chainlink_upgrade_version: - description: 'Enter Chainlink version to use for the upgrade tests. Example: v2.10.0, develop or commit sha' - required: false - type: string - test_ids: - description: 'Run tests by test ids separated by commas. Example: "run_all_in_ocr_tests_go,run_TestOCRv2Request_in_ocr2_test_go". Check all test IDs in .github/e2e-tests.yml' - required: false - type: string - test_list: - description: 'Base64 encoded list of tests (YML objects) to run. Example in run-automation-ondemand-e2e-tests.yml' - required: false - type: string - custom_test_list_json: - description: 'Custom JSON list of tests to run' - required: false - type: string - # Example: - # custom_test_list_json: > - # { - # "tests": [ - # { - # "id": "TestVRFv2Plus", - # "path": "integration-tests/smoke/vrfv2plus_test.go", - # "runs_on": "ubuntu-latest", - # "test_env_type": "docker", - # "test_cmd": "cd integration-tests/smoke && go test vrfv2plus_test.go -test.parallel=1 -timeout 3h -count=1 -json -v" - # } - # ] - # } - test_workflow: - description: 'Run tests by workflow name. Example: "Run Nightly E2E Tests"' - required: false - type: string - test_config_override_path: - description: 'Path to a test config file used to override the default test config' - required: false - type: string - enable_check_test_configurations: - description: 'Set to "true" to enable check-test-configurations job' - required: false - type: boolean - default: false - with_existing_remote_runner_version: - description: 'Use the existing remote runner version for k8s tests. Example: "d3bf5044af33e08be788a2df31c4a745cf69d787"' - required: false - type: string - test_image_suites: - description: 'Suites to build in the test image. Space separated' - required: false - type: string - default: chaos migration reorg smoke soak benchmark load ccip-load - require_chainlink_image_versions_in_qa_ecr: - description: 'Check Chainlink image versions to be present in QA ECR. If not, build and push the image to QA ECR. Takes comma separated list of Chainlink image versions. Example: "5733cdcda9a9fc6da6343798b119b2ae136146cd,0b7d2c497a508efa5a827714780d908b7b8eda19"' - required: false - type: string - require_chainlink_plugin_versions_in_qa_ecr: - description: 'Check Chainlink plugins versions to be present in QA ECR. If not, build and push the image to QA ECR. Takes comma separated list of Chainlink image versions. Example: "5733cdcda9a9fc6da6343798b119b2ae136146cd,0b7d2c497a508efa5a827714780d908b7b8eda19"' - required: false - type: string - slack_notification_after_tests: - description: 'Set to "always" to always send a slack notification after the tests. Set "on_failure" to send a notification only on test failure' - required: false - type: string - slack_notification_after_tests_channel_id: - description: 'Slack channel ID to send the notification to' - required: false - type: string - slack_notification_after_tests_name: - description: 'Name of the slack notification' - required: false - type: string - slack_notification_after_tests_notify_user_id_on_failure: - description: 'Set Slack user id to notify on test failure' - required: false - type: string - test_log_upload_on_failure: - description: 'Set to "true" to upload the test log on failure as Github artifact' - required: false - type: boolean - default: false - test_log_upload_retention_days: - description: 'Number of days to retain the test log. Default is 3 days' - required: false - type: number - default: 5 - test_log_level: - description: 'Set the log level for the tests. Default is "debug"' - required: false - type: string - default: debug - upload_cl_node_coverage_artifact: - description: 'Set to "true" to upload Chainlink node coverage artifact to as Github artifact' - required: false - type: boolean - default: false - upload_cl_node_coverage_artifact_prefix: - description: 'Prefix for the Chainlink node coverage artifact' - required: false - type: string - enable_otel_traces_for_ocr2_plugins: - description: 'Set to "true" to enable OpenTelemetry traces for OCR2 plugins tests' - required: false - type: boolean - default: false - SLACK_CHANNEL: - description: 'SLACK_CHANNEL env used to send Slack notifications from test code' - required: false - type: string - SLACK_USER: - description: 'SLACK_USER env used to send Slack notifications from test code' - required: false - type: string - outputs: - test_results: - description: 'Test results from all executed tests' - value: ${{ jobs.after_tests.outputs.test_results }} - secrets: - TEST_SECRETS_OVERRIDE_BASE64: - required: false - QA_AWS_REGION: - required: true - QA_AWS_ROLE_TO_ASSUME: - required: true - QA_AWS_ACCOUNT_NUMBER: - required: true - QA_PYROSCOPE_INSTANCE: - required: true - QA_PYROSCOPE_KEY: - required: true - QA_KUBECONFIG: - required: true - LOKI_TENANT_ID: - required: true - LOKI_URL: - required: true - LOKI_BASIC_AUTH: - required: true - GRAFANA_INTERNAL_TENANT_ID: - required: true - GRAFANA_INTERNAL_BASIC_AUTH: - required: true - GRAFANA_INTERNAL_HOST: - required: true - GRAFANA_INTERNAL_URL_SHORTENER_TOKEN: - required: true - GH_TOKEN: - required: true - AWS_REGION: - required: true - AWS_OIDC_IAM_ROLE_VALIDATION_PROD_ARN: - required: true - AWS_API_GW_HOST_GRAFANA: - required: true - SLACK_BOT_TOKEN: - required: false - # Use instead of slack_notification_after_tests_channel_id if channel id is secret - SLACK_NOTIFICATION_AFTER_TESTS_CHANNEL_ID: - required: false - # Used in some tests to send slack notifications - SLACK_API_KEY: - required: false - # Used in some tests to send slack notifications - SLACK_CHANNEL: - required: false - -env: - CHAINLINK_IMAGE: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }}.dkr.ecr.${{ secrets.QA_AWS_REGION }}.amazonaws.com/chainlink - QA_CHAINLINK_IMAGE: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }}.dkr.ecr.${{ secrets.QA_AWS_REGION }}.amazonaws.com/chainlink - DEFAULT_CHAINLINK_VERSION: ${{ inputs.chainlink_version }} - DEFAULT_CHAINLINK_PLUGINS_VERSION: ${{ inputs.chainlink_version != '' && format('{0}-plugins', inputs.chainlink_version) }} - DEFAULT_CHAINLINK_UPGRADE_VERSION: ${{ inputs.chainlink_version }} - CHAINLINK_ENV_USER: ${{ github.actor }} - CHAINLINK_COMMIT_SHA: ${{ inputs.evm-ref }} - MOD_CACHE_VERSION: 1 - TEST_LOG_LEVEL: ${{ inputs.test_log_level }} - METRICS_COLLECTION_ID: chainlink-e2e-tests - SLACK_API_KEY: ${{ secrets.SLACK_API_KEY }} - SLACK_CHANNEL: ${{ inputs.slack_notification_after_tests_channel_id || inputs.SLACK_CHANNEL || secrets.SLACK_CHANNEL }} - SLACK_USER: ${{ inputs.SLACK_USER }} - -jobs: - validate-inputs: - name: Validate workflow inputs - runs-on: ubuntu-latest - outputs: - require_chainlink_image_versions_in_qa_ecr_matrix: ${{ steps.set-required-chainlink-image-versions-matrix.outputs.versions }} - require_chainlink_plugin_versions_in_qa_ecr_matrix: ${{ steps.set-required-chainlink-plugin-versions-matrix.outputs.versions }} - steps: - - name: Check input conditions - run: | - if [[ "${{ inputs.test_ids }}" != "" && "${{ inputs.test_workflow }}" != "" ]]; then - echo "::error::Error: Both 'test_ids' and 'test_workflow' are provided. Please specify only one." - exit 1 - fi - if [[ "${{ secrets.TEST_SECRETS_OVERRIDE_BASE64 }}" != "" ]]; then - echo "Will run tests with custom test secrets" - fi - - name: Install jq - run: sudo apt-get install jq - - - name: Create matrix for required Chainlink image versions - id: set-required-chainlink-image-versions-matrix - run: | - image_versions="${{ inputs.require_chainlink_image_versions_in_qa_ecr }}" - default_version="${{ env.DEFAULT_CHAINLINK_VERSION }}" - current_sha="${{ github.sha }}" - - if [[ "$default_version" == "$current_sha" ]]; then - # Append the current SHA to required image versions - if [[ -z "$image_versions" ]]; then - image_versions="${{ github.sha }}" - else - image_versions+=",${{ github.sha }}" - fi - fi - - # Convert the comma-separated string to a JSON array - image_versions=$(echo "$image_versions" | jq -Rc 'if . == "" then "" else split(",") | if . == [""] then "" else . end end') - - echo "Required Chainlink image versions: $image_versions" - echo "versions=$image_versions" >> $GITHUB_OUTPUT - - - name: Create matrix for required Chainlink plugin versions - id: set-required-chainlink-plugin-versions-matrix - run: | - image_versions=$(echo "${{ inputs.require_chainlink_plugin_versions_in_qa_ecr }}" | jq -Rc 'if . == "" then "" else split(",") | if . == [""] then "" else . end end') - echo "Required Chainlink plugin image versions: $image_versions" - echo "versions=$image_versions" >> $GITHUB_OUTPUT - - check-test-configurations: - name: Check test configurations - if: ${{ inputs.enable_check_test_configurations }} - needs: validate-inputs - runs-on: ubuntu-latest - steps: - - name: Checkout core - uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633 # v4.1.2 - with: - fetch-depth: 0 - path: core - - name: Install citool - shell: bash - run: go install github.com/smartcontractkit/chainlink-testing-framework/tools/citool@1d8d5b55bf6379b969dbcde99abc87faa5963ea1 # v1.34.4 - - name: Run Check Tests Command - run: | - if ! citool check-tests ${{ github.workspace }}/integration-tests ${{ github.workspace }}/.github/e2e-tests.yml; then - echo "::error::Some E2E test configurations have to be added to .github/e2e-tests.yml. This file defines Github CI configuration for each E2E test or set of E2E tests." && exit 1 - fi - - get_latest_chainlink_release_version: - name: Get latest Chainlink release version - runs-on: ubuntu-latest - environment: integration - outputs: - latest_chainlink_release_version: ${{ steps.get_latest_version.outputs.latest_version }} - steps: - - name: Get Latest Version - id: get_latest_version - run: | - untrimmed_ver=$(curl --header "Authorization: token ${{ secrets.GH_TOKEN }}" --request GET https://api.github.com/repos/${{ github.repository }}/releases/latest | jq -r .name) - latest_version="${untrimmed_ver:1}" - echo "Latest Chainlink release version: $latest_version" - echo "latest_version=${latest_version}" >> "$GITHUB_OUTPUT" - # Check if latest_version is empty - if [ -z "$latest_version" ]; then - echo "Error: The latest_version is empty. The migration tests need a verison to run." - exit 1 - fi - - load-test-configurations: - name: Load test configurations - needs: [validate-inputs] - runs-on: ubuntu-latest - outputs: - run-docker-tests: ${{ steps.check-matrices.outputs.run-docker-tests }} - run-k8s-tests: ${{ steps.check-matrices.outputs.run-k8s-tests }} - docker-matrix: ${{ steps.set-docker-matrix.outputs.matrix }} - k8s-runner-matrix: ${{ steps.set-k8s-runner-matrix.outputs.matrix }} - workflow_id: ${{ steps.gen_id.outputs.workflow_id }} - steps: - - name: Checkout code - uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633 # v4.1.2 - - name: Setup Go - uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5.0.0 - with: - go-version: '1.22.6' - check-latest: true - - name: Install citool - shell: bash - run: go install github.com/smartcontractkit/chainlink-testing-framework/tools/citool@1d8d5b55bf6379b969dbcde99abc87faa5963ea1 # v1.34.4 - - name: Install jq - run: sudo apt-get install jq - - - name: Generate Docker Tests Matrix - id: set-docker-matrix - run: | - # Check if custom_test_list_json is provided and non-empty - if [[ -n '${{ inputs.custom_test_list_json }}' ]]; then - echo "Using custom test list JSON" - MATRIX_JSON=$(echo '${{ inputs.custom_test_list_json }}' | jq -c '{tests: [.tests[] | select(.test_env_type == "docker")]}') - else - echo "Using default test list" - MATRIX_JSON=$(citool filter --file ${{ github.workspace }}/.github/e2e-tests.yml --test-env-type 'docker' --test-list '${{ inputs.test_list }}' --test-ids '${{ inputs.test_ids }}' --workflow '${{ inputs.test_workflow }}') - fi - - echo "Docker tests:" - echo "$MATRIX_JSON" | jq - echo "matrix=$MATRIX_JSON" >> $GITHUB_OUTPUT - - - name: Generate K8s Tests Matrix - id: set-k8s-runner-matrix - run: | - # Check if custom_test_list_json is provided and non-empty - if [[ -n '${{ inputs.custom_test_list_json }}' ]]; then - echo "Using custom test list JSON" - MATRIX_JSON=$(echo '${{ inputs.custom_test_list_json }}' | jq -c '{tests: [.tests[] | select(.test_env_type == "k8s-remote-runner")]}') - else - echo "Using default test list" - MATRIX_JSON=$(citool filter --file ${{ github.workspace }}/.github/e2e-tests.yml --test-env-type 'k8s-remote-runner' --test-list '${{ inputs.test_list }}' --test-ids '${{ inputs.test_ids }}' --workflow '${{ inputs.test_workflow }}') - fi - - echo "K8s tests:" - echo "$MATRIX_JSON" | jq - echo "matrix=$MATRIX_JSON" >> $GITHUB_OUTPUT - - - name: Check Test Matrices - id: check-matrices - run: | - DOCKER_MATRIX_EMPTY=$(echo '${{ steps.set-docker-matrix.outputs.matrix }}' | jq '.tests == null or .tests == []') - K8S_MATRIX_EMPTY=$(echo '${{ steps.set-k8s-runner-matrix.outputs.matrix }}' | jq '.tests == null or .tests == []') - - # Check if jq commands succeeded - if [ $? -ne 0 ]; then - echo "JSON parse error occurred." - exit 1 - fi - - if [[ "$DOCKER_MATRIX_EMPTY" == "true" ]]; then - echo "run-docker-tests=false" >> $GITHUB_OUTPUT - else - echo "run-docker-tests=true" >> $GITHUB_OUTPUT - fi - if [[ "$K8S_MATRIX_EMPTY" == "true" ]]; then - echo "run-k8s-tests=false" >> $GITHUB_OUTPUT - else - echo "run-k8s-tests=true" >> $GITHUB_OUTPUT - fi - - # Check if both matrices are empty - if [[ "$DOCKER_MATRIX_EMPTY" == "true" ]] && [[ "$K8S_MATRIX_EMPTY" == "true" ]]; then - echo "No tests found for inputs. Both Docker and Kubernetes tests matrices are empty" - exit 1 - fi - shell: bash - - - name: Check if test secrets are required for any test - shell: bash - run: | - # Check if the test secret key is provided and skip the checks if it is non-empty - if [ -n "${{ secrets.TEST_SECRETS_OVERRIDE_BASE64 }}" ]; then - echo "Test secret key provided. Skipping checks for tests requiring secrets." - exit 0 - fi - - # Parse the JSON to check for test_secrets_required in Docker matrix - DOCKER_TESTS_REQUIRING_SECRETS=$(echo '${{ steps.set-docker-matrix.outputs.matrix }}' | jq 'if .tests then .tests[] | select(has("test_secrets_required") and .test_secrets_required) | .id else empty end' -r) - # Parse the JSON to check for test_secrets_required in Kubernetes matrix - K8S_TESTS_REQUIRING_SECRETS=$(echo '${{ steps.set-k8s-runner-matrix.outputs.matrix }}' | jq 'if .tests then .tests[] | select(has("test_secrets_required") and .test_secrets_required) | .id else empty end' -r) - - # Determine if any tests require secrets - if [ ! -z "$DOCKER_TESTS_REQUIRING_SECRETS" ] || [ ! -z "$K8S_TESTS_REQUIRING_SECRETS" ]; then - echo "Tests in .github/e2e-tests.yml requiring custom test secrets:" - if [ ! -z "$DOCKER_TESTS_REQUIRING_SECRETS" ]; then - echo $DOCKER_TESTS_REQUIRING_SECRETS - fi - if [ ! -z "$K8S_TESTS_REQUIRING_SECRETS" ]; then - echo $K8S_TESTS_REQUIRING_SECRETS - fi - echo "::error::Error: Some of the tests require custom test secrets to run. Please see workflow logs and set 'test_secrets_override_key' to run these tests." - exit 1 - else - echo "No tests require secrets. Proceeding without additional secret setup." - fi - - - name: Generate random workflow id - id: gen_id - run: echo "workflow_id=$(uuidgen)" >> $GITHUB_OUTPUT - - - # Check if Chainlink images required for the tests exist. If not, build and push the images to QA ECR - require-chainlink-image-versions-in-qa-ecr: - name: Get Chainlink image - needs: [validate-inputs, load-test-configurations] - if: ${{ fromJson(needs.validate-inputs.outputs.require_chainlink_image_versions_in_qa_ecr_matrix) != '' }} - runs-on: ubuntu-latest - environment: integration - permissions: - id-token: write - contents: read - env: - CHAINLINK_IMAGE: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }}.dkr.ecr.${{ secrets.QA_AWS_REGION }}.amazonaws.com/chainlink - strategy: - matrix: - version: ${{ fromJson(needs.validate-inputs.outputs.require_chainlink_image_versions_in_qa_ecr_matrix) }} - steps: - - name: Checkout the repo - uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633 # v4.1.2 - - - name: Get Chainlink image - uses: ./.github/actions/build-chainlink-image - with: - dockerfile: core/chainlink.Dockerfile - git_commit_sha: ${{ matrix.version }} - tag_suffix: '' - check_image_exists: 'true' - AWS_REGION: ${{ secrets.QA_AWS_REGION }} - AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }} - - # Check if Chainlink plugins required for the tests exist. If not, build and push the images to QA ECR - require-chainlink-plugin-versions-in-qa-ecr: - name: Get Chainlink plugins image - needs: [validate-inputs, load-test-configurations] - if: ${{ fromJson(needs.validate-inputs.outputs.require_chainlink_plugin_versions_in_qa_ecr_matrix) != '' }} - runs-on: ubuntu-latest - environment: integration - permissions: - id-token: write - contents: read - env: - CHAINLINK_IMAGE: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }}.dkr.ecr.${{ secrets.QA_AWS_REGION }}.amazonaws.com/chainlink - strategy: - matrix: - version: ${{ fromJson(needs.validate-inputs.outputs.require_chainlink_plugin_versions_in_qa_ecr_matrix) }} - steps: - - name: Checkout the repo - uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633 # v4.1.2 - - - name: Get Chainlink plugins image - uses: ./.github/actions/build-chainlink-image - with: - dockerfile: plugins/chainlink.Dockerfile - git_commit_sha: ${{ matrix.version }} - tag_suffix: '-plugins' - check_image_exists: 'true' - AWS_REGION: ${{ secrets.QA_AWS_REGION }} - AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }} - - # Run Docker tests - run-docker-tests: - name: ${{ matrix.tests.id }} - needs: [load-test-configurations, require-chainlink-image-versions-in-qa-ecr, require-chainlink-plugin-versions-in-qa-ecr, get_latest_chainlink_release_version] - # Run when none of the needed jobs fail or are cancelled (skipped or successful jobs are ok) - if: ${{ needs.load-test-configurations.outputs.run-docker-tests == 'true' && always() && !failure() && !cancelled() }} - runs-on: ${{ matrix.tests.runs_on }} - strategy: - fail-fast: false - matrix: ${{fromJson(needs.load-test-configurations.outputs.docker-matrix)}} - environment: integration - permissions: - actions: read - checks: write - pull-requests: write - id-token: write - contents: read - env: - LATEST_CHAINLINK_RELEASE_VERSION: ${{ needs.get_latest_chainlink_release_version.outputs.latest_chainlink_release_version }} - TEST_CONFIG_OVERRIDE_PATH: ${{ matrix.tests.test_config_override_path || inputs.test_config_override_path }} - TEST_ID: ${{ matrix.tests.id_sanitized || matrix.tests.id }} - steps: - - name: Collect Metrics - if: always() - id: collect-gha-metrics - uses: smartcontractkit/push-gha-metrics-action@d9da21a2747016b3e13de58c7d4115a3d5c97935 # v3.0.1 - with: - id: e2e_tests_${{ env.TEST_ID }} - org-id: ${{ secrets.GRAFANA_INTERNAL_TENANT_ID }} - basic-auth: ${{ secrets.GRAFANA_INTERNAL_BASIC_AUTH }} - hostname: ${{ secrets.GRAFANA_INTERNAL_HOST }} - this-job-name: ${{ inputs.workflow_name }} / ${{ matrix.tests.id }} - test-results-file: '{"testType":"go","filePath":"/tmp/gotest.log"}' - continue-on-error: true - - - name: Checkout repository - uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633 # v4.1.2 - - name: Install jq - run: sudo apt-get install -y jq - - - name: Show test config override path in summary - if: ${{ env.TEST_CONFIG_OVERRIDE_PATH }} - run: | - echo "### Test config override path" >> $GITHUB_STEP_SUMMARY - echo "[${{ env.TEST_CONFIG_OVERRIDE_PATH }}]($GITHUB_SERVER_URL/$GITHUB_REPOSITORY/blob/${{ github.sha }}/${{ env.TEST_CONFIG_OVERRIDE_PATH }})" >> $GITHUB_STEP_SUMMARY - - - name: Show chainlink version in summary - if: ${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_VERSION || env.DEFAULT_CHAINLINK_VERSION }} - run: | - echo "### Chainlink version" >> $GITHUB_STEP_SUMMARY - echo "${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_VERSION || env.DEFAULT_CHAINLINK_VERSION }}" >> $GITHUB_STEP_SUMMARY - - - name: Show test configuration in logs - run: echo '${{ toJson(matrix.tests) }}' | jq . - - - name: Setup GAP for Grafana - uses: smartcontractkit/.github/actions/setup-gap@d316f66b2990ea4daa479daa3de6fc92b00f863e # setup-gap@0.3.2 - id: setup-gap - timeout-minutes: 3 - with: - aws-region: ${{ secrets.AWS_REGION }} - aws-role-arn: ${{ secrets.AWS_OIDC_IAM_ROLE_VALIDATION_PROD_ARN }} - api-gateway-host: ${{ secrets.AWS_API_GW_HOST_GRAFANA }} - duplicate-authorization-header: "true" - - - name: Setup Grafana and OpenTelemetry - id: docker-setup - if: inputs.enable_otel_traces_for_ocr2_plugins && matrix.tests.test_env_vars.ENABLE_OTEL_TRACES == 'true' - run: | - # Create network - docker network create --driver bridge tracing - - # Make trace directory - cd integration-tests/smoke/ - mkdir ./traces - chmod -R 777 ./traces - - # Switch directory - cd ../../.github/tracing - - # Create a Docker volume for traces - # docker volume create otel-traces - - # Start OpenTelemetry Collector - # Note the user must be set to the same user as the runner for the trace data to be accessible - docker run -d --network=tracing --name=otel-collector \ - -v $PWD/otel-collector-ci.yaml:/etc/otel-collector.yaml \ - -v $PWD/../../integration-tests/smoke/traces:/tracing \ - --user "$(id -u):$(id -g)" \ - -p 4317:4317 otel/opentelemetry-collector:0.88.0 --config=/etc/otel-collector.yaml - - - name: Run tests - id: run_tests - uses: smartcontractkit/.github/actions/ctf-run-tests@b8731364b119e88983e94b0c4da87fc27ddb41b8 # ctf-run-tests@0.0.0 - env: - DETACH_RUNNER: true - E2E_TEST_CHAINLINK_VERSION: ${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_VERSION || env.DEFAULT_CHAINLINK_VERSION }} - E2E_TEST_CHAINLINK_UPGRADE_VERSION: ${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_UPGRADE_VERSION }} - E2E_TEST_CHAINLINK_POSTGRES_VERSION: ${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_POSTGRES_VERSION }} - E2E_TEST_SELECTED_NETWORK: ${{ matrix.tests.test_env_vars.E2E_TEST_SELECTED_NETWORK }} - E2E_TEST_LOGGING_RUN_ID: ${{ github.run_id }} - E2E_TEST_LOG_STREAM_LOG_TARGETS: ${{ vars.LOGSTREAM_LOG_TARGETS }} - E2E_TEST_LOG_COLLECT: ${{ vars.TEST_LOG_COLLECT }} - E2E_TEST_LOKI_TENANT_ID: ${{ secrets.LOKI_TENANT_ID }} - E2E_TEST_LOKI_ENDPOINT: ${{ secrets.LOKI_URL }} - E2E_TEST_LOKI_BASIC_AUTH: ${{ secrets.LOKI_BASIC_AUTH }} - E2E_TEST_GRAFANA_DASHBOARD_URL: ${{ matrix.tests.test_env_vars.E2E_TEST_GRAFANA_DASHBOARD_URL }} - E2E_TEST_GRAFANA_BEARER_TOKEN: ${{ secrets.GRAFANA_INTERNAL_URL_SHORTENER_TOKEN }} - E2E_TEST_PYROSCOPE_ENVIRONMENT: ${{ matrix.tests.pyroscope_env }} - E2E_TEST_PYROSCOPE_SERVER_URL: ${{ matrix.tests.pyroscope_env != '' && secrets.QA_PYROSCOPE_INSTANCE || '' }} - E2E_TEST_PYROSCOPE_KEY: ${{ matrix.tests.pyroscope_env != '' && secrets.QA_PYROSCOPE_KEY || '' }} - with: - test_command_to_run: ${{ matrix.tests.test_cmd }} ${{ matrix.tests.test_cmd_opts || '2>&1 | tee /tmp/gotest.log | gotestloghelper -ci -singlepackage -hidepassingtests=false' }} - test_download_vendor_packages_command: cd $(dirname ${{ matrix.tests.path }}) && go mod download - test_secrets_override_base64: ${{ secrets.TEST_SECRETS_OVERRIDE_BASE64 }} - test_config_override_path: ${{ env.TEST_CONFIG_OVERRIDE_PATH }} - test_type: ${{ matrix.tests.test_env_vars.TEST_TYPE }} - test_suite: ${{ matrix.tests.test_env_vars.TEST_SUITE }} - default_e2e_test_chainlink_image: ${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_IMAGE || env.CHAINLINK_IMAGE }} - default_e2e_test_chainlink_upgrade_image: ${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_UPGRADE_IMAGE }} - aws_registries: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }} - artifacts_name: ${{ env.TEST_ID }}-test-logs - artifacts_location: | - ./integration-tests/smoke/logs/ - ./integration-tests/smoke/db_dumps/ - /tmp/gotest.log - publish_check_name: ${{ env.TEST_ID }} - token: ${{ secrets.GH_TOKEN }} - cache_key_id: e2e-tests - go_mod_path: ./integration-tests/go.mod - QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }} - QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }} - QA_KUBECONFIG: "" - should_tidy: "false" - go_coverage_src_dir: /var/tmp/go-coverage - go_coverage_dest_dir: ${{ github.workspace }}/.covdata - - - name: Show Otel-Collector logs - if: inputs.enable_otel_traces_for_ocr2_plugins && matrix.tests.test_env_vars.ENABLE_OTEL_TRACES == 'true' - run: | - docker logs otel-collector - - - name: Permissions on traces - if: inputs.enable_otel_traces_for_ocr2_plugins && matrix.tests.test_env_vars.ENABLE_OTEL_TRACES == 'true' - run: | - ls -l ./integration-tests/smoke/traces - - - name: Upload trace data as artifact - if: inputs.enable_otel_traces_for_ocr2_plugins && matrix.tests.test_env_vars.ENABLE_OTEL_TRACES == 'true' - uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3 # v4.3.1 - with: - name: trace-data - path: ./integration-tests/smoke/traces/trace-data.json - - - name: Upload test log as artifact - uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3 # v4.3.1 - if: inputs.test_log_upload_on_failure && failure() - with: - name: test_log_${{ env.TEST_ID }} - path: /tmp/gotest.log - retention-days: ${{ inputs.test_log_upload_retention_days }} - continue-on-error: true - - - name: Upload cl node coverage data as artifact - if: inputs.upload_cl_node_coverage_artifact - uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3 - timeout-minutes: 2 - continue-on-error: true - with: - name: ${{ inputs.upload_cl_node_coverage_artifact_prefix }}${{ env.TEST_ID }} - path: .covdata - retention-days: 1 - - - name: Record test result - if: ${{ always() }} - run: | - id="${{ matrix.tests.id }}" - result="${{ steps.run_tests.outcome }}" - echo "{\"id\": \"$id\", \"result\": \"$result\"}" > test_result.json - - - name: Upload test result as artifact - uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3 - with: - name: test_result_${{ needs.load-test-configurations.outputs.workflow_id }}_${{ env.TEST_ID }} - path: test_result.json - retention-days: 1 - - - name: Upload custom test artifacts - if: failure() && matrix.tests.test_artifacts_on_failure != '' - uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3 - with: - name: custom_test_artifacts_${{ env.TEST_ID }}_${{ needs.load-test-configurations.outputs.workflow_id }} - path: ${{ matrix.tests.test_artifacts_on_failure }} - retention-days: 1 - - - name: Show Grafana url in test summary - if: always() - uses: smartcontractkit/.github/actions/ctf-show-grafana-in-test-summary@b6e37806737eef87e8c9137ceeb23ef0bff8b1db # ctf-show-grafana-in-test-summary@0.1.0 - - # Run K8s tests using old remote runner - - get-remote-runner-test-image: - needs: [load-test-configurations] - if: ${{ needs.load-test-configurations.outputs.run-k8s-tests == 'true' && always() && !failure() && !cancelled() }} - name: Get remote runner test image - runs-on: ubuntu-latest - environment: integration - permissions: - actions: read - checks: write - pull-requests: write - id-token: write - contents: read - outputs: - remote-runner-version: ${{ steps.set-remote-runner-version.outputs.remote-runner-version }} - env: - ENV_JOB_IMAGE_BASE: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }}.dkr.ecr.${{ secrets.QA_AWS_REGION }}.amazonaws.com/chainlink-tests - steps: - - name: Checkout repository - uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633 # v4.1.2 - - name: Build Test Runner Image - id: build-test-runner-image - uses: smartcontractkit/.github/actions/ctf-build-test-image@a5e4f4c8fbb8e15ab2ad131552eca6ac83c4f4b3 # ctf-build-test-image@0.1.0 - if: ${{ inputs.with_existing_remote_runner_version == '' }} - with: - QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }} - QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }} - QA_AWS_ACCOUNT_NUMBER: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }} - suites: ${{ inputs.test_image_suites }} - - name: Set Remote Runner Version - id: set-remote-runner-version - run: | - if [[ -z "${{ inputs.with_existing_remote_runner_version }}" ]]; then - echo "remote-runner-image=${{ steps.build-test-runner-image.outputs.test_image }}" >> $GITHUB_OUTPUT - echo "remote-runner-repository=${{ steps.build-test-runner-image.outputs.test_image_repository }}" >> $GITHUB_OUTPUT - echo "remote-runner-version=${{ steps.build-test-runner-image.outputs.test_image_tag }}" >> $GITHUB_OUTPUT - else - echo "remote-runner-version=${{ inputs.with_existing_remote_runner_version }}" >> $GITHUB_OUTPUT - fi - - run-k8s-runner-tests: - needs: [load-test-configurations, get-remote-runner-test-image, require-chainlink-image-versions-in-qa-ecr, require-chainlink-plugin-versions-in-qa-ecr, get_latest_chainlink_release_version] - if: ${{ needs.load-test-configurations.outputs.run-k8s-tests == 'true' && always() && !failure() && !cancelled() }} - name: ${{ matrix.tests.id }} - runs-on: ${{ matrix.tests.runs_on }} - strategy: - fail-fast: false - matrix: ${{fromJson(needs.load-test-configurations.outputs.k8s-runner-matrix)}} - environment: integration - permissions: - actions: read - checks: write - pull-requests: write - id-token: write - contents: read - env: - LATEST_CHAINLINK_RELEASE_VERSION: ${{ needs.get_latest_chainlink_release_version.outputs.latest_chainlink_release_version }} - TEST_CONFIG_OVERRIDE_PATH: ${{ matrix.tests.test_config_override_path || inputs.test_config_override_path }} - TEST_ID: ${{ matrix.tests.id_sanitized || matrix.tests.id }} - steps: - - name: Collect Metrics - if: always() - id: collect-gha-metrics - uses: smartcontractkit/push-gha-metrics-action@d9da21a2747016b3e13de58c7d4115a3d5c97935 # v3.0.1 - with: - id: e2e_tests_${{ env.TEST_ID }} - org-id: ${{ secrets.GRAFANA_INTERNAL_TENANT_ID }} - basic-auth: ${{ secrets.GRAFANA_INTERNAL_BASIC_AUTH }} - hostname: ${{ secrets.GRAFANA_INTERNAL_HOST }} - this-job-name: ${{ inputs.workflow_name }} / ${{ matrix.tests.id }} - continue-on-error: true - - - name: Checkout repository - uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633 # v4.1.2 - - name: Install jq - run: sudo apt-get install -y jq - - - name: Show test config override path in summary - if: ${{ env.TEST_CONFIG_OVERRIDE_PATH }} - run: | - echo "### Test config override path" >> $GITHUB_STEP_SUMMARY - echo "[${{ env.TEST_CONFIG_OVERRIDE_PATH }}]($GITHUB_SERVER_URL/$GITHUB_REPOSITORY/blob/${{ github.sha }}/${{ env.TEST_CONFIG_OVERRIDE_PATH }})" >> $GITHUB_STEP_SUMMARY - - - name: Show chainlink version in summary - if: ${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_VERSION || env.DEFAULT_CHAINLINK_VERSION }} - run: | - echo "### Chainlink version" >> $GITHUB_STEP_SUMMARY - echo "${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_VERSION || env.DEFAULT_CHAINLINK_VERSION }}" >> $GITHUB_STEP_SUMMARY - - - name: Show remote runner version in summary - run: | - echo "Remote Runner Version: ${{ needs.get-remote-runner-test-image.outputs.remote-runner-version }}" - echo "### Remote Runner Version" >> $GITHUB_STEP_SUMMARY - echo "${{ needs.get-remote-runner-test-image.outputs.remote-runner-version }}" >> $GITHUB_STEP_SUMMARY - - - name: Show test configuration in logs - run: echo '${{ toJson(matrix.tests) }}' | jq . - - - name: Run tests - id: run_tests - uses: smartcontractkit/.github/actions/ctf-run-tests@b6e37806737eef87e8c9137ceeb23ef0bff8b1db # ctf-run-tests@0.1.0 - env: - DETACH_RUNNER: true - RR_MEM: ${{ matrix.tests.remote_runner_memory }} - TEST_ARGS: -test.timeout 900h -test.memprofile memprofile.out -test.cpuprofile profile.out - ENV_JOB_IMAGE: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }}.dkr.ecr.${{ secrets.QA_AWS_REGION }}.amazonaws.com/chainlink-tests:${{ needs.get-remote-runner-test-image.outputs.remote-runner-version }} - INTERNAL_DOCKER_REPO: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }}.dkr.ecr.${{ secrets.QA_AWS_REGION }}.amazonaws.com - # We can comment these out when we have a stable soak test and aren't worried about resource consumption - TEST_UPLOAD_CPU_PROFILE: ${{ matrix.tests.test_env_vars.TEST_UPLOAD_CPU_PROFILE }} - TEST_UPLOAD_MEM_PROFILE: ${{ matrix.tests.test_env_vars.TEST_UPLOAD_MEM_PROFILE }} - REF_NAME: ${{ github.head_ref || github.ref_name }} - E2E_TEST_CHAINLINK_VERSION: ${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_VERSION || env.DEFAULT_CHAINLINK_VERSION }} - E2E_TEST_CHAINLINK_UPGRADE_VERSION: ${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_UPGRADE_VERSION }} - E2E_TEST_CHAINLINK_POSTGRES_VERSION: ${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_POSTGRES_VERSION }} - E2E_TEST_SELECTED_NETWORK: ${{ matrix.tests.test_env_vars.E2E_TEST_SELECTED_NETWORK }} - E2E_TEST_LOGGING_RUN_ID: ${{ github.run_id }} - E2E_TEST_LOG_STREAM_LOG_TARGETS: ${{ vars.LOGSTREAM_LOG_TARGETS }} - E2E_TEST_LOG_COLLECT: ${{ vars.TEST_LOG_COLLECT }} - E2E_TEST_LOKI_TENANT_ID: ${{ secrets.LOKI_TENANT_ID }} - E2E_TEST_LOKI_ENDPOINT: ${{ secrets.LOKI_URL }} - E2E_TEST_LOKI_BASIC_AUTH: ${{ secrets.LOKI_BASIC_AUTH }} - E2E_TEST_GRAFANA_DASHBOARD_URL: ${{ matrix.tests.test_env_vars.E2E_TEST_GRAFANA_DASHBOARD_URL }} - E2E_TEST_GRAFANA_BEARER_TOKEN: ${{ secrets.GRAFANA_INTERNAL_URL_SHORTENER_TOKEN }} - E2E_TEST_PYROSCOPE_ENVIRONMENT: ${{ matrix.tests.pyroscope_env }} - E2E_TEST_PYROSCOPE_SERVER_URL: ${{ matrix.tests.pyroscope_env != '' && secrets.QA_PYROSCOPE_INSTANCE || '' }} - E2E_TEST_PYROSCOPE_KEY: ${{ matrix.tests.pyroscope_env != '' && secrets.QA_PYROSCOPE_KEY || '' }} - DATABASE_URL: postgresql://postgres:node@localhost:5432/chainlink_test?sslmode=disable - with: - test_command_to_run: ${{ matrix.tests.test_cmd }} ${{ matrix.tests.test_cmd_opts || '2>&1 | tee /tmp/gotest.log | gotestloghelper -ci -singlepackage -hidepassingtests=false' }} - test_download_vendor_packages_command: make gomod - test_secrets_override_base64: ${{ secrets.TEST_SECRETS_OVERRIDE_BASE64 }} - test_config_override_path: ${{ env.TEST_CONFIG_OVERRIDE_PATH }} - test_type: ${{ matrix.tests.test_env_vars.TEST_TYPE }} - test_suite: ${{ matrix.tests.test_env_vars.TEST_SUITE }} - default_e2e_test_chainlink_image: ${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_IMAGE || env.CHAINLINK_IMAGE }} - default_e2e_test_chainlink_upgrade_image: ${{ matrix.tests.test_env_vars.E2E_TEST_CHAINLINK_UPGRADE_IMAGE }} - token: ${{ secrets.GH_TOKEN }} - should_cleanup: false - cache_key_id: e2e-tests - go_mod_path: ./integration-tests/go.mod - QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }} - QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }} - QA_KUBECONFIG: ${{ secrets.QA_KUBECONFIG }} - - - name: Upload test log as artifact - uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3 - if: inputs.test_log_upload_on_failure && failure() - with: - name: test_log_${{ env.TEST_ID }} - path: /tmp/gotest.log - retention-days: ${{ inputs.test_log_upload_retention_days }} - continue-on-error: true - - - name: Upload custom test artifacts - if: failure() && matrix.tests.test_artifacts_on_failure != '' - uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3 - with: - name: custom_test_artifacts_${{ env.TEST_ID }}_${{ needs.load-test-configurations.outputs.workflow_id }} - path: ${{ matrix.tests.test_artifacts_on_failure }} - retention-days: 1 - - - name: Show Grafana url in test summary - if: always() - uses: smartcontractkit/.github/actions/ctf-show-grafana-in-test-summary@b6e37806737eef87e8c9137ceeb23ef0bff8b1db # ctf-show-grafana-in-test-summary@0.1.0 - - after_tests: - needs: [load-test-configurations, run-docker-tests, run-k8s-runner-tests] - if: always() - name: After tests - runs-on: ubuntu-latest - # Set to access secrets like secrets.QA_SLACK_API_KEY that are set in the "integration" environment - environment: integration - outputs: - test_results: ${{ steps.set_test_results.outputs.results }} - steps: - - name: Download all test result artifacts - uses: actions/download-artifact@9c19ed7fe5d278cd354c7dfd5d3b88589c7e2395 # v4.1.6 - with: - path: test_results - pattern: test_result_${{ needs.load-test-configurations.outputs.workflow_id }}_* - - - name: Set detailed test results - id: set_test_results - run: | - if [ -d "test_results" ]; then - cd test_results - ls -R . - # Combine JSON files into one - find . -name '*.json' -exec cat {} + | jq -s '.' > test_results.json - # Display the combined JSON - jq . test_results.json - # Set the combined results as an output - echo "results=$(jq -c . test_results.json)" >> $GITHUB_OUTPUT - else - echo "No test results directory found." - echo "results=[]" >> $GITHUB_OUTPUT - fi - - - name: Set short SHA - id: set_short_sha - shell: bash - run: echo "short_sha=$(echo ${{ github.sha }} | cut -c1-7)" >> $GITHUB_OUTPUT - - - name: Send Slack notification - uses: slackapi/slack-github-action@6c661ce58804a1a20f6dc5fbee7f0381b469e001 # v1.25.0 - if: ${{ inputs.slack_notification_after_tests == 'true' || inputs.slack_notification_after_tests == 'always' || (inputs.slack_notification_after_tests == 'on_failure' && contains(join(needs.*.result, ','), 'failure')) }} - id: slack - env: - SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }} - with: - channel-id: ${{ inputs.slack_notification_after_tests_channel_id || secrets.SLACK_NOTIFICATION_AFTER_TESTS_CHANNEL_ID }} - payload: | - { - "attachments": [ - { - "color": "${{ contains(join(needs.*.result, ','), 'failure') && '#C62828' || contains(join(needs.*.result, ','), 'cancelled') && '#FFA000' || '2E7D32' }}", - "blocks": [ - { - "type": "section", - "text": { - "type": "mrkdwn", - "text": "${{ inputs.slack_notification_after_tests_name }} - ${{ contains(join(needs.*.result, ','), 'failure') && 'Failed :x:' || contains(join(needs.*.result, ','), 'cancelled') && 'Cancelled :warning:' || 'Passed :white_check_mark:' }}" - } - }, - { - "type": "section", - "text": { - "type": "mrkdwn", - "text": "${{ github.ref_name }} | <${{ github.server_url }}/${{ github.repository }}/commit/${{ github.sha }}|${{ steps.set_short_sha.outputs.short_sha }}> | <${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View Details>" - } - } - ] - } - ] - } - - - name: Notify user in Slack message if tests failed - if: ${{ inputs.slack_notification_after_tests != '' && contains(join(needs.*.result, ','), 'failure') && inputs.slack_notification_after_tests_notify_user_id_on_failure != '' }} - uses: slackapi/slack-github-action@6c661ce58804a1a20f6dc5fbee7f0381b469e001 # v1.25.0 - env: - SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }} - with: - channel-id: ${{ inputs.slack_notification_after_tests_channel_id || secrets.SLACK_NOTIFICATION_AFTER_TESTS_CHANNEL_ID }} - payload: | - { - "thread_ts": "${{ steps.slack.outputs.thread_ts }}", - "blocks": [ - { - "type": "section", - "text": { - "type": "mrkdwn", - "text": "Notifying <@${{ inputs.slack_notification_after_tests_notify_user_id_on_failure }}>, please check the test results." - } - } - ] - } - - # Run K8s tests using new remote runner - # remote-runner-k8s-tests: - # runs-on: ubuntu-latest - # container: - # image: golang:1.18 - # steps: - # - name: Checkout repository - # uses: actions/checkout@v2 - - # - name: Set up Go - # uses: actions/setup-go@v2 - # with: - # go-version: '1.18' - - # - name: Load Runner Config - # run: echo "$RUNNER_CONFIG" > runner.toml - # env: - # RUNNER_CONFIG: | - # # Runner configuration - # detached_mode = true - # debug = false - - # [[test_runs]] - # namespace = "dev-env" - # rbac_role_name = "dev-role" - # rbac_service_account_name = "dev-service-account" - # sync_value = "unique-sync-value-1" - # ttl_seconds_after_finished = 300 - # image_registry_url = "https://myregistry.dev/" - # image_name = "dev-image" - # image_tag = "v1.0.0" - # test_name = "TestMercuryLoad/all_endpoints" - # test_config_base64_env_name = "CONFIG_ENV_DEV" - # test_config_file_path = "/configs/dev/test-config.toml" - # test_config_base64 = "dGVzdCBjb25maWcgdmFsdWUgZGV2" - # test_timeout = "30m" - # resources_requests_cpu = "500m" - # resources_requests_memory = "1Gi" - # resources_limits_cpu = "1000m" - # resources_limits_memory = "2Gi" - # job_count = 2 - # chart_path = "/charts/dev" - # [envs] - # WASP_LOG_LEVEL = "info" - # TEST_LOG_LEVEL = "info" - # MERCURY_TEST_LOG_LEVEL = "info" - - # [[test_runs]] - # namespace = "prod-env" - # rbac_role_name = "prod-role" - # rbac_service_account_name = "prod-service-account" - # sync_value = "unique-sync-value-2" - # ttl_seconds_after_finished = 600 - # image_registry_url = "https://myregistry.prod/" - # image_name = "prod-image" - # image_tag = "v1.0.1" - # test_name = "TestMercuryLoad/all_endpoints" - # test_config_base64_env_name = "CONFIG_ENV_PROD" - # test_config_file_path = "/configs/prod/test-config.toml" - # test_config_base64 = "dGVzdCBjb25maWcgdmFsdWUgcHJvZA==" - # test_timeout = "45m" - # resources_requests_cpu = "800m" - # resources_requests_memory = "2Gi" - # resources_limits_cpu = "1500m" - # resources_limits_memory = "4Gi" - # job_count = 3 - # chart_path = "/charts/prod" - # [envs] - # WASP_LOG_LEVEL = "info" - # TEST_LOG_LEVEL = "info" - # MERCURY_TEST_LOG_LEVEL = "info" - - # # Schedule the tests in K8s in remote runner - # - name: Run Kubernetes Tests - # run: go run ./cmd/main.go run -c runner.toml diff --git a/.github/workflows/run-nightly-e2e-tests.yml b/.github/workflows/run-nightly-e2e-tests.yml index 151217180b0..671248b2ba2 100644 --- a/.github/workflows/run-nightly-e2e-tests.yml +++ b/.github/workflows/run-nightly-e2e-tests.yml @@ -9,10 +9,11 @@ on: jobs: call-run-e2e-tests-workflow: name: Run E2E Tests - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: chainlink_version: develop - test_workflow: Nightly E2E Tests + test_path: .github/e2e-tests.yml + test_trigger: Nightly E2E Tests slack_notification_after_tests: true slack_notification_after_tests_channel_id: "#team-test-tooling-internal" slack_notification_after_tests_name: Nightly E2E Tests diff --git a/.github/workflows/run-selected-e2e-tests.yml b/.github/workflows/run-selected-e2e-tests.yml index 2ff3fbb979d..e81d4f0d6de 100644 --- a/.github/workflows/run-selected-e2e-tests.yml +++ b/.github/workflows/run-selected-e2e-tests.yml @@ -20,11 +20,6 @@ on: description: 'Path to a test config file used to override the default test config' required: false type: string - enable_check_test_configurations: - description: 'Set to "true" to enable check-test-configurations job' - required: false - type: boolean - default: false with_existing_remote_runner_version: description: 'Use the existing remote runner version for k8s tests. Example: "d3bf5044af33e08be788a2df31c4a745cf69d787"' required: false @@ -40,14 +35,13 @@ run-name: ${{ inputs.workflow_run_name }} jobs: call-run-e2e-tests-workflow: name: Run E2E Tests - uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml + uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@aad83f232743646faa35f5ac03ee3829148d37ce with: chainlink_version: ${{ github.event.inputs.chainlink_version }} + test_path: .github/e2e-tests.yml test_ids: ${{ github.event.inputs.test_ids }} test_config_override_path: ${{ github.event.inputs.test_config_override_path }} with_existing_remote_runner_version: ${{ github.event.inputs.with_existing_remote_runner_version }} - # Use fromJSON to convert string to boolean. More info: https://github.com/actions/runner/issues/2206#issuecomment-1532246677 - enable_check_test_configurations: ${{ fromJSON(github.event.inputs.enable_check_test_configurations) }} secrets: QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }} QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }} diff --git a/integration-tests/go.mod b/integration-tests/go.mod index ddf31828fff..af53c0538df 100644 --- a/integration-tests/go.mod +++ b/integration-tests/go.mod @@ -42,7 +42,7 @@ require ( github.com/smartcontractkit/chainlink-ccip v0.0.0-20240924070918-21dc2dadfdad github.com/smartcontractkit/chainlink-common v0.2.3-0.20240919092417-53e784c2e420 github.com/smartcontractkit/chainlink-testing-framework/havoc v1.50.0 - github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.8 + github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.9 github.com/smartcontractkit/chainlink-testing-framework/lib/grafana v1.50.0 github.com/smartcontractkit/chainlink-testing-framework/seth v1.50.1 github.com/smartcontractkit/chainlink-testing-framework/wasp v1.50.0 diff --git a/integration-tests/go.sum b/integration-tests/go.sum index 8b11b1da718..ae8935211d6 100644 --- a/integration-tests/go.sum +++ b/integration-tests/go.sum @@ -1439,8 +1439,8 @@ github.com/smartcontractkit/chainlink-starknet/relayer v0.0.1-beta-test.0.202409 github.com/smartcontractkit/chainlink-starknet/relayer v0.0.1-beta-test.0.20240911194142-506bc469d8ae/go.mod h1:ec/a20UZ7YRK4oxJcnTBFzp1+DBcJcwqEaerUMsktMs= github.com/smartcontractkit/chainlink-testing-framework/havoc v1.50.0 h1:mgjBQIEy+3V3G6K8e+6by3xndgsXdYYsdy+7kzQZwSk= github.com/smartcontractkit/chainlink-testing-framework/havoc v1.50.0/go.mod h1:pdIxrooP5CFGmC0p5NTOBiZAFtMw+5pTT4de5GY3ywA= -github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.8 h1:AdFXA2XxiU5uqLS4ZxA7v+RCz8mhqz2aCxgzbpnja98= -github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.8/go.mod h1:7R5wGWWJi0dr5Y5cXbLQ4vSeIj0ElvhBaymcfvqqUmo= +github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.9 h1:/2kAb6y854viKigkdFMWDNNbaz3zD0gAkbZoSHC8Rrg= +github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.9/go.mod h1:7R5wGWWJi0dr5Y5cXbLQ4vSeIj0ElvhBaymcfvqqUmo= github.com/smartcontractkit/chainlink-testing-framework/lib/grafana v1.50.0 h1:VIxK8u0Jd0Q/VuhmsNm6Bls6Tb31H/sA3A/rbc5hnhg= github.com/smartcontractkit/chainlink-testing-framework/lib/grafana v1.50.0/go.mod h1:lyAu+oMXdNUzEDScj2DXB2IueY+SDXPPfyl/kb63tMM= github.com/smartcontractkit/chainlink-testing-framework/seth v1.50.1 h1:2OxnPfvjC+zs0ZokSsRTRnJrEGJ4NVJwZgfroS1lPHs= diff --git a/integration-tests/load/go.mod b/integration-tests/load/go.mod index 96b9015a02d..5a42e837ce6 100644 --- a/integration-tests/load/go.mod +++ b/integration-tests/load/go.mod @@ -16,7 +16,7 @@ require ( github.com/rs/zerolog v1.33.0 github.com/slack-go/slack v0.12.2 github.com/smartcontractkit/chainlink-common v0.2.3-0.20240919092417-53e784c2e420 - github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.8 + github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.9 github.com/smartcontractkit/chainlink-testing-framework/seth v1.50.1 github.com/smartcontractkit/chainlink-testing-framework/wasp v1.50.0 github.com/smartcontractkit/chainlink/integration-tests v0.0.0-20240214231432-4ad5eb95178c diff --git a/integration-tests/load/go.sum b/integration-tests/load/go.sum index 2ec777306d9..e9e1d440edf 100644 --- a/integration-tests/load/go.sum +++ b/integration-tests/load/go.sum @@ -1413,8 +1413,8 @@ github.com/smartcontractkit/chainlink-starknet/relayer v0.0.1-beta-test.0.202409 github.com/smartcontractkit/chainlink-starknet/relayer v0.0.1-beta-test.0.20240911194142-506bc469d8ae/go.mod h1:ec/a20UZ7YRK4oxJcnTBFzp1+DBcJcwqEaerUMsktMs= github.com/smartcontractkit/chainlink-testing-framework/havoc v1.50.0 h1:mgjBQIEy+3V3G6K8e+6by3xndgsXdYYsdy+7kzQZwSk= github.com/smartcontractkit/chainlink-testing-framework/havoc v1.50.0/go.mod h1:pdIxrooP5CFGmC0p5NTOBiZAFtMw+5pTT4de5GY3ywA= -github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.8 h1:AdFXA2XxiU5uqLS4ZxA7v+RCz8mhqz2aCxgzbpnja98= -github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.8/go.mod h1:7R5wGWWJi0dr5Y5cXbLQ4vSeIj0ElvhBaymcfvqqUmo= +github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.9 h1:/2kAb6y854viKigkdFMWDNNbaz3zD0gAkbZoSHC8Rrg= +github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.9/go.mod h1:7R5wGWWJi0dr5Y5cXbLQ4vSeIj0ElvhBaymcfvqqUmo= github.com/smartcontractkit/chainlink-testing-framework/lib/grafana v1.50.0 h1:VIxK8u0Jd0Q/VuhmsNm6Bls6Tb31H/sA3A/rbc5hnhg= github.com/smartcontractkit/chainlink-testing-framework/lib/grafana v1.50.0/go.mod h1:lyAu+oMXdNUzEDScj2DXB2IueY+SDXPPfyl/kb63tMM= github.com/smartcontractkit/chainlink-testing-framework/seth v1.50.1 h1:2OxnPfvjC+zs0ZokSsRTRnJrEGJ4NVJwZgfroS1lPHs=