Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TT-1806] add support for chain.link labels #15325

Closed
wants to merge 11 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/workflows/automation-benchmark-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,22 @@ on:
required: true
default: benchmark
type: string
team:
description: Team to run the tests for (e.g. BIX, CCIP)
required: true
type: string

jobs:
run-e2e-tests-workflow:
name: Run E2E Tests
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@5412507526722a7b1c5d719fa686eed5a1bc4035 # ctf-run-tests@0.2.0
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@0d4a2b2b009c87b5c366d0b97f7a8d7de2f60760
kalverra marked this conversation as resolved.
Show resolved Hide resolved
with:
test_path: .github/e2e-tests.yml
test_ids: '${{ inputs.testType }}/automation_test.go:TestAutomationBenchmark'
test_config_override_path: ${{ inputs.test_config_override_path }}
SLACK_USER: ${{ inputs.slackMemberID }}
SLACK_CHANNEL: C03KJ5S7KEK
team: ${{ inputs.team }}
secrets:
QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }}
QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }}
Expand Down
9 changes: 7 additions & 2 deletions .github/workflows/automation-load-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,23 @@ on:
description: Notifies test results (Not your @)
required: true
default: U02Q14G80TY
type: string
type: string
team:
description: Team to run the tests for (e.g. BIX, CCIP)
required: true
type: string

jobs:
run-e2e-tests-workflow:
name: Run E2E Tests
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@5412507526722a7b1c5d719fa686eed5a1bc4035 # ctf-run-tests@0.2.0
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@0d4a2b2b009c87b5c366d0b97f7a8d7de2f60760
with:
test_path: .github/e2e-tests.yml
test_ids: 'load/automationv2_1/automationv2_1_test.go:TestLogTrigger'
test_config_override_path: ${{ inputs.test_config_override_path }}
SLACK_USER: ${{ inputs.slackMemberID }}
SLACK_CHANNEL: C03KJ5S7KEK
team: ${{ inputs.team }}
secrets:
QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }}
QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }}
Expand Down
12 changes: 8 additions & 4 deletions .github/workflows/automation-ondemand-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,11 @@ on:
with_existing_remote_runner_version:
description: 'Tag of the existing remote runner version to use (Leave empty to build from head/ref)'
required: false
type: string
type: string
team:
description: Team to run the tests for (e.g. BIX, CCIP). Required for load and chaos tests.
required: false
type: string

jobs:
# Set tests to run based on the workflow inputs
Expand Down Expand Up @@ -84,11 +88,10 @@ jobs:
if [ "${{ github.event.inputs.chainlinkImageUpdate }}" = "QA_ECR" ]; then
chainlink_image_versions+="${{ env.upgrade_version }}"
fi
echo "require_chainlink_image_versions_in_qa_ecr=$chainlink_image_versions" >> $GITHUB_OUTPUT
echo "require_chainlink_image_versions_in_qa_ecr=$chainlink_image_versions" >> $GITHUB_OUTPUT
- name: Set tests to run
id: set-tests
run: |

# Always run upgrade tests
cat > test_list.yaml <<EOF
- id: smoke/automation_upgrade_test.go:^TestAutomationNodeUpgrade/registry_2_0
Expand Down Expand Up @@ -153,14 +156,15 @@ jobs:
call-run-e2e-tests-workflow:
name: Run E2E Tests
needs: set-tests-to-run
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@5412507526722a7b1c5d719fa686eed5a1bc4035 # ctf-run-tests@0.2.0
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@0d4a2b2b009c87b5c366d0b97f7a8d7de2f60760
with:
test_path: .github/e2e-tests.yml
test_list: ${{ needs.set-tests-to-run.outputs.test_list }}
require_chainlink_image_versions_in_qa_ecr: ${{ needs.set-tests-to-run.outputs.require_chainlink_image_versions_in_qa_ecr }}
with_existing_remote_runner_version: ${{ github.event.inputs.with_existing_remote_runner_version }}
test_log_upload_on_failure: true
test_log_upload_retention_days: 7
team: ${{ inputs.team }}
secrets:
QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }}
QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }}
Expand Down
9 changes: 8 additions & 1 deletion .github/workflows/ccip-chaos-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@ on:
# types: [ completed ]
# branches: [ develop ]
workflow_dispatch:
inputs:
team:
description: Team to run the tests for (e.g. BIX, CCIP)
required: true
default: "ccip"
type: string

# Only run 1 of this workflow at a time per PR
concurrency:
Expand All @@ -15,7 +21,7 @@ concurrency:
jobs:
run-e2e-tests-workflow:
name: Run E2E Tests
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@5412507526722a7b1c5d719fa686eed5a1bc4035 # ctf-run-tests@0.2.0
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@0d4a2b2b009c87b5c366d0b97f7a8d7de2f60760
with:
test_path: .github/e2e-tests.yml
chainlink_version: ${{ github.sha }}
Expand All @@ -25,6 +31,7 @@ jobs:
slack_notification_after_tests: on_failure
slack_notification_after_tests_channel_id: '#ccip-testing'
slack_notification_after_tests_name: CCIP Chaos E2E Tests
team: ${{ inputs.team }}
secrets:
QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }}
QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }}
Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/ccip-load-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,12 @@ on:
chainlink_version:
description: Chainlink image version to use. Commit sha if not provided
required: false
type: string
type: string
team:
description: Team to run the tests for (e.g. BIX, CCIP)
required: true
default: "ccip"
type: string

# Only run 1 of this workflow at a time per PR
concurrency:
Expand All @@ -31,7 +36,7 @@ concurrency:
jobs:
run-e2e-tests-workflow:
name: Run E2E Tests
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@5412507526722a7b1c5d719fa686eed5a1bc4035 # ctf-run-tests@0.2.0
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@0d4a2b2b009c87b5c366d0b97f7a8d7de2f60760
with:
test_path: .github/e2e-tests.yml
test_trigger: E2E CCIP Load Tests
Expand All @@ -41,6 +46,8 @@ jobs:
slack_notification_after_tests_channel_id: '#ccip-testing'
slack_notification_after_tests_name: CCIP E2E Load Tests
test_image_suites: ccip-load
team: ${{ inputs.team }}

secrets:
QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }}
QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }}
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -296,7 +296,7 @@ jobs:
contents: read
needs: [build-chainlink, changes]
if: github.event_name == 'pull_request' && (needs.changes.outputs.ccip_changes == 'true' || needs.changes.outputs.github_ci_changes == 'true')
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@5412507526722a7b1c5d719fa686eed5a1bc4035 # ctf-run-tests@0.2.0
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@0d4a2b2b009c87b5c366d0b97f7a8d7de2f60760
with:
workflow_name: Run CCIP E2E Tests For PR
chainlink_version: ${{ inputs.evm-ref || github.sha }}
Expand All @@ -306,6 +306,7 @@ jobs:
upload_cl_node_coverage_artifact: true
upload_cl_node_coverage_artifact_prefix: cl_node_coverage_data_
enable_otel_traces_for_ocr2_plugins: ${{ contains(join(github.event.pull_request.labels.*.name, ' '), 'enable tracing') }}
team: "CCIP"
secrets:
QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }}
QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }}
Expand Down
9 changes: 7 additions & 2 deletions .github/workflows/on-demand-ocr-soak-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,18 +34,23 @@ on:
slackMemberID:
description: Slack Member ID (Not your @)
required: true
default: U01A2B2C3D4
default: U01A2B2C3D4
team:
description: Team to run the tests for (e.g. BIX, CCIP)
required: true
type: string

jobs:
run-e2e-tests-workflow:
name: Run E2E Tests
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@5412507526722a7b1c5d719fa686eed5a1bc4035 # ctf-run-tests@0.2.0
uses: smartcontractkit/.github/.github/workflows/run-e2e-tests.yml@0d4a2b2b009c87b5c366d0b97f7a8d7de2f60760
with:
test_path: .github/e2e-tests.yml
test_ids: ${{ inputs.testToRun}}
test_config_override_path: ${{ inputs.test_config_override_path }}
chainlink_version: ${{ inputs.chainlink_version }}
SLACK_USER: ${{ inputs.slackMemberID }}
team: ${{ inputs.team }}
secrets:
QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }}
QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }}
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/.tool-versions
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@ golang 1.23.3
k3d 5.4.6
kubectl 1.25.5
nodejs 20.13.1
golangci-lint 1.61.1
golangci-lint 1.62.0
task 3.35.1
15 changes: 12 additions & 3 deletions integration-tests/benchmark/automation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ func TestAutomationBenchmark(t *testing.T) {
config, err := tc.GetConfig([]string{testType}, tc.Automation)
require.NoError(t, err, "Error getting test config")

testEnvironment, benchmarkNetwork := SetupAutomationBenchmarkEnv(t, &config)
testEnvironment, benchmarkNetwork := SetupAutomationBenchmarkEnv(t, testType, &config)
if testEnvironment.WillUseRemoteRunner() {
return
}
Expand Down Expand Up @@ -245,7 +245,7 @@ var networkConfig = map[string]NetworkConfig{
},
}

func SetupAutomationBenchmarkEnv(t *testing.T, keeperTestConfig types.AutomationBenchmarkTestConfig) (*environment.Environment, blockchain.EVMNetwork) {
func SetupAutomationBenchmarkEnv(t *testing.T, testType string, keeperTestConfig types.AutomationBenchmarkTestConfig) (*environment.Environment, blockchain.EVMNetwork) {
l := logging.GetTestLogger(t)
testNetwork := networks.MustGetSelectedNetworkConfig(keeperTestConfig.GetNetworkConfig())[0] // Environment currently being used to run benchmark test on
blockTime := "1"
Expand All @@ -259,6 +259,12 @@ func SetupAutomationBenchmarkEnv(t *testing.T, keeperTestConfig types.Automation
networkName = strings.ReplaceAll(networkName, "_", "-")
testNetwork.Name = networkName

nsLabels, err := environment.GetRequiredChainLinkNamespaceLabels(string(tc.Keeper), testType)
require.NoError(t, err, "Error creating required chain.link labels for namespace")

workloadPodLabels, err := environment.GetRequiredChainLinkWorkloadAndPodLabels(string(tc.Keeper), testType)
require.NoError(t, err, "Error creating required chain.link labels for workloads and pods")

testEnvironment := environment.New(&environment.Config{
TTL: time.Hour * 720, // 30 days,
NamespacePrefix: fmt.Sprintf(
Expand All @@ -269,6 +275,9 @@ func SetupAutomationBenchmarkEnv(t *testing.T, keeperTestConfig types.Automation
),
Test: t,
PreventPodEviction: true,
Labels: nsLabels,
WorkloadLabels: workloadPodLabels,
PodLabels: workloadPodLabels,
})

dbResources := dbResources
Expand Down Expand Up @@ -316,7 +325,7 @@ func SetupAutomationBenchmarkEnv(t *testing.T, keeperTestConfig types.Automation
},
}))
}
var err error

if testNetwork.Simulated {
// TODO we need to update the image in CTF, the old one is not available anymore
// deploy blockscout if running on simulated
Expand Down
12 changes: 12 additions & 0 deletions integration-tests/ccip-tests/testsetups/ccip.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ import (
"go.uber.org/zap/zapcore"
"golang.org/x/sync/errgroup"

tc "github.com/smartcontractkit/chainlink/integration-tests/testconfig"

chainselectors "github.com/smartcontractkit/chain-selectors"

"github.com/smartcontractkit/chainlink-testing-framework/lib/blockchain"
Expand Down Expand Up @@ -1414,9 +1416,19 @@ func (o *CCIPTestSetUpOutputs) CreateEnvironment(
}

func createEnvironmentConfig(t *testing.T, envName string, testConfig *CCIPTestConfig, reportPath string) *environment.Config {
testType := testConfig.TestGroupInput.Type
nsLabels, err := environment.GetRequiredChainLinkNamespaceLabels(string(tc.CCIP), testType)
require.NoError(t, err, "Error creating required chain.link labels for namespace")

workloadPodLabels, err := environment.GetRequiredChainLinkWorkloadAndPodLabels(string(tc.CCIP), testType)
require.NoError(t, err, "Error creating required chain.link labels for workloads and pods")

envConfig := &environment.Config{
NamespacePrefix: envName,
Test: t,
Labels: nsLabels,
WorkloadLabels: workloadPodLabels,
PodLabels: workloadPodLabels,
// PreventPodEviction: true, //TODO: enable this once we have a way to handle pod eviction
}
if pointer.GetBool(testConfig.TestGroupInput.StoreLaneConfig) {
Expand Down
9 changes: 9 additions & 0 deletions integration-tests/chaos/automation_chaos_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -197,11 +197,20 @@ func TestAutomationChaos(t *testing.T) {
t.Parallel()
network := networks.MustGetSelectedNetworkConfig(config.Network)[0] // Need a new copy of the network for each test

nsLabels, err := environment.GetRequiredChainLinkNamespaceLabels(string(tc.Automation), "chaos")
require.NoError(t, err, "Error creating required chain.link labels for namespace")

workloadPodLabels, err := environment.GetRequiredChainLinkWorkloadAndPodLabels(string(tc.Automation), "chaos")
require.NoError(t, err, "Error creating required chain.link labels for workloads and pods")

testEnvironment := environment.
New(&environment.Config{
NamespacePrefix: fmt.Sprintf("chaos-automation-%s", name),
TTL: time.Hour * 1,
Test: t,
Labels: nsLabels,
WorkloadLabels: workloadPodLabels,
PodLabels: workloadPodLabels,
}).
AddHelm(testCase.networkChart).
AddHelm(testCase.clChart)
Expand Down
11 changes: 10 additions & 1 deletion integration-tests/chaos/ocr_chaos_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -141,15 +141,24 @@ func TestOCRChaos(t *testing.T) {
t.Run(fmt.Sprintf("OCR_%s", name), func(t *testing.T) {
t.Parallel()

nsLabels, err := environment.GetRequiredChainLinkNamespaceLabels("data-feedsv1.0", "chaos")
require.NoError(t, err, "Error creating required chain.link labels for namespace")

workloadPodLabels, err := environment.GetRequiredChainLinkWorkloadAndPodLabels("data-feedsv1.0", "chaos")
require.NoError(t, err, "Error creating required chain.link labels for workloads and pods")

testEnvironment := environment.New(&environment.Config{
NamespacePrefix: fmt.Sprintf("chaos-ocr-%s", name),
Test: t,
Labels: nsLabels,
WorkloadLabels: workloadPodLabels,
PodLabels: workloadPodLabels,
}).
AddHelm(mockservercfg.New(nil)).
AddHelm(mockserver.New(nil)).
AddHelm(testCase.networkChart).
AddHelm(testCase.clChart)
err := testEnvironment.Run()
err = testEnvironment.Run()
require.NoError(t, err)
if testEnvironment.WillUseRemoteRunner() {
return
Expand Down
2 changes: 2 additions & 0 deletions integration-tests/example.env
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

########## General Test Settings ##########
export CHAINLINK_ENV_USER="Satoshi-Nakamoto" # Name of the person running the tests (change to your own)
export CHAINLINK_USER_TEAM="My awesome team" # Name of the team you are running the test for (change to your own)
export TEST_LOG_LEVEL="info" # info | debug | trace

########## Soak/Chaos/Load Test Specific Settings ##########
Expand All @@ -16,6 +17,7 @@ export SLACK_API_KEY="xoxb-example-key" # API key used to report soak test resul
export SLACK_CHANNEL="C000000000" # Channel ID for the slack bot to post test results
export SLACK_USER="U000000000" # User ID of the person running the soak tests to properly notify them

##### ---- applicable only, when using legacy EVMClient ---- #####
########## Network Settings ##########
# General EVM Settings, used only for quick prototyping when using GENERAL as the SELECTED_NETWORK
export EVM_NAME="General EVM"
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ require (
github.com/smartcontractkit/chainlink-common v0.3.1-0.20241125150608-97ceadb2072d
github.com/smartcontractkit/chainlink-protos/job-distributor v0.6.0
github.com/smartcontractkit/chainlink-testing-framework/havoc v1.50.2
github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.17
github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.18
github.com/smartcontractkit/chainlink-testing-framework/lib/grafana v1.50.0
github.com/smartcontractkit/chainlink-testing-framework/seth v1.50.9
github.com/smartcontractkit/chainlink-testing-framework/wasp v1.50.2
Expand Down
4 changes: 2 additions & 2 deletions integration-tests/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1423,8 +1423,8 @@ github.com/smartcontractkit/chainlink-starknet/relayer v0.1.1-0.20241017135645-1
github.com/smartcontractkit/chainlink-starknet/relayer v0.1.1-0.20241017135645-176a23722fd8/go.mod h1:WkBqgBo+g34Gm5vWkDDl8Fh3Mzd7bF5hXp7rryg0t5o=
github.com/smartcontractkit/chainlink-testing-framework/havoc v1.50.2 h1:GDGrC5OGiV0RyM1znYWehSQXyZQWTOzrEeJRYmysPCE=
github.com/smartcontractkit/chainlink-testing-framework/havoc v1.50.2/go.mod h1:DsT43c1oTBmp3iQkMcoZOoKThwZvt8X3Pz6UmznJ4GY=
github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.17 h1:Fw2F8fKa5QdOUzLAj6Y/EB6XFC0QtK2pw5bqQSatL4A=
github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.17/go.mod h1:NwmlNKqrb02v4Sci4b5KW644nfH2BW+FrKbWwTN5r6M=
github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.18 h1:a3xetGZh2nFO1iX5xd9OuqiCkgbWLvW6fTN6fgVubPo=
github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.18/go.mod h1:NwmlNKqrb02v4Sci4b5KW644nfH2BW+FrKbWwTN5r6M=
github.com/smartcontractkit/chainlink-testing-framework/lib/grafana v1.50.0 h1:VIxK8u0Jd0Q/VuhmsNm6Bls6Tb31H/sA3A/rbc5hnhg=
github.com/smartcontractkit/chainlink-testing-framework/lib/grafana v1.50.0/go.mod h1:lyAu+oMXdNUzEDScj2DXB2IueY+SDXPPfyl/kb63tMM=
github.com/smartcontractkit/chainlink-testing-framework/seth v1.50.9 h1:yB1x5UXvpZNka+5h57yo1/GrKfXKCqMzChCISpldZx4=
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,12 @@ Load Config:
loadDuration := time.Duration(*loadedTestConfig.Automation.General.Duration) * time.Second
automationDefaultLinkFunds := big.NewInt(0).Mul(big.NewInt(1e18), big.NewInt(int64(10000))) //10000 LINK

nsLabels, err := environment.GetRequiredChainLinkNamespaceLabels(string(tc.Automation), testType)
require.NoError(t, err, "Error creating required chain.link labels for namespace")

workloadPodLabels, err := environment.GetRequiredChainLinkWorkloadAndPodLabels(string(tc.Automation), testType)
require.NoError(t, err, "Error creating required chain.link labels for workloads and pods")

testEnvironment := environment.New(&environment.Config{
TTL: loadDuration.Round(time.Hour) + time.Hour,
NamespacePrefix: fmt.Sprintf(
Expand All @@ -205,6 +211,9 @@ Load Config:
),
Test: t,
PreventPodEviction: true,
Labels: nsLabels,
WorkloadLabels: workloadPodLabels,
PodLabels: workloadPodLabels,
})

testEnvironment.
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/load/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ require (
github.com/rs/zerolog v1.33.0
github.com/slack-go/slack v0.15.0
github.com/smartcontractkit/chainlink-common v0.3.1-0.20241125150608-97ceadb2072d
github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.17
github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.18
github.com/smartcontractkit/chainlink-testing-framework/seth v1.50.9
github.com/smartcontractkit/chainlink-testing-framework/wasp v1.50.2
github.com/smartcontractkit/chainlink/deployment v0.0.0-00010101000000-000000000000
Expand Down
Loading
Loading