Skip to content

Commit

Permalink
feat: allow more detailed additional test configurations in assertoor…
Browse files Browse the repository at this point in the history
…_params (#498)

This adds the ability do use complex objects that are passed through to
the assertoor config in json format.
It allows defining assertoor tests like this via the kurtosis params:
```
assertoor_params:
  run_stability_check: false
  run_block_proposal_check: true
  tests:
    - file: https://raw.githubusercontent.com/ethpandaops/assertoor-test/master/assertoor-tests/block-proposal-check.yaml
      timeout: 2h
      config:
        someCustomTestConfig: "some value"
```

The old way to specify the link to the test only is still supported:
```
assertoor_params:
  run_stability_check: false
  run_block_proposal_check: true
  tests:
    - "https://raw.githubusercontent.com/ethpandaops/assertoor-test/master/assertoor-tests/block-proposal-check.yaml"
```

Effectively both formats can be used.


The downstream implementation is unfortunately not super nice 😅
It embeds the test configs as json objects within the yaml
configuration.
This works fine as yaml inherits the json syntax.
  • Loading branch information
pk910 committed Feb 22, 2024
1 parent e48483a commit fe2de7e
Show file tree
Hide file tree
Showing 3 changed files with 19 additions and 2 deletions.
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -473,8 +473,12 @@ assertoor_params:
run_lifecycle_test: false
# Run additional tests from external test definitions
# Entries may be simple strings (link to the test file) or dictionaries with more flexibility
# eg:
# - https://raw.githubusercontent.com/ethpandaops/assertoor/master/example/tests/block-proposal-check.yaml
# - file: "https://raw.githubusercontent.com/ethpandaops/assertoor/master/example/tests/block-proposal-check.yaml"
# config:
# someCustomTestConfig: "some value"
tests: []
Expand Down
15 changes: 14 additions & 1 deletion src/assertoor/assertoor_launcher.star
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,19 @@ def get_config(
def new_config_template_data(
listen_port_num, client_info, validator_client_info, assertoor_params
):
additional_tests = []
for index, testcfg in enumerate(assertoor_params.tests):
if type(testcfg) == "dict":
additional_tests.append(json.encode(testcfg))
else:
additional_tests.append(
json.encode(
{
"file": testcfg,
}
)
)

return {
"ListenPortNum": listen_port_num,
"ClientInfo": client_info,
Expand All @@ -147,7 +160,7 @@ def new_config_template_data(
"RunTransactionTest": assertoor_params.run_transaction_test,
"RunBlobTransactionTest": assertoor_params.run_blob_transaction_test,
"RunOpcodesTransactionTest": assertoor_params.run_opcodes_transaction_test,
"AdditionalTests": assertoor_params.tests,
"AdditionalTests": additional_tests,
}


Expand Down
2 changes: 1 addition & 1 deletion static_files/assertoor-config/config.yaml.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -51,5 +51,5 @@ externalTests:
- file: /tests/validator-lifecycle-test.yaml
{{ end }}
{{ range $test := .AdditionalTests }}
- file: "{{ $test }}"
- {{ $test }}
{{- end }}

0 comments on commit fe2de7e

Please sign in to comment.