Artillery v2.0.21

Core

• Add --env-file flag as an alternative for --dotenv flag. This makes it consistent with the Node.js --env-file flag. The --dotenv flag will be deprecated in a future release (#3376)
• Add tracking of response times by HTTP status code. A new set of metrics (e.g. http.response_time.2xx or http.response_time.5xx) is now reported to provide more granular view of response times in a test (#3326)
• Fix an issue that caused incorrect "multiple batches of metics" warnings when running tests with pause phases (#3331)

Playwright

• Upgrade Playwright to v1.48.0

Azure

• Fix: Make values loaded from an env file with --dotenv / --env-file flag available to workers containers (rather than just the Artillery process running inside the worker) (#3376)
• Stagger startup of containers in large load tests to prevent rate limit errors from Azure services (#3371)

OpenTelemetry

• Add support for setting resource-level attributes (#3335)

Artillery Cloud

• Send CI related information to Artillery Cloud. This makes the following information available in Artillery Cloud:
  • Whether a test run was triggered in CI or not, and which CI service was used
  • For tests triggered in GitHub Actions - provide a link back to the job run on GitHub Actions

Dependencies

• Upgrade json-plus to address a critical security vulnerability (#3369)

---

With contributions from @hassy @Archangelza1 @dirkluijk

Artillery v2.0.20

Core & CLI

• New: add ability to set a custom content type for multipart/form-data form fields (#3316) — docs
• Fix: don't print an unnecessary warning when loadAll is not set by @hassy in (#3303)
• Remove legacy Artillery Pro integration (#3320)

Artillery Cloud

• New: add --name option to set the name of the test to be shown in Artillery Cloud dashboard (#3317) — docs
• Improve pre-flight checks when recording reports to Artillery Cloud to detect potential firewall/proxy issues (#3314)
• Fix: include organization IDs in the test report URLs

Azure

• Fix: bundle separate config files provided with --config option correctly (#3312)
• Fix: bundle dotenv files provided with --dotenv correctly (#3313)
• Fix: bundle custom .npmrc and other dotfiles correctly (#3312)
• Improve error handling and reporting for container provisioning errors (#3313)

---

With contributions from @hassy & @RikdeVos

Artillery v2.0.19

CLI

• Fix bug preventing custom code using ES modules from loading on Windows (#2662)
• Prevent setting individual CSV rows in context when using loadAll (#3277)

Fargate

• Add ability to set DNS servers using the --container-dns-servers flag when running on ECS/EC2 (#3301)
• Add ability to override the ephemeral storage amount for each task via the --task-ephemeral-storage flag (#3301)

Playwright

• Upgrade Playwright to v1.45.3 (#3294)

---

With contributions from @bernardobridge, @hassy and @thrandale

Artillery v2.0.18

CLI

• Metrics reported for dynamic URLs are grouped by default now (#3264) - docs

Playwright

• Upgrade Playwright to v1.45.2 (#3278)

Artillery v2.0.17

HTTP

• Add support for file uploads (#3232) - docs

Playwright

• Upgrade Playwright to v1.45.0 (#3245)

---

With contributions from @bernardobridge @InesNi

Artillery v2.0.16

AWS ECS

• Fix issue preventing running tests on AWS ECS when using an EC2 launch type (#3192, a49e371)
• Wait for IAM role for ECS workers to be ready when running tests for the first time (#3198)
• Add run:ecs and run-ecs aliases to run-fargate command (78683fa)

CLI

• Update tough-cookie package to v5 to prevent usage of deprecated package (#3209)
• Prevent setting the wrong value when using the --platform flag (#3227)

---

With contributions from: @bernardobridge @InesNi @hassy

Artillery v2.0.15

AWS Lambda

• AWS Lambda support is now considered stable (#2815)
• Fix error happening when multiple Lambda tests were created in parallel for the first time (#2802)
• Remove --container flag (all run-lambda tests now run using Lambda Container Images) (#2816)
• Exit with non-zero exit code when expectations fail in AWS Lambda workers (#2808)

AWS Fargate

• Prevent CLI hanging when workers are terminated early (e.g. due to early termination when running tests on Fargate Spot) #3190
• Exit with non-zero exit code when expectations fail in AWS Fargate workers (#2808)

Playwright

• Update Playwright version to 1.44.1 (#3189)

CLI

• Make HTTP tests take strictCapture option into account when set in defaults (#2795)

---

With contributions from: @bernardobridge @hassy

Artillery v2.0.14

This is a hotfix release to fix an issue introduced in Artillery v2.0.13 which would lead to excessive logging when running Playwright tests.

Artillery v2.0.13

AWS Lambda

Lambda Container Images

Artillery will now always use Lambda Container Images to run tests on AWS Lambda. The --container flag introduced in v2.0.12 is no longer required and will be removed in the next release. Creating Lambda functions from .zip archives is no longer supported.

All existing tests should continue running normally without needing any changes. If you run into any issues please open an issue.

This change brings improved startup times for tests running on AWS Lambda and brings AWS Lambda support to parity with AWS Fargate.

Fixes

• Increase startup timeout time for Lambdas running in a VPC (#2778)
• Exit with non-zero code if a Lambda worker fails (#2766)
• Fix issue that could lead tests to crash when triggered from a Windows machine (#2767)
• Expose current worker ID to Lambda workers as WORKED_ID environment variable (#2754)

Playwright

• Accept 0 values for defaultNavigationTimeout and defaultTimeout settings (#2786)
• A warning will be printed if a Playwright testFunction cannot be found (#2782)

Monitoring & observability (publish-metrics plugin)

• Add url attribute to page spans when tracing is enabled for Playwright tests (#2779)
• Fix issue which could cause Core Web Vitals to not be set as span attributes (#2779)
• A test_id attribute is no longer set on all metrics. The built-in $testId variable may be used to set that attribute when needed (#2779)
• Request timing phases such as DNS lookups and SSL handshake timings are now recorded as attributes instead of spans for HTTP tests (#2779)

Slack integration

Fix a number of issues that prevented the Slack plugin from working with tests running on AWS Fargate and AWS Lambda (#2754)

WebSocket

• Errors thrown by hook functions will now be tracked and reported (#2783)

---

With contributions from: @bbeesley @HugoImaios @InesNi @bernardobridge @hassy

Artillery v2.0.12

AWS Lambda

Artillery can now use Lambda Container Images to run tests on AWS Lambda. Pass the --container flag to run-lambda command to use a container instead of a zip file. No further configuration is required.

This change brings two main benefits:

1. Much faster startup times for tests
2. Ability to use certain features that were previously only available in tests running on AWS Fargate:

• Run test scripts that TypeScript hooks
• Use third-party dependencies
• Use the built-in publish-metrics plugin

Container-based Lambda functions will become the default option in a future release of Artillery.

• #2674 • #2724

Playwright

• Update Playwright to v1.44.0
• Fix tests failing on Fargate with ENOSPC error with Playwright tracing enabled #2717
• Improve reliability of trace collection:
  • Reduce trace collection cooldown period to 1-5m
  • Increase the number of concurrent trace recordings to 3 up from 1
• Track the number of collected/discarded recordings with browser.traces_collected and browser.traces_discarded metrics

CLI & Core

• Make path to the config directory available to test scripts via the special $dirname variable #2614 docs
• The --scenario-name flag will now default to using an exact string match, and fall back to using a regex #2709
• Clarify the meaning of --tags flag #2699
• Remove an unnecessary dependency #2703

Slack integration

• Artillery now includes an official Slack integration, which can be used to post test results to a Slack channel automatically #2688 docs

---

With contributions from: @bernardobridge @InesNi @hassy @43081j @MIJOTHY-V2