Skip to content
This repository has been archived by the owner on Jun 2, 2023. It is now read-only.

Metrics coming from plugin with compound name are not usable in thresholds #1

Open
aymeric-cr opened this issue Nov 23, 2022 · 0 comments

Comments

@aymeric-cr
Copy link

aymeric-cr commented Nov 23, 2022

Problem

I want to use metrics provided by artillery-plugin-metrics-by-endpoint in thresholds. The problem is that checks always fails whatever the threshold value or the response time of measured requests.

Example

According the following configuration:

config:
  plugins:
    metrics-by-endpoint:
      useOnlyRequestNames: true
    ensure: { }

  ensure:
    thresholds:
      - plugins.metrics-by-endpoint.response_time.publish.p99: 9999
      - plugins.metrics-by-endpoint.response_time.filter.p99: 9999

Note: metrics plugins.metrics-by-endpoint.response_time.publish and plugins.metrics-by-endpoint.response_time.filter come from the artillery-plugin-metrics-by-endpoint plugin.

And the following results:

All VUs finished. Total time: 1 minute, 53 seconds

--------------------------------
Summary report @ 16:24:19(+0100)
--------------------------------

http.codes.200: ................................................................ 8166
http.request_rate: ............................................................. 60/sec
http.requests: ................................................................. 8166
http.response_time:
  min: ......................................................................... 12
  max: ......................................................................... 654
  median: ...................................................................... 24.8
  p95: ......................................................................... 278.7
  p99: ......................................................................... 478.3
http.responses: ................................................................ 8166
plugins.metrics-by-endpoint.filter.codes.200: .................................. 92
plugins.metrics-by-endpoint.publish.codes.200: ................................. 8074
plugins.metrics-by-endpoint.response_time.filter:
  min: ......................................................................... 71
  max: ......................................................................... 594
  median: ...................................................................... 106.7
  p95: ......................................................................... 399.5
  p99: ......................................................................... 528.6
plugins.metrics-by-endpoint.response_time.publish:
  min: ......................................................................... 12
  max: ......................................................................... 654
  median: ...................................................................... 24.8
  p95: ......................................................................... 278.7
  p99: ......................................................................... 478.3
vusers.completed: .............................................................. 826
vusers.created: ................................................................ 826
vusers.created_by_name.Create a session with 10 user actions: .................. 734
vusers.created_by_name.Filter sessions: ........................................ 92
vusers.failed: ................................................................. 0
vusers.session_length:
  min: ......................................................................... 73.7
  max: ......................................................................... 4318.1
  median: ...................................................................... 284.3
  p95: ......................................................................... 3262.4
  p99: ......................................................................... 4231.1
Log file: ./test/artillery/artillery.json

Checks an these metrics keep failing:

Checks:
fail: plugins.metrics-by-endpoint.response_time.publish.p99 < 9999
fail: plugins.metrics-by-endpoint.response_time.filter.p99 < 9999

Process finished with exit code 1

Hypotesis

This issue happened when a metric name containing - is provided to the function compileExpression from filtrex module because the minus signs are interpreted as subtraction operators.

This behavior is problematic since the naming standard of artillery plugins is to used - in compound names.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant