add support to api security sampling #4755

IlyasShabi · 2024-10-03T07:48:12Z

What does this PR do?

Implements a new API security sampling algorithm using an LRU cache with a 30s TTL
Delays schema extraction until the end of the request processing

Motivation

The current API security sampling method randomly selects 10% of requests. This PR introduces new algorithm based on request priority, utilizing an LRU cache with TTL for each { url, statusCode, method }

Delaying the schema extraction to the end of the request ensures that we have an accurate priority value

Plugin Checklist

github-actions · 2024-10-03T07:48:51Z

Overall package size

Self size: 7.97 MB
Deduped: 65.01 MB
No deduping: 65.35 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/native-appsec | 8.2.1 | 19.18 MB | 19.19 MB | | @datadog/native-iast-taint-tracking | 3.2.0 | 13.9 MB | 13.91 MB | | @datadog/pprof | 5.4.1 | 9.76 MB | 10.13 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.5.0 | 2.51 MB | 2.65 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.0.1 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 826.22 kB | | msgpack-lite | 0.1.26 | 201.16 kB | 281.59 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | semver | 7.6.3 | 95.82 kB | 95.82 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.1 | 51.46 kB | 51.46 kB | | int64-buffer | 0.1.10 | 49.18 kB | 49.18 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | path-to-regexp | 0.1.10 | 6.38 kB | 6.38 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

pr-commenter · 2024-10-03T12:21:16Z

Benchmarks

Benchmark execution time: 2024-11-13 08:30:06

Comparing candidate commit 37d6aab in PR branch api-security-sampling with baseline commit 1ee8000 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 259 metrics, 7 unstable metrics.

index.d.ts

packages/dd-trace/src/appsec/api_security_sampler_cache.js

packages/dd-trace/src/appsec/index.js

packages/dd-trace/src/appsec/api_security_sampler.js

docs/test.ts

packages/dd-trace/src/appsec/api_security_sampler.js

integration-tests/standalone-asm.spec.js

packages/dd-trace/src/config.js

packages/dd-trace/src/appsec/api_security_sampler.js

simon-id

haven't reviewed it all yet but that should keep you busy:

packages/dd-trace/src/appsec/remote_config/capabilities.js

index.d.ts

packages/dd-trace/src/appsec/api_security_sampler.js

integration-tests/standalone-asm.spec.js

packages/dd-trace/test/config.spec.js

packages/dd-trace/test/appsec/index.express.plugin.spec.js

simon-id · 2024-11-07T16:14:48Z

Code LGTM now, but haven't had time to fully review the tests. still I found some problems

packages/dd-trace/src/appsec/api_security_sampler.js

simon-id · 2024-11-12T13:30:29Z

You got some system tests failures for API Security scenario, is it just the old tests that we should disable now ?

simon-id · 2024-11-12T15:21:38Z

Code and tests LGTM, but let's resolve that CI failure

IlyasShabi · 2024-11-13T08:29:31Z

In some scenarios, we need to specify that the sampling delay is 0 "DD_API_SECURITY_SAMPLE_DELAY": "0.0" which means we want to sample all requests. Unfortunately, the @isaacs/ttlcache library does not support ttl: 0, so we need to handle this ourselves.

Here are some failing system test scenarios:

Solutions:

As implemented in the last commit, if the delay is 0, we instantiate a dummy TTL cache with basic functions to handle this edge case.
Completely remove the @isaacs/ttlcache library and replace it with a custom implementation, similar to the python implementation

WDYT? @iunanua @simon-id

packages/dd-trace/src/appsec/api_security_sampler.js

* add support to api security sampling * fix express plugin schema extraction * use priority simpler to get span priority * use lru cache package * use route path instead of url * use route.path or url to generate the key * use ttlcache * Fix standalone integration test * Increase test timeout * simplify force sample * avoid checking is sampled twice * use route.path or url to generate the key * remove unnecessary tests of sample delay * fix non experimental options test * remove unused isSampled * always sample request if delay is 0 --------- Co-authored-by: Igor Unanua <igor.unanua@datadoghq.com> Co-authored-by: simon-id <simon.id@datadoghq.com>

IlyasShabi force-pushed the api-security-sampling branch from 90fe2c8 to 6987a66 Compare October 3, 2024 12:12

IlyasShabi added appsec semver-minor labels Oct 3, 2024

IlyasShabi self-assigned this Oct 3, 2024

IlyasShabi force-pushed the api-security-sampling branch from e5a079e to ca8bb68 Compare October 3, 2024 15:09

IlyasShabi marked this pull request as ready for review October 3, 2024 15:17

IlyasShabi requested review from a team as code owners October 3, 2024 15:17

IlyasShabi commented Oct 3, 2024

View reviewed changes

index.d.ts Show resolved Hide resolved

CarlesDD reviewed Oct 4, 2024

View reviewed changes

packages/dd-trace/src/appsec/api_security_sampler_cache.js Outdated Show resolved Hide resolved

uurien reviewed Oct 4, 2024

View reviewed changes

packages/dd-trace/src/appsec/index.js Show resolved Hide resolved

packages/dd-trace/src/appsec/api_security_sampler.js Outdated Show resolved Hide resolved

uurien requested a review from iunanua October 4, 2024 10:43

IlyasShabi force-pushed the api-security-sampling branch 2 times, most recently from f87c0d0 to 7d0ace0 Compare October 7, 2024 12:22

IlyasShabi requested a review from a team as a code owner October 8, 2024 07:59

simon-id reviewed Oct 8, 2024

View reviewed changes

docs/test.ts Show resolved Hide resolved

iunanua reviewed Oct 8, 2024

View reviewed changes

packages/dd-trace/src/appsec/api_security_sampler.js Outdated Show resolved Hide resolved

IlyasShabi force-pushed the api-security-sampling branch 6 times, most recently from a84bfce to f5c3bd3 Compare October 10, 2024 17:43

iunanua reviewed Oct 11, 2024

View reviewed changes

integration-tests/standalone-asm.spec.js Outdated Show resolved Hide resolved

uurien reviewed Oct 11, 2024

View reviewed changes

packages/dd-trace/src/config.js Show resolved Hide resolved

packages/dd-trace/src/appsec/api_security_sampler.js Outdated Show resolved Hide resolved

packages/dd-trace/src/appsec/api_security_sampler.js Outdated Show resolved Hide resolved

IlyasShabi force-pushed the api-security-sampling branch from f5c3bd3 to 3ecd200 Compare October 11, 2024 14:58

simon-id reviewed Oct 12, 2024

View reviewed changes

iunanua reviewed Oct 14, 2024

View reviewed changes

integration-tests/standalone-asm.spec.js Outdated Show resolved Hide resolved

Merge branch 'master' into api-security-sampling

2c4dbb5

simon-id reviewed Nov 7, 2024

View reviewed changes

packages/dd-trace/test/config.spec.js Outdated Show resolved Hide resolved

simon-id reviewed Nov 7, 2024

View reviewed changes

packages/dd-trace/test/config.spec.js Show resolved Hide resolved

simon-id reviewed Nov 7, 2024

View reviewed changes

packages/dd-trace/test/config.spec.js Outdated Show resolved Hide resolved

simon-id reviewed Nov 7, 2024

View reviewed changes

packages/dd-trace/test/appsec/index.express.plugin.spec.js Outdated Show resolved Hide resolved

IlyasShabi dismissed iunanua’s stale review via 56d45e9 November 12, 2024 09:29

IlyasShabi added 2 commits November 12, 2024 11:15

use route.path or url to generate the key

1146382

remove unnecessary tests of sample delay

fad6a65

IlyasShabi force-pushed the api-security-sampling branch from 56d45e9 to fad6a65 Compare November 12, 2024 10:15

fix non experimental options test

7f1898f

simon-id reviewed Nov 12, 2024

View reviewed changes

packages/dd-trace/src/appsec/api_security_sampler.js Outdated Show resolved Hide resolved

remove unused isSampled

12ac84c

always sample request if delay is 0

37d6aab

simon-id reviewed Nov 13, 2024

View reviewed changes

packages/dd-trace/src/appsec/api_security_sampler.js Show resolved Hide resolved

simon-id approved these changes Nov 13, 2024

View reviewed changes

IlyasShabi requested review from iunanua, uurien and CarlesDD November 13, 2024 15:49

IlyasShabi merged commit bdbeb02 into master Nov 18, 2024
214 checks passed

IlyasShabi deleted the api-security-sampling branch November 18, 2024 08:30

rochdev mentioned this pull request Nov 19, 2024

v4.50.0 proposal #4887

Merged

rochdev mentioned this pull request Nov 19, 2024

v5.26.0 proposal #4888

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add support to api security sampling #4755

add support to api security sampling #4755

IlyasShabi commented Oct 3, 2024

github-actions bot commented Oct 3, 2024 •

edited

Loading

pr-commenter bot commented Oct 3, 2024 •

edited

Loading

simon-id left a comment

simon-id commented Nov 7, 2024 •

edited

Loading

simon-id commented Nov 12, 2024

simon-id commented Nov 12, 2024

IlyasShabi commented Nov 13, 2024

add support to api security sampling #4755

add support to api security sampling #4755

Conversation

IlyasShabi commented Oct 3, 2024

What does this PR do?

Motivation

Plugin Checklist

github-actions bot commented Oct 3, 2024 • edited Loading

Overall package size

pr-commenter bot commented Oct 3, 2024 • edited Loading

Benchmarks

simon-id left a comment

Choose a reason for hiding this comment

simon-id commented Nov 7, 2024 • edited Loading

simon-id commented Nov 12, 2024

simon-id commented Nov 12, 2024

IlyasShabi commented Nov 13, 2024

github-actions bot commented Oct 3, 2024 •

edited

Loading

pr-commenter bot commented Oct 3, 2024 •

edited

Loading

simon-id commented Nov 7, 2024 •

edited

Loading