Releases: DataDog/datadog-lambda-extension
v67
Warning
BREAKING CHANGE
This version sets the Next Generation Datadog Lambda Extension as default, to force launch of the compatibility
(previous) Extension, set the DD_EXTENSION_VERSION=compatibility
The Next-Gen Datadog Lambda Extension is designed specifically for Lambda and makes different tradeoffs around the lifecycle and processing of data than the original, converted datadog-agent
. While we expect the features and configuration to be compatible, there are timing changes to when data flushes, or aggregation changes which are best suited to the Lambda lifecycle. As a result, this is a major version change and we strongly recommend testing this release in dev/staging/pre-production environment before moving to production.
Read about the Next Generation Datadog Lambda Extension.
Important
Compatibility (previous) Extension Agent
version is v7.59.0
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:67
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:67
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:67
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:67
What's Changed
Next Generation Extension:
- rename failover to fallback by @alexgallotta in #465
- feat(bottlecap): add
runtime
tag by @duncanista in #464 - feat(bottlecap): handle timeouts by @duncanista in #466
- feat(bottlecap): handle OOM exceptions by @duncanista in #467
- fix(bottlecap): fallback when otel set by @duncanista in #470
- fix(bottlecap): fix
aws.lambda
service naming by @duncanista in #471 - fix(bottlecap): try kms decrypt without context as in goagent by @alexgallotta in #472
- feat(bottlecap): fallback on opted out only by @duncanista in #473
- fix(bottlecap): fallback on yaml otel config by @duncanista in #474
- feat(bottlecap): honor serverless_logs by @astuyve in #475
- chore: update Go version in multiple places by @duncanista in #477
- fix(bottlecap): priority of decrypt: kms > arn > use dd_api_key by @alexgallotta in #478
- feat(bottlecap): Flush timeouts by @astuyve in #480
- fix(bottlecap): dogstatsd to filter invocations from lambda library by @duncanista in #481
- feat(bottlecap): support adaptive flush strategy again by @astuyve in #482
- feat(bottlecap): 1min default by @astuyve in #484
Compatibility (Previous) Extension:
- change env variable for bottlecap failover->fallback DataDog/datadog-agent@877d60d
- run remote config service in extension and start trace agent config endpoint DataDog/datadog-agent@53070e2
Full Changelog: v66...v67
v66
Important
Agent
version is v7.59.0
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:66
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:66
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:66
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:66
What's Changed
Main Extension:
- Move flags for stripping and scrubbing keys to agent func DataDog/datadog-agent@e1ea1f1
- Read 128 bit trace ID in End Invocation DataDog/datadog-agent@5cfbd26
- [Serverless] Reset both inferred spans on each invocation. DataDog/datadog-agent@9b6d767
- [serverless] Extract EventBridge from SNS/SQS Payloads and Create EventBridge Spans DataDog/datadog-agent@024af28
- Fix port collision in serverless daemon tests. DataDog/datadog-agent@231be18
- [serverless] Add EventBridge Inferred Spans DataDog/datadog-agent@be76ade
- Enhanced Metrics Fix DataDog/datadog-agent@0df804c
- [SVLS-5034] Create trace context from Step Function execution details DataDog/datadog-agent@a4534bc
- [SVLS-5551] Add lambda threads enhanced metrics DataDog/datadog-agent@e99d8db
Next Generation Extension:
Important
Customers using .NET, Java, Ruby, and Go can now opt-in to take advantage of the performance improvements.
- chore: decouple
hyper
fromtrace_processor
by @duncanista in #390 - fix: sync with fixed dogstatsd by @alexgallotta in #402
- feat: support HTTPS_PROXY by @astuyve in #381
- fix: add
ap-southeast-5
region by @duncanista in #411 - fix: display correct Agent version in Alpine builds by @nhulston in #422
- fix: aj/honor both proxies in order by @astuyve in #410
- fix: bump go version in alpine image to 1.23.2 by @duncanpharvey in #426
- chore: adding static flag to build parameters for alpine images by @TophrC-dd in #438
- feat(bottlecap): add Universal Instrumentation by @duncanista, @alexgallotta, @shreyamalpani, and @astuyve in #374
New Contributors
- @nhulston made their first contribution in #422
- @TophrC-dd made their first contribution in #438
Full Changelog: v65...v66
v65
What's Changed
Agent Tag
Agent Version: 7.57.0
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:65
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:65
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:65
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:65
Main Extension:
- [SVLS-5257] Add lambda file descriptor enhanced metrics DataDog/datadog-agent@0dea3e9
- [ASCII-2222] Fix serverless log level logic DataDog/datadog-agent@849a13c
- [SVLS-5268] Disable flush ticker for AWS Lambda DataDog/datadog-agent@381e420
Next generation Datadog Extension:
- chore(bottlecap): fast failover by @duncanista in #371
- aj/fix log level casing by @astuyve in #372
- chore(bottlecap): update dev image by @duncanista in #375
- feat: break if we shouldn't wait for the end by @astuyve in #377
- feat: increase to 10k contexts by @astuyve in #380
- Use upstream dogtstatsd by @alexgallotta in #368
- fix: licenses by @astuyve in #386
- feat: Specify aws_lc_rs as our default crypto provider by @astuyve in #385
- feat: Remove unused deps, drop minor version pins and update by @astuyve in #387
- feat: failover on dd proxy by @astuyve in #391
Full Changelog: v64...v65
New Contributors
- @ichinaski made their first contribution in #379
v64
What's Changed
Agent Tag
Agent Version: 7.55.3
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:64
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:64
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:64
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:64
Main Extension:
- fix: get lambda runtime API endpoint dynamically DataDog/datadog-agent@1d07d29
Next generation Datadog Extension:
- feat: Use stable instead of nightly by @astuyve in #355
- feat: Allowlist additional env vars by @astuyve in #354
- chore(bottlecap): reduce allocations for logs processing by @duncanista in #345
- aj/allow apm replace tags array by @astuyve in #358
- fix: update docs by @astuyve in #360
- fix(bottlecap): set explicit deny list and allow yaml usage by @duncanista in #363
- Modularize dogstatsd by @alexgallotta in #361
Full Changelog: v63...v64
v63
What's Changed
Agent Tag
Agent Version: 7.55.3
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:63
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:63
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:63
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:63
Datadog Agent:
-
[Bugfix] [SLES-1749] histogram_aggregates and histogram_percentiles added to serverless in… DataDog/datadog-agent@94c9e78
-
[serverless] Add debugging log line for OOM substrings DataDog/datadog-agent@3b686ac
Rust:
- feat: bias main select by @astuyve in #339
- fix(bottlecap): update build images by @duncanista in #340
- fix(bottlecap): remove
nightly
in favor orstable
by @duncanista in #341 - feat: Remove excepts from secrets, drop into idle loop when local by @astuyve in #342
- feat: Open up more env vars which we don't rely on by @astuyve in #344
- Scan latest releases for vulnerabilities. by @purple4reina in #343
- feat: Allow trace disabled plugins by @astuyve in #348
Full Changelog: v62...v63
v62
What's Changed
⚠️ Container Images
A change was made a while ago in our documentation which requires to update which artifacts are copied from our datadog/lambda-extension
image. Ensure you modify it from the previous instrumentation to the new one. Starting v61
this is required to get your instrumentation working properly.
Expected:
# now
COPY --from=public.ecr.aws/datadog/lambda-extension:<TAG> /opt/. /opt/
Previous:
# previous
COPY --from=public.ecr.aws/datadog/lambda-extension:<TAG> /opt/extensions/ /opt/extensions
Agent Tag
Agent Version: 7.55.1
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:62
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:62
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:62
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:62
Datadog Agent:
- feat(serverless): temporary set dogstatsd socket to empty DataDog/datadog-agent@a5c3d2b
Rust:
- fix(bottlecap): remove log that might confuse customers by @duncanista in #333
- fix: image builder agent resolution by @duncanista in #332
- fix dogstatsd multiline by @alexgallotta in #335
- chore(bottlecap): add cold start environment variables by @duncanista in #337
Full Changelog: v61...v62
v61
⚠️ Container Images
A change was made a while ago in our documentation which requires to update which artifacts are copied from our datadog/lambda-extension
image. Ensure you modify it from the previous instrumentation to the new one. Since v61
requires this to work properly.
Expected:
# now
COPY --from=public.ecr.aws/datadog/lambda-extension:<TAG> /opt/. /opt/
Previous:
# previous
COPY --from=public.ecr.aws/datadog/lambda-extension:<TAG> /opt/extensions/ /opt/extensions
🚨 There is a known issue with this version. We recommend using v62
:
- An issue was encountered too when using container images, we published a faulty image to the
latest
,latest-alpine
,61
, and61-alpine
images. But this has been remediated and fixed. - This version reports the agent version as
6.0.0
instead of the expected. - An UDS error message appears on initialization, it's benign, but doesn't provide a great experience for customers.
Agent Tag
Agent Version: 7.55.1
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:61
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:61
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:61
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:61
What's Changed
🔔 This version was release off of the main branch, unlike previous releases where commits are cherry-picked. Below are Serverless specific changes:
Datadog Agent:
- [SVLS-5070][serverless] Fix Serverless logs request sync with flush DataDog/datadog-agent@6332366
- [SVLS-5001] Add tmp_used and tmp_max enhanced lambda metrics DataDog/datadog-agent@293e98e
- Support noop trace agent for serverless. DataDog/datadog-agent@6cdd940
- bump go-libddwaf to v3.3.0 DataDog/datadog-agent@946cd11
- feat(serverless): send bottlecap failover metric DataDog/datadog-agent@a0f8078
- [SVLS-4912] Generate CPU utilization metrics DataDog/datadog-agent@827a81b
- [SVLS-4936] Add lambda network enhanced metrics DataDog/datadog-agent@1cfabef
- [SVLS-4640][serverless] Fix dropped logs in AWS Lambda DataDog/datadog-agent@7e6dc75
- [SVLS-3697] Fix missing post runtime duration metrics and log values after a shutdown DataDog/datadog-agent@6e4f4f8
- Serverless Agent supports
stream-logs
DataDog/datadog-agent@4660cce
Rust:
- feat(bottlecap): warn on hello by @astuyve in #314
- feat(bottlecap): warn by default by @astuyve in #316
- chore(bottlecap): add
hyper@0.x
in telemetry api by @duncanista in #312 - chore(bottlecap: )add mock for integration test for metrics by @alexgallotta in #310
- fix(bottlecap): hyper refactor removed an error type by @alexgallotta in #321
- chore(documentation): Fix dead link for contributing.md by @lym953 in #322
- feat(bottlecap): multi-metric support by @astuyve in #317
- chore(bottlecap): add license by @alexgallotta in #320
- chore(bottlecap): fallback on
datadog.yaml
usage by @duncanista in #326 - feat(bottlecap): add
runtime
tag by @alexgallotta in #327 - feat(bottlecap): add
dd_extension_version
to tags by @duncanista in #328 - fix(bottlecap): filter debug logs from external crates by @duncanista in #329
- fix(bottlecap): use /proc/pid/environ to read the env variable for runtime by @alexgallotta in #330
- chore(bottlecap): switch flushing strategy to race by @astuyve in #318
and many more...
New Contributors
Full Changelog: v60...v61
v60
Agent Tag
Agent Version: 7.54.0
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:60
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:60
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:60
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:60
What's Changed
🔔 This version is a duplicate of v58.
-
Enable to set the buffering config of Lambda Logs API via env vars DataDog/datadog-agent@1eb4f52
-
[Serverless] Reuse DD_ENHANCED_METRICS for new system enhanced metrics DataDog/datadog-agent@62532f6
-
Improve logging in the Lambda Logs API handler DataDog/datadog-agent@e5b69ff
-
[SVLS-4850] Generate lambda enhanced CPU system, user, and total metrics DataDog/datadog-agent@fa5ba1b
-
[SVLS] Mark serverless unit test TestLoadConfigShouldBeFast as flaky DataDog/datadog-agent@ebe363c
-
Set _dd.p.dm trace field to null in Serverless Integration Tests Snapshots DataDog/datadog-agent@f20521b
-
[ASCII-1218] Only initialize needed configs for serverless DataDog/datadog-agent@09161cd
-
[SVLS-4563] Adds provided.al2023 as a runtime tag DataDog/datadog-agent@3b65821
-
[SVLS] Add AWS Account Id Fields to Credit Card Obfuscation Exclusion DataDog/datadog-agent@527cc4f
v59
🚨 There is a known issue with this version. We recommend using another version as we continue to investigate.
Agent Tag
Agent Version: 7.54.1
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:59
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:59
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:59
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:59
What's Changed
🔔 This version was release off of the main branch, unlike previous releases where commits are cherry-picked. Below are Serverless specific changes:
-
feat(serverless): send bottlecap failover metric DataDog/datadog-agent@a0f8078
-
[SVLS-4912] Generate CPU utilization metrics DataDog/datadog-agent@827a81b
-
[serverless-build-workflow] Compare with merge-base instead of just base ref DataDog/datadog-agent@b950f74
-
[SVLS-4936] Add lambda network enhanced metrics DataDog/datadog-agent@1cfabef
-
[SVLS-4640][serverless] Fix dropped logs in AWS Lambda DataDog/datadog-agent@7e6dc75
-
[Serverless] Maintain serverless PR comments independently. DataDog/datadog-agent@4683873
-
[Serverless] Post binary size change when serverless deps file changes. DataDog/datadog-agent@f0722c6
-
Move serverless vuln scan to extension repo. DataDog/datadog-agent@41f10b8
-
[SVLS-3697] Fix missing post runtime duration metrics and log values after a shutdown DataDog/datadog-agent@6e4f4f8
-
Serverless Agent supports
stream-logs
DataDog/datadog-agent@4660cce
v58
Agent Tag
Agent Version: 7.54.0
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:58
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:58
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:58
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:58
What's Changed
🔔 This version was release off of the main branch, unlike previous releases where commits are cherry-picked. Below are Serverless specific changes:
-
Enable to set the buffering config of Lambda Logs API via env vars DataDog/datadog-agent@1eb4f52
-
[Serverless] Reuse DD_ENHANCED_METRICS for new system enhanced metrics DataDog/datadog-agent@62532f6
-
Improve logging in the Lambda Logs API handler DataDog/datadog-agent@e5b69ff
-
[SVLS-4850] Generate lambda enhanced CPU system, user, and total metrics DataDog/datadog-agent@fa5ba1b
-
[SVLS] Mark serverless unit test TestLoadConfigShouldBeFast as flaky DataDog/datadog-agent@ebe363c
-
Set _dd.p.dm trace field to null in Serverless Integration Tests Snapshots DataDog/datadog-agent@f20521b
-
[ASCII-1218] Only initialize needed configs for serverless DataDog/datadog-agent@09161cd
-
[SVLS-4563] Adds provided.al2023 as a runtime tag DataDog/datadog-agent@3b65821
-
[SVLS] Add AWS Account Id Fields to Credit Card Obfuscation Exclusion DataDog/datadog-agent@527cc4f