v1.70.1

Fix the version number reported by dd-trace-go to avoid the release candidate label and add a fix for internal/civisiblity.

Please refer to v1.70.0 release notes for more detail.

Full Changelog: v1.70.0...v1.70.1

v1.70.0

This release includes ASM Exploit Prevention's General Availability and offers multiple fixes in contribs, improved client-side stats implementation, and multiple environment variables for configuration:

• DD_PROFILING_FLUSH_ON_EXIT: if set to 1, the profiler will upload the profiles in progress when profiler.Stop is called. Be mindful of using this setting for short-lived programs (e.g. lambdas, which we do not currently support for Go) as it may lead to inflated host counts. Also note that stopping the CPU profiler takes 200ms. See https://go.dev/issue/63043.
• DD_PROFILING_ENABLED: if set to false, then calling profiler.Start will not enable profiling. Note that setting this to true is not sufficient to enable profiling; you still need to call profiler.Start.
• DD_TRACE_LOG_DIRECTORY: it allows specifying a log directory for tracer logs (details).
• DD_APPSEC_RASP_ENABLED: if set to false, disables ASM Exploit Prevention (defaults to true)

What's Changed

Application Performance Monitoring (APM)

• contrib/segmentio/kafka.go.v0: refactor tracing code by @rarguelloF in #2885
• contrib/slog: clone record before calling Add by @felixge in #2929
• contrib/confluentinc/confluent-kafka-go: fix goroutine leak in Produce by @rarguelloF in #2924
• contrib/jackc/pgx.v5: wrap previous tracer by @rarguelloF in #2932
• contrib/net/http: refactor tracing by @rarguelloF in #2921
• contrib/confluentinc/confluent-kafka-go: split tracing code by @rarguelloF in #2907
• [godfathering] contrib/dimfeld/httptreemux.v5: failing tests for path variable replacement by @darccio in #2938
• [serverless] Inject trace context into SQS/SNS/EventBridge by @nhulston in #2917
• contrib/log/slog: fix WithAttrs and WithGroup implementation by @rarguelloF in #2857
• [SVLS-5560] Inject DD trace context into AWS Step Functions input by @DylanLovesCoffee in #2942
• APMSP-1241 Directly import trace-agent stats code for client-side stats by @ajgajg1134 in #2817
• [fix][internal/httptrace]: integration-level error codes override global by @mtoffl01 in #2946
• contrib/valyala/fasthttp.v1: fix memory leak of spanOpts by @0angelic0 in #2962
• contrib/go-chi: Apply DD_TRACE_HTTP_SERVER_ERROR_STATUSES by @mtoffl01 in #2960
• [fix][tracer] DD_TRACE_HEADER_TAGS treats trailing colon as invalid input by @mtoffl01 in #2913
• Fix: Support custom propagators in startup log by @mtoffl01 in #2925
• fix(options): Don't override c.httpClient if it is set via the options by @BaptisteFoy in #2970

Application Security Management (ASM)

We’re pleased to release Exploit Prevention (aka Run-time Application Self-Protection (RASP)) to protect your Go services against exploits of SQL injections (SQLi), Server-Side Request Forgeries (SSRF) and Local File Inclusion (LFI) vulnerabilities.
We recommend leveraging orchestrion, our new automatic Go instrumentation tool, to benefit from this new type of application security monitoring automatically. Note that some of those features are exclusive to orchestrion, such as LFI or the upcoming Command Injection (CMDi), which is coming later this quarter.
Please refer to the documentation or our blog post for more information.

• appsec: enable SQLi and SSRF exploit preventions by default in monitoring-only mode by @eliottness in #2952
• appsec: add tracer start option for appsec enablement by @RomainMuller in #2966
• appsec: differentiate user login and user set event by @eliottness in #2956

CI Visibility

• internal/civisibility: add early flake detection feature by @tonyredondo in #2916
• internal/civisibility: adds git tree upload feature by @tonyredondo in #2927
• internal/civisibility: add handshake linkname to orchestrion api to address linkname lock down by @tonyredondo in #2934
• internal/civisibility: add more nil checks to increase resilience by @tonyredondo in #2944
• internal/civisibility: auto test retries max retries fix by @tonyredondo in #2947
• internal/civisibility: intelligent test runner support by @tonyredondo in #2943
• internal/civisibility: test with efd enabled disable atr for that test by @tonyredondo in #2958
• internal/civisibility: add support for unskippable tests and suites by @tonyredondo in #2957
• internal/civisibility: api refactor and support for telemetry metrics by @tonyredondo in #2963
• Add CI Visibility data to client stats (APMSP-1241) by @ajgajg1134 in #2969

Profiling

• profiler: add enable flag to control profiler activation by @korECM in #2840
• profiler: add DD_PROFILING_FLUSH_ON_EXIT to upload current profiles before exiting by @jinroh in #2926

General

• Implement DD_TRACE_LOG_DIRECTORY by @mtoffl01 in #2901
• internal/osinfo: setup for kernel info by @eliottness in #2933
• fix and cleanup the Content-Length handling in transport by @paulcacheux in #2954
• ddtrace/tracer: support reading statsd port from tracer by @hannahkm in #2931
• ddtrace/tracer: fix trace_agent_url on startup log by @rachelyangdog in #2949

New Contributors

• @nhulston made their first contribution in #2917
• @korECM made their first contribution in #2840
• @DylanLovesCoffee made their first contribution in #2942
• @paulcacheux made their first contribution in #2954
• @0angelic0 made their first contribution in #2962
• @jinroh made their first contribution in #2926
• @rachelyangdog made their first contribution in #2949

Full Changelog: v1.69.1...v1.70.0

v1.69.1

Summary

This patch release introduces multiple bug fixes and refactors mainly related to contribs for integrating dd-trace-go better with Orchestrion. More details below.

Changes

Application Performance Monitoring (APM)

• contrib/confluentinc/confluent-kafka-go: split tracing code #2907
• contrib/confluentinc/confluent-kafka-go: fix goroutine leak in Produce #2924
• contrib/dimfeld/httptreemux.v5: failing tests for path variable replacement #2938
• contrib/jackc/pgx.v5: wrap previous tracer #2932
• contrib/log/slog: fix WithAttrs and WithGroup implementation #2857
• contrib/net/http: refactor tracing #2921
• contrib/segmentio/kafka.go.v0: refactor tracing code #2885
• contrib/slog: clone record before calling Add #2929
• [fix][tracer] DD_TRACE_HEADER_TAGS treats trailing colon as invalid input #2913
• Fix: Support custom propagators in startup log #2925

Full Changelog: v1.69.0...v1.69.1

v1.69.0

This release offers a lot of QOL changes and fixes for DataStreams Monitoring (DSM) and for two contribs: 99designs/gqlgen and aws/aws-sdk-go-v2/. Application Security now offers new attacker fingerprinting capabilities and support for entire read-only filesystem on linux

Important

If you are using gopkg.in/DataDog/dd-trace-go.v1/contrib/net/http.WrapRoundTripper, you can expect new errors to be reported in the span generated by your outgoing HTTP requests. The behaviour expected across all tracers was to report status codes between 400 and 500 as errors. Meanwhile dd-trace-go was reporting errors when the status code was between 500 and 600, this is now fixed. To revert to the previous behaviour. Please use the following environement variable: DD_TRACE_HTTP_CLIENT_ERROR_STATUSES=500-600

What's Changed

Application Performance Monitoring (APM)

• contrib/aws/aws-sdk-go-v2/aws: Fix streamName nil pointer panic by @webdestroya in #2846
• contrib/99designs/gqlgen/tracer.go: nil check response by @dienvoandpadcojp in #2792
• Fix and Feature: Make configuration options consistent across tracing libraries: @mtoffl01 in #2873
  • cf-connecting-ipv6 replaces cf-connecting-ip in HTTP headers tracked for http.client_ip tag when DD_TRACE_CLIENT_IP_ENABLED is set to true
  • DD_TRACE_HTTP_CLIENT_TAG_QUERY_STRING: Boolean that determines whether or not to include query params in http.url tag for http client spans
  • DD_TRACE_HTTP_CLIENT_ERROR_STATUSES: Customize errors on http client spans
  • DD_TRACE_HTTP_SERVER_ERROR_STATUSES: Customize errors on http server spans

Application Security Management (ASM)

• appsec: use memfd on linux instead of dumping libddwaf.so in /tmp by @eliottness in DataDog/go-libddwaf#106
• appsec: Suspicious Attacker Blocking by @eliottness in #2878
• appsec: Attacker Fingerprinting by @eliottness in #2899
• internal/appsec: upgrade default WAF ruleset for fingerprinting and RASP rules by @eliottness in DataDog/appsec-internal-go#33
• internal/appsec: support for _dd.appsec.rasp.* metrics by @eliottness in DataDog/go-libddwaf#109
• internal/appsec: refactor listeners by @eliottness in #2862

Profiling

• profiler: avoid metrics profile log noise when stopping profiling by @nsrip-dd in #2865

CI Visibility

• internal/civisibility/utils/net: a new http client for rapid endpoints by @tonyredondo in #2869
• internal/civisibility: adds the new vcpu_count metric for tslv events by @tonyredondo in #2883
• internal/civisibility: change the default value for DD_CIVISIBILITY_ENABLED to false by @tonyredondo in #2893
• internal/civisibility/integrations/gotesting: add the DD_CIVISIBILITY_ENABLED kill switch environment variable by @tonyredondo in #2861
• internal/civisibility: test session logical names by @tonyredondo in #2904
• internal/civisibility: automatic test retries by @tonyredondo in #2892
• internal/civisibility: fixes the test parent status when the auto-retry feature ended up with a failed test and then a successful test. by @tonyredondo in #2910
• internal/civisibility: add support for the test.source.end tag by @tonyredondo in #2911
• internal/civisibility: add and improve debug logs by @tonyredondo in #2912
• internal/civisibility/integrations/gotesting: fixes for orchestrion autoinstrumentation by @tonyredondo in #2844
• internal/civisibility/integrations/gotesting: improve the isCiVisibilityEnabled and enable reporting Code Coverage percentage. by @tonyredondo in #2866

General

• internal/stacktrace: upgrade tinylib/msgp and add stacktrace.GetSpanValue by @eliottness in #2864
• Regenerate stats payload to add is_trace_root by @ajgajg1134 in #2856
• ddtrace/tracer: update log msg to accurately count dropped traces by @hannahkm in #2845
• internal/datastreams: fix Processor goroutine leaks by @ggambetti in #2880

New Contributors

• @webdestroya made their first contribution in #2846
• @ggambetti made their first contribution in #2880
• @dienvoandpadcojp made their first contribution in #2792
• @fkmy made their first contribution in #2898

Full Changelog: v1.68.0...v1.69.0

v1.68.0

With the release of Go 1.23, we are dropping support for Go 1.21. Our minimum supported Go release is now Go 1.22.

This release removes the gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc.v12 integration, which wrapped grpc v0.12.0. This release also brings several bug fixes to the APM and ASM libraries.

What's Changed

Application Performance Monitoring (APM)

• contrib/gocql/gocql: implement observer api based tracing by @rarguelloF in #2805
• Drop support for ancient and unused grpc.v12 by @ajgajg1134 in #2820
• ddtrace/tracer: dereference pointers to supported types in span.SetTag by @darccio in #2796
• ddtrace/tracer: add uptrace/bun to telemetry's supported contribs by @darccio in #2810
• chore(w3c): ensure last parent id is never set to 16 zeros by @mabdinur in #2808
• ddtrace/tracer: add IsTraceRoot to clients-side-stats by @knusbaum in #2821
• ddtrace/tracer: fixed resampling to occur on root span only by @dianashevchenko in #2824
• Log regex compilation as error by @ALX99 in #2809

Application Security Management (ASM)

• internal/appsec: fix missing ResponseHeaderCopier call by @eliottness in #2841
• appsec: missing SQLi and LFI RC capabilities by @eliottness in #2851

General

• all: upgrade minimum supported Go release to 1.22 by @nsrip-dd in #2850
• go.mod: retract experimental versions by @darccio in #2806
• [CI Visibility] Refactor gotesting integration to be used by orchestrion by @tonyredondo in #2843
• internal/globalconfig: make statsTags un-changeable from outside of the globalconfig package by @mtoffl01 in #2842
• fix(ci): smoke tests due to broken SQLite driver & update upload-artifact action by @darccio in #2848

New Contributors

• @ALX99 made their first contribution in #2809

Full Changelog: v1.67.1...v1.68.0

v1.67.1

Summary

This patch release introduces two key changes to sampling:

• Resampling now occurs only on root spans, eliminating redundant sampling tags on child spans.
• Client stats have been updated to align with the Trace Agent version, enabling support for adaptive sampling.

Important

Depending on your sampling rules, and especially if you have trace sampling rules that match child spans, you may notice an increase in ingested spans. This increase is expected and a result of improved sampling accuracy. If the ingested volume is problematic, reduce it using APM Ingestion Controls. For any questions or issues, please contact Datadog Support.

Changes

Application Performance Monitoring (APM)

• ddtrace/tracer: add IsTraceRoot to clients-side-stats in #2821
• ddtrace/tracer: fixed resampling to occur on root span only in #2824

Full Changelog: v1.67.0...v1.67.1

v1.67.0

Summary

In this release, Application Performance Monitoring (APM) introduces optimized W3C context propagation, reduced lock contention on Git metadata tags and support for span events in OpenTelemetry. It improves multiple contribs, see below, and releases a new one for uptrace/bun. It also introduces multiple fixes, improves support for Orchestrion and deprecates hostname detection, now available behind a compatibility environment variable DD_TRACE_CLIENT_HOSTNAME_COMPAT=v1.66.0.

Contribs updated in this release:

• gocql/gocql: support for MapScanCAS and the Observer API, and bug fixed in cluster and datacenter tag values, old behaviour available behind compatibility environment variable DD_TRACE_GOCQL_COMPAT=v1.65.
• gorm.io/gorm.v1: adds parent span propagation.
• IBM/sarama.v1: support for DSM.
• net/http: support for rwUnwrapper interface.
• uptrace/bun: new contrib available.

Application Security Management (ASM) improves WAF run operations and upgrades go-libddwaf.

Changes

Application Performance Monitoring (APM)

• ddtrace/tracer: setup GLS-stored context by @eliottness in #2761
• tracer: allow x-datadog headers to pass as header tags by @mtoffl01 in #2744
• internal: cache build info to reduce lock contention in GetGitMetadataTags by @darccio in #2770
• ddtrace/tracer,profiler: share agent default URL resolution logic by @nsrip-dd in #2719
• ddtrace/tracer: put hostname detection behind a compat environment variable by @darccio in #2786
• ddtrace/tracer: optimize textmap.composeTracestate by @darccio in #2794
• ddtrace/tracer: noDebugStack config reports to telemetry under distinct key debug_stack_enabled by @mtoffl01 in #2798
• ddtracer/tracer: fix appending 'tracer_version' to the statsd tags by @nsrip-dd in #2801
• Mtoff/span events by @mtoffl01 in #2780
• ddtrace/mocktracer: fix SetUser & WithUserMetadata by @eliottness in #2787
• ddtrace/tracer: Add test to ensure global config inherits statsd address when configured with tracer.WithDogstatsdAddress by @mtoffl01 in #2683
• ddtrace/tracer: update the tag naming convention for otel env var telemetry metrics by @khanayan123 in #2760
• ddtrace/tracer: update the expected telemetry tags for OTel env-var mapping by @mcculls in #2797
• [contrib/gocql] adding support for MapScanCAS in gocql (cassandra) by @ufoot in #2752
• contrib/uptrace/bun: initial implementation by @darccio in #2771
• contrib/net/http: implement rwUnwrapper by @rarguelloF in #2775
• [contrib/gocql] reporting both cluster and datacenter by @ufoot in #2577
• contrib/IBM/sarama.v1: add DSM support by @rarguelloF in #2788
• contrib/gorm.io/gorm.v1: propagate parent span by @System-Glitch in #2759
• contrib/jackc/pgx.v5: acquire release by @ngavinsir in #2722
• contrib/jackc/pgx.v5: publish pgxpool stats to statsd client by @bengentree in #2692
• contrib/gocql/gocql: implement observer api based tracing (#2805)

Application Security Management (ASM)

• internal/appsec: refactor simple WAF run operations by @eliottness in #2776
• upgrade go-libddwaf & appsec-internal-go and remove occurences of go 1.20 by @eliottness in #2782

General

• build(deps): bump github.com/vektah/gqlparser/v2 from 2.5.8 to 2.5.15 by @dependabot in #2755
• build(deps): bump github.com/gofiber/fiber/v2 from 2.52.1 to 2.52.5 by @dependabot in #2767
• build(deps): bump github.com/hashicorp/go-retryablehttp from 0.7.4 to 0.7.7 by @dependabot in #2757
• internal/exectracetest: fix TestExecutionTraceSpans flakes by @nsrip-dd in #2769
• Add datadog static analyzer by @juli1 in #2741
• ci: upgrade golangci-lint 1.52.2 -> 1.59.1 by @eliottness in #2777
• Add instruction how to update Go version for macrobenchmarks by @ddyurchenko in #2793

New Contributors

• @juli1 made their first contribution in #2741
• @System-Glitch made their first contribution in #2759
• @ngavinsir made their first contribution in #2722
• @bengentree made their first contribution in #2692
• @mcculls made their first contribution in #2797

Full Changelog: v1.66.0...v1.67.0

v1.66.0

Summary

In this release, Application Performance Monitoring (APM) adds support for log/slog instrumentation.
And Application Security Management (ASM) now monitors protects SQL queries performed via database/sql against SQL injections.

This release aligns our Go support policy with the official policy of the Go project, brings several bug fixes to the tracer, contrib packages and appsec.

Changes

Application Performance Monitoring (APM)

• ddtrace/tracer: add propagation style info to the startup logs by @khanayan123 in #2748
• contrib: add log/slog instrumentation by @rarguelloF in #2765
• ddtrace/opentelemetry: Fix http remapping to account for int values by @zarirhamza in #2737
• ddtrace/tracer: update sample rate configuration parsing to avoid logging a warning when empty by @zacharycmontoya in #2751
• contrib/99designs/gqlgen: additional options to reduce noise by @samsullivan in #2695
• ddtracer/tracer: fix appending 'tracer_version' to the statsd tags by @dianashevchenko in #2784

Application Security Management (ASM)

• appsec: support SQLi Exploit Prevention (RASP) by @Hellzy in #2730

General

• datastreams: Remove one colon check for well-formed edge tags by @juliannzhou in #2729
• internal/telemetry: don't send app-dependencies-loaded in the absence of Go deps by @Julio-Guerra in #2740
• readme: drop go 1.19 from support table by @eliottness in #2758
• all: align Go support with upstream policy by @nsrip-dd in #2705
• internal/datastreams: Always try sending data stream stats by @piochelepiotr in #2756
• .gitlab/macrobenchmarks: Upgrade macrobenchmarks setup, add CPU / RSS utilization overhead measurements by @ddyurchenko in #2763
• .github/workflows: run smoke tests on latest Go 1.23 release candidate by @darccio in #2764

New Contributors

• @zacharycmontoya made their first contribution in #2751
• @khanayan123 made their first contribution in #2748

Full Changelog: v1.65.1...v1.66.0

v1.65.1

Summary

This patch release provide two small fixes:

• Workaround of a bug found in go 1.20 that concerns all users of the appsec/events.IsSecurityError() that made the function always return false. Seeing spans in the DataDog's UI containing errors with the following message can also be a symptom of this issue: request blocked by WAF
• Removal of a unsolicitated warning log in the form of: ignoring DD_TRACE_SAMPLE_RATE, error: strconv.ParseFloat: parsing "": invalid syntax

Changes

Application Performance Monitoring (APM)

• ddtrace/tracer: update sample rate configuration parsing to avoid logging a warning when empty in #2751

Application Security Management (ASM)

• appsec: fix IsSecurityError in #2746

Full Changelog: v1.65.0...v1.65.1

v1.65.0

Summary

In this release, Application Performance Monitoring (APM) improved trace and span IDs generation using math/rand/v2 for Go 1.22 and later. APM introduced support for Go 1.22 ServeMux patterns and OTel env vars and better OTel HTTP status code remapping. Also, APM fixed a race condition when using spans concurrently and how dropped traces were reported to the agent.

Application Security Management (ASM) introduces its new "Exploit Prevention" in public beta, a new type of in-app security monitoring that detects and blocks vulnerability exploits. This release includes Exploit Prevention of Server-Side Request Forgeries (SSRF) for HTTP client requests.

Data Streams Monitoring (DSM) updates sketch mapping to improve accuracy and moves to google.golang.org/protobuf.

And last, multiple dependencies have been upgraded to fixed versions for reported vulnerabilities and to reduce our dependency on github.com/golang/protobuf.

Changes

Application Performance Monitoring (APM)

• go.mod: update github.com/DataDog/sketches-go to v1.4.5 by @darccio in #2698
• contrib/database/sql: allow peer.service to be set explicitly and add into propagated context by @tabgok in #2679
• contrib/net/http: Support go1.22 ServeMux patterns by @felixge in #2716
• ddtrace/tracer: use math/rand/v2 for SpanID and TraceID by @knusbaum in #2689
• ddtrace/tracer: fix concurrent map writes when applying trace sampling rules and setting tags concurrently by @darccio in #2727
• w3c: ensure _dd.parent_id is set using x-datadog-parent-id by @mabdinur in #2696
• ddtrace/tracer: increment droppedP0Traces iff it's not going to be sent by @darccio in #2713
• ddtrace/tracer: support default origin on dynamic config to support Active Tracing telemetry spec by @darccio in #2623
• ddtrace/tracer: only finish execution trace task, restore pprof labels once by @nsrip-dd in #2708
• ddtrace/tracer: add support for OTEL env vars by @mtoffl01 in #2715
• ddtrace/tracer: report trace and span sampling rules separately by @darccio in #2718
• ddtrace/opentelemetry: adds http status code remapping for otel trace metrics by @zarirhamza in #2728

Application Security Management (ASM)

• appsec: support for SSRF Exploit Prevention by @Hellzy in #2707
• appsec/grpc: fix rpc message blocking by @Julio-Guerra in #2723

Data Streams Monitoring

• Move to google.golang.org/protobuf by @piochelepiotr in #2694
• datastreams: Update mapping by @piochelepiotr in #2703
• [DSMS-19] Fix SNS topic format issue by @juliannzhou in #2725

General

• internal: stacktrace: skip internal packages by @eliottness in #2697
• go.mod: upgrade github.com/mattn/go-sqlite3 up to v1.14.18 (#2693) by @darccio in #2714
• go.mod: upgrade github.com/go-jose/go-jose/v3 to v3.0.3 by @darccio in #2700
• go.mod: upgrade google.golang.org/protobuf to v1.33.0 by @darccio in #2699
• go.mod: upgrade richardartoul/molecule by @darccio in #2724

New Contributors

• @juliannzhou made their first contribution in #2725

Full Changelog: v1.64.0...v1.65.0