Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Span Link support for compound header tag extractions with conflicting trace IDs #2948

Merged
merged 32 commits into from
Dec 9, 2024

Conversation

mhlidd
Copy link
Contributor

@mhlidd mhlidd commented Oct 24, 2024

What does this PR do?

Adds support for span links when trace propagators extract headers with conflicting trace IDs.

  • Under the above condition, the (*chainedPropagator).Extract function now appends spanLinks to the returned SpanContext
  • All integrations that invoke tracer.Extract now check for spanLinks on the returned SpanContext; if spanLinks is not nil, pass tracer.WithSpanLinks as a StartSpanOption.

Motivation

Make distrubuted tracing with span links consistent across Datadog libraries according the following RFC.

APMAPI-762

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

Unsure? Have a question? Request a review!

@pr-commenter
Copy link

pr-commenter bot commented Oct 24, 2024

Benchmarks

Benchmark execution time: 2024-12-09 12:43:26

Comparing candidate commit 08f578a in PR branch mhlidd/tracecontext_span_links with baseline commit 79a1f60 in branch main.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 57 metrics, 0 unstable metrics.

scenario:BenchmarkSetTagMetric-24

  • 🟥 execution_time [+4.934ns; +7.986ns] or [+4.147%; +6.713%]

scenario:BenchmarkSetTagStringer-24

  • 🟥 execution_time [+3.249ns; +9.551ns] or [+2.315%; +6.805%]

@mhlidd mhlidd force-pushed the mhlidd/tracecontext_span_links branch from 3a74a18 to b6e05e6 Compare October 25, 2024 18:42
ddtrace/tracer/option.go Outdated Show resolved Hide resolved
@mhlidd mhlidd requested review from mtoffl01 and darccio October 31, 2024 20:35
ddtrace/tracer/spancontext.go Outdated Show resolved Hide resolved
ddtrace/tracer/stats_payload_msgp.go Outdated Show resolved Hide resolved
@mhlidd mhlidd force-pushed the mhlidd/tracecontext_span_links branch from 0aec80a to 4f7ef39 Compare November 4, 2024 19:26
Copy link
Contributor

@mtoffl01 mtoffl01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting various changes, lmk if anything unclear!

ddtrace/tracer/span.go Show resolved Hide resolved
ddtrace/tracer/spancontext.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
.vscode/launch.json Outdated Show resolved Hide resolved
contrib/IBM/sarama.v1/sarama.go Outdated Show resolved Hide resolved
Copy link
Contributor

@mtoffl01 mtoffl01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting various changes, lmk if anything unclear!

Copy link
Contributor

@mtoffl01 mtoffl01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly nits...
Just be sure all the existing extract tests pass, like in textmap_test.go etc.

ddtrace/tracer/context_test.go Outdated Show resolved Hide resolved
ddtrace/tracer/context_test.go Outdated Show resolved Hide resolved
contrib/IBM/sarama.v1/sarama.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap_test.go Outdated Show resolved Hide resolved
@mhlidd mhlidd requested a review from mtoffl01 November 7, 2024 21:05
@github-actions github-actions bot added the apm:ecosystem contrib/* related feature requests or bugs label Nov 7, 2024
@mhlidd mhlidd changed the title initial code for adding spanlinks for span context Adding Span Link creation due to compound header tag extractions for invalid traces Nov 7, 2024
@mhlidd mhlidd changed the title Adding Span Link creation due to compound header tag extractions for invalid traces Adding Span Link support for compound header tag extractions with invalid traces Nov 7, 2024
@mhlidd mhlidd marked this pull request as ready for review November 7, 2024 21:10
@mhlidd mhlidd requested review from a team as code owners November 7, 2024 21:10
@mhlidd mhlidd requested a review from darccio November 7, 2024 21:11
contrib/IBM/sarama.v1/sarama.go Outdated Show resolved Hide resolved
ddtrace/ddtrace.go Outdated Show resolved Hide resolved
ddtrace/ddtrace.go Outdated Show resolved Hide resolved
ddtrace/tracer/context_test.go Outdated Show resolved Hide resolved
ddtrace/tracer/context_test.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap_test.go Outdated Show resolved Hide resolved
@mhlidd mhlidd force-pushed the mhlidd/tracecontext_span_links branch from 18a3541 to 54ad062 Compare November 8, 2024 20:02
@mhlidd mhlidd force-pushed the mhlidd/tracecontext_span_links branch from 54ad062 to ec2d4f1 Compare November 11, 2024 19:50
}
} else { // Trace IDs do not match - create span links
link := ddtrace.SpanLink{TraceID: extractedCtx2.TraceID(), SpanID: extractedCtx2.SpanID(), TraceIDHigh: extractedCtx2.TraceIDUpper(), Attributes: map[string]string{"reason": "terminated_context", "context_headers": "tracecontext"}}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mtoffl01 FYI the context_headers field is supposed to hold whichever propagator type was used to extract the violating context. For instance, if the propagator type was Datadog and we have violating trace IDs, we want context_headers to have the value of datadog

@mtoffl01 mtoffl01 marked this pull request as ready for review November 21, 2024 18:05
@mtoffl01 mtoffl01 self-requested a review November 21, 2024 18:05
Copy link
Contributor

@mtoffl01 mtoffl01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mhlidd , make sure you update the expected "context_headers" in TestSpanLinks::Links_on_divergent_trace_IDs test!

Copy link
Contributor Author

@mhlidd mhlidd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few small comments!

ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
ddtrace/tracer/textmap.go Outdated Show resolved Hide resolved
Comment on lines 288 to 290
extractedCtx2, ok1 := extractedCtx.(*spanContext)
ctx2, ok2 := ctx.(*spanContext)
// If we can't cast to SpanContextW3C, we can't propgate tracestate or create span links
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for casting to spanContext struct? We had conversation in the past to use SpanContextW3C instead of spanContext so I'm wondering what the motivation to change it back is.

PS: Should update the comment below if the decision is to cast to spanContext

Copy link
Contributor

@mtoffl01 mtoffl01 Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To my memory, I think the cast to SpanContextW3C was never even used except to further cast to spanContext.
(Like for example, pW3C.propagateTracestate requires spanContext type)

I do remember that casting directly to spanContext made this code a lot easier to read.

@mtoffl01 mtoffl01 changed the title Adding Span Link support for compound header tag extractions with invalid traces Adding Span Link support for compound header tag extractions with conflicting trace IDs Nov 25, 2024
Copy link
Contributor Author

@mhlidd mhlidd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't approve my own PR but LGTM! The flow is a lot clearer, thanks for helping out!

@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Dec 5, 2024

Datadog Report

Branch report: mhlidd/tracecontext_span_links
Commit report: fad2f96
Test service: dd-trace-go

✅ 0 Failed, 5100 Passed, 67 Skipped, 2m 54.86s Total Time

Copy link
Member

@darccio darccio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mtoffl01 mtoffl01 enabled auto-merge (squash) December 9, 2024 16:15
@mtoffl01 mtoffl01 merged commit abfec9a into main Dec 9, 2024
158 of 160 checks passed
@mtoffl01 mtoffl01 deleted the mhlidd/tracecontext_span_links branch December 9, 2024 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
apm:ecosystem contrib/* related feature requests or bugs do-not-merge/WIP
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants