Skip to content

Commit

Permalink
fix: Net::HTTP instrumentation no-op on untraced context
Browse files Browse the repository at this point in the history
Update Net::HTTP instrumentation to no-op on HTTP requests within untraced
contexts. Prevents the logging of "called finish on an ended Span" messages.

Example scenario:

The Resque instrumentation can be configured to call
`OpenTelemetry::SDK::TraceProvider#force_flush` at the end of each job. That
causes the batch span processor to immediately send everything it has to the
OTLP exporter. The exporter then marks the HTTP request used to send the data
to the APM server as "untraced", by adding an `untraced` key to the context and
setting it to `true`. Before this commit, the `Net::HTTP` instrumentation did
not check for the `untraced` flag. So it would try set attributes and call
`finish` on a span that was not recording, creating one "called finish on an
ended Span" log entry for each batch of exported spans. On a large application,
that results in millions of those log entries per minute.
  • Loading branch information
brunofacca committed Nov 20, 2023
1 parent 0f90b20 commit 75a3aed
Show file tree
Hide file tree
Showing 2 changed files with 36 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -86,12 +86,20 @@ def tracer
end

def untraced?
untraced_context? || untraced_host?
end

def untraced_host?
return true if Net::HTTP::Instrumentation.instance.config[:untraced_hosts]&.any? do |host|
host.is_a?(Regexp) ? host.match?(@address) : host == @address
end

false
end

def untraced_context?
OpenTelemetry::Common::Utilities.untraced?
end
end
end
end
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,34 @@
_(span.attributes['net.peer.port']).must_equal(80)
end
end

describe 'untraced context' do
it 'no-ops on #request' do
# Calling `tracer.in_span` within an untraced context causes the logging of "called
# finish on an ended Span" messages. To avoid log noise, the instrumentation must
# no-op (i.e., not call `tracer.in_span`) when the context is untraced.
expect(instrumentation.tracer).not_to receive(:in_span)

OpenTelemetry::Common::Utilities.untraced do
Net::HTTP.get('example.com', '/body')
end

_(exporter.finished_spans.size).must_equal 0
end

it 'no-ops on #connect' do
expect(instrumentation.tracer).not_to receive(:in_span)

OpenTelemetry::Common::Utilities.untraced do
uri = URI.parse('http://example.com/body')
http = Net::HTTP.new(uri.host, uri.port)
http.send(:connect)
http.send(:do_finish)
end

_(exporter.finished_spans.size).must_equal 0
end
end
end

describe '#connect' do
Expand Down

0 comments on commit 75a3aed

Please sign in to comment.