diff --git a/README.md b/README.md index a8c94509..095dff14 100644 --- a/README.md +++ b/README.md @@ -113,63 +113,6 @@ In addition to this, Orchestrion only supports projects using [Go modules][go-mo > $ go get github.com/datadog/orchestrion@latest > ``` -## How it works - -The go toolchain's `-toolexec` feature invokes `orchestrion toolexec` with the complete list of arguments for each -toolchain command invocation, allowing `orchestrion` to inspect and modify those before executing the actual command. -This allows `orchestrion` to inspect all the go source files that contribute to the complete application, and to modify -these to include instrumentation code where appropriate. Orchestrion uses [`dave/dst`][dave-dst] to parse and modify the -go source code. Orchestrion adds `//line` directive comments in modified source files to make sure the stack trace -information produced by the final application are not affected by additional code added during instrumentation. - -Since the instrumentation may use packages not present in the original code, `orchestrion` also intercepts the standard -go linker command invocations to make the relevant packages available to the linker. - -[dave-dst]: https://github.com/dave/dst - -### Directive comments - -Directive comments are special single-line comments with no space between then `//` and the directive name. These allow -influencing the behavior of Orchestrion in a declarative manner. - -#### `//dd:ignore` - -The `//dd:ignore` directive instructs Orchestrion not to perform any code changes in Go code nested in the decorated -scope: when applied to a statement, it prevents instrumentations from being added to any component of this statement, -and when applied to a block, or function, it prevents instrumentation from being added anywhere in that block or -function. - -This is useful when you specifically want to opt out of instrumenting certain parts of your code, either because it has -already been instrumented manually, or because the tracing is undesirable (not useful, adds too much overhead in a -performance-critical section, etc...). - -#### `//dd:span` - -Use a `//dd:span` comment before any function or method to create specific spans from your automatically instrumented -code. Spans will include tags described as arguments in the `//dd:span`. In order for the directive to be recognized, -the line-comment must be spelled out with no white space after the `//` comment start. - -A function or method annotated with `//dd:span` must receive an argument of type `context.Context` or `*http.Request`. -The context or request is required for trace information to be passed through function calls in a Go program. If this -condition is met, the `//dd:span` comment is scanned and code is inserted in the function preamble (before any other -code). - -Span tags are specified as a space-delimited series of `name:value` pairs, or as simple expressions referring to -argument names (or access to fields thereof). All `name:value` pairs are provided as strings, and expressions are -expected to evaluate to strings as well. - -```go -//dd:span my:tag type:request name req.Method -func HandleRequest(name string, req *http.Request) { - // ↓↓↓↓ Instrumentation added by Orchestrion ↓↓↓↓ - req = req.WithContext(instrument.Report(req.Context(), event.EventStart, "function-name", "HandleRequest", "my", "tag", "type", "request", "name", name, "req.Method", req.Method)) - defer instrument.Report(req.Context(), event.EventEnd, "function-name", "HandleRequest", "my", "tag", "type", "request", "name", name, "req.Method", req.Method) - // ↑↑↑↑ End of added instrumentation ↑↑↑↑ - - // your code here -} -``` - ## Supported libraries Orchestrion supports automatic tracing of the following libraries: @@ -235,3 +178,12 @@ You can inspect everything Orchestrion is doing by adding the `-work` argument t the build will emit a `WORK=` line pointing to a working directory that is retained after the build is finished. The contents of this directory contains all updated source code Orchestrion produced and additional metadata that can help diagnosing issues. + +## More information + +Orchestrion's documentation can be found at [datadoghq.dev/orchestrion](https://datadoghq.dev/orchestrion); in +particular: +- the [user guide](https://datadoghq.dev/orchestrion/docs/) provides information about available configuration, and how + to customize the traces produced by your application; +- the [contributor guide](https://datadoghq.dev/orchestrion/contributing/) provides more detailed information about how + orchestrion works and how to contribute new instrumentation to it. diff --git a/docs/config/_default/hugo.toml b/docs/config/_default/hugo.toml index ec511f5b..33fe1e97 100644 --- a/docs/config/_default/hugo.toml +++ b/docs/config/_default/hugo.toml @@ -13,7 +13,7 @@ unsafe = true noClasses = false [[menu.main]] -name = 'User Documentation' +name = 'User Guide' pageRef = '/docs' weight = 2 diff --git a/docs/content/docs/built-in/_index.md b/docs/content/docs/built-in/_index.md index d6ff1997..4c09d7c4 100644 --- a/docs/content/docs/built-in/_index.md +++ b/docs/content/docs/built-in/_index.md @@ -1,6 +1,6 @@ --- title: Built-in Configuration -weight: 10 +weight: 80 prev: /docs/getting-started next: /docs/troubleshooting --- diff --git a/docs/content/docs/custom-trace.md b/docs/content/docs/custom-trace.md new file mode 100644 index 00000000..1689ec41 --- /dev/null +++ b/docs/content/docs/custom-trace.md @@ -0,0 +1,183 @@ +--- +title: Trace Customization +weight: 20 +prev: /docs/features +next: /docs/built-in +--- + +Orchestrion offers serveral ways to control the traces produced by instrumented +applications. + +## Prevent instrumentation of a section of code + +By default, `orchestrion` injects instrumentation _everywhere_ possible. This +ensures users get the maximum possible coverage from their applications, as it +removes the possibility of someone forgetting to instrument a particular call. + +There are however cases where you may want specific sections of your application +to not be instrumented, either because they result in excessively verbose +traces, or because those trace spans would be duplicated (for example, when +using a custom-made `net/http` middleware stack). + +The `//dd:ignore` directive can be added anywhere in your application's code, +and will disable all `orchestrion` instrumentation in the annotated syntax tree. + +For example: + +```go +package demo + +import "net/http" + +//dd:ignore I don't want any of this to be instrumented, ever. +func noInstrumentationThere() { + // Orchestrion will never add or modify any code in this function + // ... etc ... +} + +func definitelyInstrumented() { + // Orchestrion may add or modify code in this function + // ... etc ... + + //dd:ignore This particular server will NOT be instrumented + server := &http.Server { + Addr: "127.0.0.1:8080", + Handler: internalServerHandler, + } + + // Orchestrion may add or modify code further down in this function + // ... etc ... +} +``` + +{{}} +In certain cases, `orchestrion` adds instrumentation on the library side +(sometimes referred to as _callee_ instrumentation; as opposed to _call site_ +instrumentation). + +In such cases, it is currently not possible to opt-out of instrumentation. This +is the case for: +- `net/http` client instrumentation +- `github.com/gorilla/mux` middleware instrumentation +{{}} + +## Creating custom trace spans + +Any function annotated with the `//dd:span` directive will result in a trace +span being created around the function's execution. The directive can optionally +provide custom span tags as `key:value` pairs (all parsed as literal strings): + +```go +//dd:span tag-name:for other-tag:bar +func tracedFunction() { + // This function will be represented as a span named "tracedFunction" +} +``` + +### Result Capture + +Functions annotated with `//dd:span` which return an `error` value will +automatically annotate spans with the returned `error` information if that is +non-`nil`. + +```go +package demo + +import "errors" + +//dd:span +func failableFunction() (any, error) { + // This span will have error information attached automatically. + return nil, errors.ErrUnsupported +} +``` + +### Operation Name + +The name of the operation (span name) is determined using the following +precedence list (first non-empty is selected): + +- The `span.name` tag specified as a directive argument + ```go + //dd:span span.name:operationName + func tracedFunction() { + // This function will be represented as a span named "operationName" + } + ``` +- The name of the function (closures do not have a name) + ```go + //dd:span tag-name:for other-tag:bar + func tracedFunction() { + // This function will be represented as a span named "tracedFunction" + } + ``` +- The value of the very first tag from the directive arguments list + ```go + //dd:span tag-name:spanName other-tag:bar + tracedFunction := func() { + // This function will be represented as a span named "spanName" + } + ``` + +### Trace Context Propagation + +If the annotated function accepts a {{}} argument, +that context will be used for trace propagation. Otherwise, if the function +accepts a {{}} argument, the request's context +will be used for trace propagation. + +Functions that accept neither solely rely on _goroutine local storage_ for trace +propagation. This means that traces may be split on _goroutine_ boundaries +unless a {{}} or {{}} +value carying trace context is passed across. + +Trace context carrying {{}} values are those that: + +- have been received by a `//dd:span` annotated function, as instrumentation + will create a new trace root span if if did not already carry trace context +- are returned by: + - {{}} + - {{}} + +```go +package demo + +//dd:span +func caller(ctx context.Context) { + wait := make(chan struct{}, 1) + defer close(wait) + + // Weaving the span context into the child goroutine + go callee(ctx, wait) + <-wait +} + +//dd:span +func callee(ctx context.Context, done chan<- struct{}) { + done <- struct{}{} +} +``` + +### Manual Instrumentation + +The {{}} library can be used to +manually instrument sections of your code even when building with `orchestrion`. + +You can use APIs such as {{}} +to create spans in any section of your code. This can be useful to delimit a +specific section of your code with a span without having to refactor it in a +separate function (which would allow the use of the `//dd:span` directive), or +when you need to customize the span more than the `//dd:span` directive allows. + +{{}} +You may also use integrations from the packages within +{{}}, although this may result +in duplicated trace spans if `orchestrion` supports automatic instrumentation of +the same integration. + +This can be useful to instrument calls that `orchestrion` does not yet support. +If you directly use integrations, we encourage you carefully review the +[release notes](https://github.com/DataDog/orchestrion/releases) before +upgrading to a new `orchestrion` release, so you can remove manual +instrumentation that was made redundant as necessary. +{{}} diff --git a/docs/content/docs/features.md b/docs/content/docs/features.md new file mode 100644 index 00000000..7a91a682 --- /dev/null +++ b/docs/content/docs/features.md @@ -0,0 +1,73 @@ +--- +title: Feature Activation +weight: 10 + +prev: /getting-started +next: /custom-trace +--- + +## Custom Go Tracer start-up + +All applications built using `orchestrion` automatically start the Datadog +tracer at the beginning of the `main` function using the tracer library's +default configuration. The recommended way to configure the tracer is by using +the designated environment variables, such as `DD_ENV`, `DD_SERVICE`, +`DD_VERSION`, etc... You can get more information on what environment variables +are available in the [documentation][env-var-doc]. + +If the `main` function is annotated with the `//dd:ignore` directive, the tracer +will not be started automatically, and you are responsible for calling +{{}} with your +preferred configuration options. + +[env-var-doc]: https://docs.datadoghq.com/tracing/trace_collection/library_config/go/#unified-service-tagging + +## Enabling the Go Profiler + +All applications built using `orchestrion` automatically start the Datadog +continuous profiler if the `DD_PROFILING_ENABLED` environment variable is set +to `1` or `true`. If profiling is enabled via the +[Datadog Admission Controller][dd-adm-controller], `DD_PROFILING_ENABLED` can be +set to `auto`. + +When enabled, the continuous profiler will activate the following profiles: +- {{}} +- {{}} +- {{}} +- {{}} + +[dd-adm-controller]: https://docs.datadoghq.com/containers/cluster_agent/admission_controller/?tab=datadogoperator + +## Enabling Application Security features + +Datadog Application Security (ASM) features are built into the tracer library, +but need to be enabled at run-time. The [Enabling ASM for Go][asm-for-go] +documentation explains how to enable Application Security for instrumented go +applications. + +In the majority of cases, all that's needed is to set `DD_APPSEC_ENABLED` to `1` +or `true`. + +{{}} +Datadog's Application Security features are only supported on Linux (AMD64, +ARM64) and macOS (AMD64, ARM64). + +On Linux platforms, the [Datadog in-app WAF][libddwaf] needs the `libc.so.6` and +`libpthread.so.0` shared libraries to be available; even if `CGO_ENABLED=1`. + +If your are building your applications in environments where `CGO_ENABLED=0`, +Application Security features are only available if you specify the `appsec` +build tag (`orchestrion go build -tags=appsec .`). + +For more information, refer to the [Enabling ASM for Go][asm-for-go] +documentation. + +[libddwaf]: https://github.com/DataDog/libddwaf +[asm-for-go]: https://docs.datadoghq.com/security/application_security/threats/setup/threat_detection/go/ +{{}} + +Building applications with `orchestrion` allows you to maximize coverage for +RASP features, such as +automatic protection against SQL Injection attacks. + +[asm-for-go]: https://docs.datadoghq.com/security/application_security/threats/setup/threat_detection/go/ diff --git a/docs/content/docs/getting-started.md b/docs/content/docs/getting-started.md index b86f23c5..b1ee62c3 100644 --- a/docs/content/docs/getting-started.md +++ b/docs/content/docs/getting-started.md @@ -3,7 +3,7 @@ title: "Getting Started" weight: 1 prev: /docs -next: /docs/built-in +next: /features --- ## Requirements