From 3f3186a4578f6590d378b0b1dc462fa8508be53d Mon Sep 17 00:00:00 2001 From: Lukasz Gut Date: Tue, 3 Dec 2024 11:11:46 +0100 Subject: [PATCH 1/6] Add MeasurementProcessor specification to Metrics SDK TODO Closes #4298. --- specification/metrics/sdk.md | 78 +++++++++++++++++++++++++++++++++++- 1 file changed, 76 insertions(+), 2 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 325c252fd7e..aed8721e25b 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -51,6 +51,11 @@ linkTitle: SDK * [Instrument advisory parameters](#instrument-advisory-parameters) * [Instrument enabled](#instrument-enabled) - [Attribute limits](#attribute-limits) +- [MeasurementProcessor](#measurementprocessor) + * [MeasurementProcessor operations](#measurementprocessor-operations) + + [OnMeasure](#onmeasure) + + [Shutdown](#shutdown-1) + + [ForceFlush](#forceflush-1) - [Exemplar](#exemplar) * [ExemplarFilter](#exemplarfilter) + [AlwaysOn](#alwayson) @@ -64,9 +69,9 @@ linkTitle: SDK - [MetricReader](#metricreader) * [MetricReader operations](#metricreader-operations) + [Collect](#collect) - + [Shutdown](#shutdown-1) + + [Shutdown](#shutdown-2) * [Periodic exporting MetricReader](#periodic-exporting-metricreader) - + [ForceFlush](#forceflush-1) + + [ForceFlush](#forceflush-2) - [MetricExporter](#metricexporter) * [Push Metric Exporter](#push-metric-exporter) + [Interface Definition](#interface-definition) @@ -986,6 +991,75 @@ Attributes which belong to Metrics are exempt from the time. Attribute truncation or deletion could affect identity of metric time series and the topic requires further analysis. +## MeasurementProcessor + +`MeasurementProcessor` is an interface which allows hooks when a `Measurement` is recorded by an `Instrument`. + +`MeasurementProcessors` can be registered directly on SDK `MeterProvider` and they are invoked in the same order as they were registered. + +SDK MUST allow users to implement and configure custom processors. + +The following diagram shows `MeasurementProcessor`'s relationship to other components in the SDK: + +```plaintext ++------------------+ +| MeterProvider | +----------------------+ +-----------------+ +| Meter A | Measurements... | | Metrics... | | +| Instrument X |-----------------> MeasurementProcessor +------------> In-memory state | +| Instrument Y + | | | | +| Meter B | +----------------------+ +-----------------+ +| Instrument Z | +| ... | +----------------------+ +-----------------+ +| ... | Measurements... | | Metrics... | | +| ... |-----------------> MeasurementProcessor +------------> In-memory state | +| ... | | | | | +| ... | +----------------------+ +-----------------+ ++------------------+ +``` + +### MeasurementProcessor operations + +#### OnMeasure + +`OnMeasure` is called when a `Measurement` is recorded. This method is called synchronously on the thread that emitted the `Measurement`, therefore it SHOULD NOT block or throw exceptions. + +**Parameters:** + +* `measurement` - a [Measurement](./api.md#measurement) that was recorded +* `context` - the resolved `Context` (the explicitly passed `Context` or the current `Contex`) + +**Returns:** Void + +For a `MeasurementProcessor` registered directly on SDK `MeterProvider`, the `measurement` mutations MUST be visible in next registered processors. + +A `MeasuremenetProcessor` may freely modify `measurement` for the duration of the `OnMeasure` call. + +#### Shutdown + +Shuts down the processor. Called when the SDK is shut down. This is an opportunity for the processor to do any cleanup required. + +`Shutdown` SHOULD be called only once for each`MeasurementProcessor` instance. After the call to `Shutdow`, subsequent calls to `OnMeasure` are not allowed. SDKs SHOULD ignore these calls gracefully, if possible. + +`Shutdown` SHOULD provide a way to let the caller know whether it succeeded, failed or timed out. + +`Shutdown` MUST include the effects of `ForceFlush`. + +`Shutdown` SHOULD complete or abort within some timeout. `Shutdown` can be implemented as a blocking API or an asynchronous API which notifies the caller via a callback or an event. OpenTelemetry SDK authors can decide if they want to make the shutdown timeout configurable. + +#### ForceFlush + +This is a hint to ensure that any tasks associated with `Measurements` for which the `MeasurementProcessor` had already received events prior to the call to `ForceFlush` SHOULD be completed as soon as possible, preferably before returning from this method. + + + +In particular, if any `MeasurementProcessor` has any associated exporter, it SHOULD try to call the exporter's `Export` with all `Measurements` for which this was not already done and then invoke `ForceFlush` on it. If a timeout is specified (see below), the `MeasurementProcessor` MUST prioritize honoring the timeout over finishing all calls. It MAY skip or abort some or all `Export` or `ForceFlush` calls it has made to achieve this goal. + +`ForceFlush` SHOULD provide a way to let the caller know whether it succeeded, failed or timed out. + +`ForceFlush` SHOULD only be called in cases where it is absolutely necessary, such as when using some FaaS providers that may suspend the process after an invocation, but before the `MeasurementProcessor` exports the emitted `Measuremenets`. + +`ForceFlush` SHOULD complete or abort within some timeout. `ForceFlush` can be implemented as a blocking API or an asynchronous API which notifies the caller via a callback or an event. OpenTelemetry SDK authors can decide if they want to make the flush timeout configurable. + ## Exemplar **Status**: [Stable](../document-status.md) From a8f5de39859491b002fb0c44bf0124ffc0bc301b Mon Sep 17 00:00:00 2001 From: Lukasz Gut Date: Tue, 3 Dec 2024 11:33:40 +0100 Subject: [PATCH 2/6] Update TODO --- specification/metrics/sdk.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index aed8721e25b..2e17d4a3b21 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -1050,7 +1050,7 @@ Shuts down the processor. Called when the SDK is shut down. This is an opportuni This is a hint to ensure that any tasks associated with `Measurements` for which the `MeasurementProcessor` had already received events prior to the call to `ForceFlush` SHOULD be completed as soon as possible, preferably before returning from this method. - + In particular, if any `MeasurementProcessor` has any associated exporter, it SHOULD try to call the exporter's `Export` with all `Measurements` for which this was not already done and then invoke `ForceFlush` on it. If a timeout is specified (see below), the `MeasurementProcessor` MUST prioritize honoring the timeout over finishing all calls. It MAY skip or abort some or all `Export` or `ForceFlush` calls it has made to achieve this goal. From 1ce9e4dd24148c86bfe8eb1059989eb341ec88a7 Mon Sep 17 00:00:00 2001 From: Lukasz Gut <40406905+Blinkuu@users.noreply.github.com> Date: Tue, 3 Dec 2024 16:57:45 +0100 Subject: [PATCH 3/6] Update specification/metrics/sdk.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add status field Co-authored-by: Robert Pająk --- specification/metrics/sdk.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 2e17d4a3b21..25d5b53b177 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -993,6 +993,8 @@ series and the topic requires further analysis. ## MeasurementProcessor +**Status**: [Development](../document-status.md) + `MeasurementProcessor` is an interface which allows hooks when a `Measurement` is recorded by an `Instrument`. `MeasurementProcessors` can be registered directly on SDK `MeterProvider` and they are invoked in the same order as they were registered. From 4b0a58d077271bf2b0e24296a62d5a42def4eaa7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Robert=20Paj=C4=85k?= Date: Tue, 3 Dec 2024 17:39:11 +0100 Subject: [PATCH 4/6] Update specification/metrics/sdk.md Co-authored-by: Reiley Yang --- specification/metrics/sdk.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 25d5b53b177..5d46af32677 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -1028,7 +1028,7 @@ The following diagram shows `MeasurementProcessor`'s relationship to other compo **Parameters:** * `measurement` - a [Measurement](./api.md#measurement) that was recorded -* `context` - the resolved `Context` (the explicitly passed `Context` or the current `Contex`) +* `context` - the resolved `Context` (the explicitly passed `Context` or the current `Context`) **Returns:** Void From 449d2fb8593645db8ea3871210546e05fa224ea6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Robert=20Paj=C4=85k?= Date: Tue, 3 Dec 2024 17:50:12 +0100 Subject: [PATCH 5/6] Update specification/metrics/sdk.md Co-authored-by: Reiley Yang --- specification/metrics/sdk.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 5d46af32677..81933d4c6b9 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -1040,7 +1040,7 @@ A `MeasuremenetProcessor` may freely modify `measurement` for the duration of th Shuts down the processor. Called when the SDK is shut down. This is an opportunity for the processor to do any cleanup required. -`Shutdown` SHOULD be called only once for each`MeasurementProcessor` instance. After the call to `Shutdow`, subsequent calls to `OnMeasure` are not allowed. SDKs SHOULD ignore these calls gracefully, if possible. +`Shutdown` SHOULD be called only once for each `MeasurementProcessor` instance. After the call to `Shutdown`, subsequent calls to `OnMeasure` are not allowed. SDKs SHOULD ignore these calls gracefully, if possible. `Shutdown` SHOULD provide a way to let the caller know whether it succeeded, failed or timed out. From 60adbd3265005cd8cbf4e897075182245d1097f6 Mon Sep 17 00:00:00 2001 From: Lukasz Gut Date: Fri, 6 Dec 2024 12:25:52 +0100 Subject: [PATCH 6/6] Remove Shutdown and ForceFlush from MeasurementProcessor spec --- specification/metrics/sdk.md | 52 +++++++++--------------------------- 1 file changed, 12 insertions(+), 40 deletions(-) diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 81933d4c6b9..a683327c438 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -54,8 +54,6 @@ linkTitle: SDK - [MeasurementProcessor](#measurementprocessor) * [MeasurementProcessor operations](#measurementprocessor-operations) + [OnMeasure](#onmeasure) - + [Shutdown](#shutdown-1) - + [ForceFlush](#forceflush-1) - [Exemplar](#exemplar) * [ExemplarFilter](#exemplarfilter) + [AlwaysOn](#alwayson) @@ -69,9 +67,9 @@ linkTitle: SDK - [MetricReader](#metricreader) * [MetricReader operations](#metricreader-operations) + [Collect](#collect) - + [Shutdown](#shutdown-2) + + [Shutdown](#shutdown-1) * [Periodic exporting MetricReader](#periodic-exporting-metricreader) - + [ForceFlush](#forceflush-2) + + [ForceFlush](#forceflush-1) - [MetricExporter](#metricexporter) * [Push Metric Exporter](#push-metric-exporter) + [Interface Definition](#interface-definition) @@ -1005,17 +1003,17 @@ The following diagram shows `MeasurementProcessor`'s relationship to other compo ```plaintext +------------------+ -| MeterProvider | +----------------------+ +-----------------+ -| Meter A | Measurements... | | Metrics... | | -| Instrument X |-----------------> MeasurementProcessor +------------> In-memory state | -| Instrument Y + | | | | -| Meter B | +----------------------+ +-----------------+ +| MeterProvider | +----------------------+ +-----------------+ +| Meter A | Measurements... | | Measurements... | | +| Instrument X |-----------------> MeasurementProcessor +-----------------> In-memory state | +| Instrument Y + | | | | +| Meter B | +----------------------+ +-----------------+ | Instrument Z | -| ... | +----------------------+ +-----------------+ -| ... | Measurements... | | Metrics... | | -| ... |-----------------> MeasurementProcessor +------------> In-memory state | -| ... | | | | | -| ... | +----------------------+ +-----------------+ +| ... | +----------------------+ +-----------------+ +| ... | Measurements... | | Measurements... | | +| ... |-----------------> MeasurementProcessor +-----------------> In-memory state | +| ... | | | | | +| ... | +----------------------+ +-----------------+ +------------------+ ``` @@ -1036,32 +1034,6 @@ For a `MeasurementProcessor` registered directly on SDK `MeterProvider`, the `me A `MeasuremenetProcessor` may freely modify `measurement` for the duration of the `OnMeasure` call. -#### Shutdown - -Shuts down the processor. Called when the SDK is shut down. This is an opportunity for the processor to do any cleanup required. - -`Shutdown` SHOULD be called only once for each `MeasurementProcessor` instance. After the call to `Shutdown`, subsequent calls to `OnMeasure` are not allowed. SDKs SHOULD ignore these calls gracefully, if possible. - -`Shutdown` SHOULD provide a way to let the caller know whether it succeeded, failed or timed out. - -`Shutdown` MUST include the effects of `ForceFlush`. - -`Shutdown` SHOULD complete or abort within some timeout. `Shutdown` can be implemented as a blocking API or an asynchronous API which notifies the caller via a callback or an event. OpenTelemetry SDK authors can decide if they want to make the shutdown timeout configurable. - -#### ForceFlush - -This is a hint to ensure that any tasks associated with `Measurements` for which the `MeasurementProcessor` had already received events prior to the call to `ForceFlush` SHOULD be completed as soon as possible, preferably before returning from this method. - - - -In particular, if any `MeasurementProcessor` has any associated exporter, it SHOULD try to call the exporter's `Export` with all `Measurements` for which this was not already done and then invoke `ForceFlush` on it. If a timeout is specified (see below), the `MeasurementProcessor` MUST prioritize honoring the timeout over finishing all calls. It MAY skip or abort some or all `Export` or `ForceFlush` calls it has made to achieve this goal. - -`ForceFlush` SHOULD provide a way to let the caller know whether it succeeded, failed or timed out. - -`ForceFlush` SHOULD only be called in cases where it is absolutely necessary, such as when using some FaaS providers that may suspend the process after an invocation, but before the `MeasurementProcessor` exports the emitted `Measuremenets`. - -`ForceFlush` SHOULD complete or abort within some timeout. `ForceFlush` can be implemented as a blocking API or an asynchronous API which notifies the caller via a callback or an event. OpenTelemetry SDK authors can decide if they want to make the flush timeout configurable. - ## Exemplar **Status**: [Stable](../document-status.md)