diff --git a/docs/src/main/asciidoc/sqs.adoc b/docs/src/main/asciidoc/sqs.adoc index 8acffc7ea..3b1156fc4 100644 --- a/docs/src/main/asciidoc/sqs.adoc +++ b/docs/src/main/asciidoc/sqs.adoc @@ -355,6 +355,37 @@ Optional> receivedMessage = template .receive(queue, SampleRecord.class); ``` +==== Observability Support + +The framework instruments the codebase for sending messages to SQS to publish observations. +This instrumentation will create 2 types of observations: + +* `sqs.single.message.publish` when a single SQS message is published by the service. It propagates the outbound tracing information by setting it up in `MessageHeaders`. +* `sqs.batch.message.publish` when multiple SQS messages are published by the service. It propagates the outbound tracing information by setting it up in `MessageHeaders` of each SQS message. + +Additionally, both types of observations measure the time taken to publish SQS messages and create the following `KeyValues`: + +.Low cardinality Keys +[cols="2"] +|=== +| Name | Description +| `messaging.operation` |The mode of SQS message publishing (values: `single message publish` or `batch message publish`). +|=== + +.High cardinality Keys +[cols="2"] +|=== +| Name | Description +| `messaging.message.id` |The ID of either a single published SQS message or concatenation IDs of all SQS messages in the entire published batch. +|=== + +When using a `Spring Boot` application with `auto-configuration` and the default `SqsTemplate`, enabling this functionality requires only the `ObservationRegistry` bean to be available. In any other case, the `ObservationRegistry` needs to be set in the `SqsTemplate` builder: + +[source, java] +---- +SqsTemplate.builder().observationRegistry(registry); +---- + === Receiving Messages The framework offers the following options to receive messages from a queue. @@ -1751,3 +1782,82 @@ Sample IAM policy granting access to SQS: "Resource": "yourARN" } ---- + +=== Observability Support + +The framework offers instrumentation for the SQS processing codebase to publish observations. +This instrumentation is available with SQS polling and `SqsTemplate`. +The observations measure the time taken to process SQS messages and create the following `KeyValues`: + +.High cardinality Keys +[cols="2"] +|=== +| Name | Description +| `messaging.message.id` |The ID of either a single processed SQS message or the concatenation IDs of all SQS messages in the entire processed batch. +|=== + +.Low cardinality Keys for SQS polling +[cols="2"] +|=== +| Name | Description +| `messaging.operation` |The mode of SQS message processing (values: `single message polling process` or `batch message polling process`). +|=== + +.Low cardinality Keys for `SqsTemplate` +[cols="2"] +|=== +| Name | Description +| `messaging.operation` |The mode of SQS message processing (values: `single message manual process` or `batch message manual process`). +|=== + + +==== SQS polling + +The instrumentation will create two types of observations: + +* `sqs.single.message.polling.process` when a SQS messages from the batch are processed by the service. They propagate the inbound tracing information by looking it up in `MessageHeaders`. +* `sqs.batch.message.polling.process` when the entire received batch of SQS messages is processed by the service. They propagate the inbound tracing information in the form the of a list of tracing `Link` by looking it up in the `MessageHeaders` of incoming traces in the entire received batch. + +When using a `Spring Boot` application with `auto-configuration` and the default `SqsMessageListenerContainerFactory`, enabling this functionality requires only the `ObservationRegistry` bean to be available. In any other case, the `ObservationRegistry` needs to be set in the `MessageListenerContainerFactory`: + +[source, java] +---- +@Bean +SqsMessageListenerContainerFactory sqsListenerContainerFactory(SqsAsyncClient sqsAsyncClient, ObservationRegistry observationRegistry) { + return SqsMessageListenerContainerFactory + .builder() + .sqsAsyncClientSupplier(sqsAsyncClient) + .observationRegistry(observationRegistry) + .build(); +} +---- + +Or directly in the `MessageListenerContainer`: + +[source, java] +---- +@Bean +MessageListenerContainer listenerContainer(SqsAsyncClient sqsAsyncClient, ObservationRegistry observationRegistry) { + return SqsMessageListenerContainer + .builder() + .sqsAsyncClient(sqsAsyncClient) + .observationRegistry(observationRegistry) + .build(); +} +---- + +==== SqsTemplate + +The instrumentation will create two types of observations: + +* `sqs.single.message.manual.process` when an individual SQS message from the batch is processed by the service. The inbound tracing information is not propagated from messages, and a new tracing context is created instead. +* `sqs.batch.message.manual.process` when the entire batch of received SQS messages is processed by the service. The inbound tracing information is not propagated from messages, and a new tracing context is created instead. + +*Note*: The inbound tracing information is not propagated, and a new tracing context is created because the user of SqsTemplate can invoke this API within the scope of an existing tracing context, where their trace ID should be reused. Additionally, the headers of received messages contain the required tracing information, allowing the user to decide whether to use the inbound tracing or not. + +When using a `Spring Boot` application with `auto-configuration` and the default `SqsTemplate`, enabling this functionality requires only the `ObservationRegistry` bean to be available. In any other case, the `ObservationRegistry` needs to be set in the `SqsTemplate` builder: + +[source, java] +---- +SqsTemplate.builder().observationRegistry(registry); +---- diff --git a/spring-cloud-aws-autoconfigure/src/main/java/io/awspring/cloud/autoconfigure/sqs/SqsAutoConfiguration.java b/spring-cloud-aws-autoconfigure/src/main/java/io/awspring/cloud/autoconfigure/sqs/SqsAutoConfiguration.java index c06b65b29..a5601c1de 100644 --- a/spring-cloud-aws-autoconfigure/src/main/java/io/awspring/cloud/autoconfigure/sqs/SqsAutoConfiguration.java +++ b/spring-cloud-aws-autoconfigure/src/main/java/io/awspring/cloud/autoconfigure/sqs/SqsAutoConfiguration.java @@ -15,6 +15,8 @@ */ package io.awspring.cloud.autoconfigure.sqs; +import static org.springframework.boot.actuate.autoconfigure.tracing.MicrometerTracingAutoConfiguration.RECEIVER_TRACING_OBSERVATION_HANDLER_ORDER; + import com.fasterxml.jackson.databind.ObjectMapper; import io.awspring.cloud.autoconfigure.AwsAsyncClientCustomizer; import io.awspring.cloud.autoconfigure.core.AwsClientBuilderConfigurer; @@ -30,21 +32,28 @@ import io.awspring.cloud.sqs.listener.errorhandler.ErrorHandler; import io.awspring.cloud.sqs.listener.interceptor.AsyncMessageInterceptor; import io.awspring.cloud.sqs.listener.interceptor.MessageInterceptor; +import io.awspring.cloud.sqs.observation.BatchMessageProcessTracingObservationHandler; import io.awspring.cloud.sqs.operations.SqsTemplate; import io.awspring.cloud.sqs.operations.SqsTemplateBuilder; import io.awspring.cloud.sqs.support.converter.MessagingMessageConverter; import io.awspring.cloud.sqs.support.converter.SqsMessagingMessageConverter; +import io.micrometer.observation.ObservationRegistry; +import io.micrometer.tracing.Tracer; +import io.micrometer.tracing.propagation.Propagator; import org.springframework.beans.factory.ObjectProvider; import org.springframework.boot.autoconfigure.AutoConfiguration; import org.springframework.boot.autoconfigure.AutoConfigureAfter; import org.springframework.boot.autoconfigure.EnableAutoConfiguration; +import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; import org.springframework.boot.autoconfigure.condition.ConditionalOnClass; import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.boot.context.properties.EnableConfigurationProperties; import org.springframework.boot.context.properties.PropertyMapper; import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Import; +import org.springframework.core.annotation.Order; import software.amazon.awssdk.services.sqs.SqsAsyncClient; import software.amazon.awssdk.services.sqs.SqsAsyncClientBuilder; import software.amazon.awssdk.services.sqs.model.Message; @@ -56,6 +65,7 @@ * @author Maciej Walkowiak * @author Wei Jiang * @author Dongha Kim + * @author Mariusz Sondecki * @since 3.0 */ @AutoConfiguration @@ -87,13 +97,15 @@ public SqsAsyncClient sqsAsyncClient(AwsClientBuilderConfigurer awsClientBuilder @ConditionalOnMissingBean @Bean public SqsTemplate sqsTemplate(SqsAsyncClient sqsAsyncClient, ObjectProvider objectMapperProvider, - MessagingMessageConverter messageConverter) { + MessagingMessageConverter messageConverter, + ObjectProvider observationRegistry) { SqsTemplateBuilder builder = SqsTemplate.builder().sqsAsyncClient(sqsAsyncClient) .messageConverter(messageConverter); objectMapperProvider.ifAvailable(om -> setMapperToConverter(messageConverter, om)); if (sqsProperties.getQueueNotFoundStrategy() != null) { builder.configure((options) -> options.queueNotFoundStrategy(sqsProperties.getQueueNotFoundStrategy())); } + observationRegistry.ifAvailable(builder::observationRegistry); return builder.build(); } @@ -104,7 +116,8 @@ public SqsMessageListenerContainerFactory defaultSqsListenerContainerFac ObjectProvider> errorHandler, ObjectProvider> asyncInterceptors, ObjectProvider> interceptors, ObjectProvider objectMapperProvider, - MessagingMessageConverter messagingMessageConverter) { + MessagingMessageConverter messagingMessageConverter, + ObjectProvider observationRegistry) { SqsMessageListenerContainerFactory factory = new SqsMessageListenerContainerFactory<>(); factory.configure(this::configureProperties); @@ -115,6 +128,8 @@ public SqsMessageListenerContainerFactory defaultSqsListenerContainerFac asyncInterceptors.forEach(factory::addMessageInterceptor); objectMapperProvider.ifAvailable(om -> setMapperToConverter(messagingMessageConverter, om)); factory.configure(options -> options.messageConverter(messagingMessageConverter)); + observationRegistry.ifAvailable(factory::setObservationRegistry); + return factory; } @@ -149,4 +164,17 @@ public SqsListenerConfigurer objectMapperCustomizer(ObjectProvider }; } + @Configuration(proxyBeanMethods = false) + @ConditionalOnBean({ Tracer.class, Propagator.class }) + public static class SqsTracingConfiguration { + + @Bean + @ConditionalOnMissingBean + @Order(RECEIVER_TRACING_OBSERVATION_HANDLER_ORDER - 100) + public BatchMessageProcessTracingObservationHandler batchMessageProcessTracingObservationHandler(Tracer tracer, + Propagator propagator) { + return new BatchMessageProcessTracingObservationHandler(tracer, propagator); + } + + } } diff --git a/spring-cloud-aws-autoconfigure/src/test/java/io/awspring/cloud/autoconfigure/sqs/SqsAutoConfigurationTest.java b/spring-cloud-aws-autoconfigure/src/test/java/io/awspring/cloud/autoconfigure/sqs/SqsAutoConfigurationTest.java index 9ad9bc5d6..753eb25f1 100644 --- a/spring-cloud-aws-autoconfigure/src/test/java/io/awspring/cloud/autoconfigure/sqs/SqsAutoConfigurationTest.java +++ b/spring-cloud-aws-autoconfigure/src/test/java/io/awspring/cloud/autoconfigure/sqs/SqsAutoConfigurationTest.java @@ -35,9 +35,12 @@ import io.awspring.cloud.sqs.listener.QueueNotFoundStrategy; import io.awspring.cloud.sqs.listener.errorhandler.AsyncErrorHandler; import io.awspring.cloud.sqs.listener.interceptor.AsyncMessageInterceptor; +import io.awspring.cloud.sqs.observation.BatchMessageProcessTracingObservationHandler; import io.awspring.cloud.sqs.operations.SqsTemplate; import io.awspring.cloud.sqs.support.converter.MessagingMessageConverter; import io.awspring.cloud.sqs.support.converter.SqsMessagingMessageConverter; +import io.micrometer.tracing.Tracer; +import io.micrometer.tracing.propagation.Propagator; import java.net.URI; import java.time.Duration; import java.util.List; @@ -242,6 +245,13 @@ void configuresMessageConverter() { }); } + @Test + void configureSqsObservation() { + this.contextRunner.withPropertyValues("spring.cloud.aws.sqs.enabled:true") + .withUserConfiguration(TracingConfiguration.class) + .run(context -> assertThat(context).hasSingleBean(BatchMessageProcessTracingObservationHandler.class)); + } + @Configuration(proxyBeanMethods = false) static class CustomComponentsConfiguration { @@ -300,4 +310,19 @@ public SdkAsyncHttpClient asyncHttpClient() { } } + @Configuration(proxyBeanMethods = false) + static class TracingConfiguration { + + @Bean + Propagator propagator() { + return Propagator.NOOP; + } + + @Bean + Tracer tracer() { + return Tracer.NOOP; + } + + } + } diff --git a/spring-cloud-aws-sqs/pom.xml b/spring-cloud-aws-sqs/pom.xml index e8d139b0e..ba3d7a4a5 100644 --- a/spring-cloud-aws-sqs/pom.xml +++ b/spring-cloud-aws-sqs/pom.xml @@ -34,6 +34,10 @@ org.springframework.retry spring-retry + + io.micrometer + micrometer-tracing + com.fasterxml.jackson.core jackson-databind @@ -57,7 +61,26 @@ junit-jupiter test - + + io.micrometer + micrometer-observation-test + test + + + io.micrometer + micrometer-tracing-test + test + + + io.micrometer + micrometer-tracing-bridge-brave + test + + + io.zipkin.brave + brave-tests + test + diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/ContextScopeManager.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/ContextScopeManager.java new file mode 100644 index 000000000..269ce3535 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/ContextScopeManager.java @@ -0,0 +1,113 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs; + +import io.micrometer.context.ContextSnapshot; +import io.micrometer.observation.ObservationRegistry; +import java.util.concurrent.CompletableFuture; +import java.util.function.Function; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.lang.Nullable; + +/** + * @author Mariusz Sondecki + */ +public final class ContextScopeManager { + + private static final Logger logger = LoggerFactory.getLogger(ContextScopeManager.class); + + private final ContextSnapshot currentContextSnapshot; + + @Nullable + private final ContextSnapshot previousContextSnapshot; + + private final ObservationRegistry observationRegistry; + + @Nullable + private ContextSnapshot.Scope currentScope; + + public ContextScopeManager(ContextSnapshot currentContextSnapshot, + @Nullable ContextSnapshot previousContextSnapshot, ObservationRegistry observationRegistry) { + this.currentContextSnapshot = currentContextSnapshot; + this.previousContextSnapshot = previousContextSnapshot; + this.observationRegistry = observationRegistry; + + logNoOpObservationRegistry(); + } + + public void restoreScope() { + if (observationRegistry.isNoop()) { + return; + } + closeCurrentScope(); // Ensure current scope is closed + if (observationRegistry.getCurrentObservationScope() == null && previousContextSnapshot != null) { + restorePreviousScope(); + } + else { + logger.trace( + "Current observation scope is active or previous scope is not available; not restoring previous scope"); + } + } + + // @formatter:off + public CompletableFuture manageContextWhileComposing(CompletableFuture future, + Function> fn) { + if (observationRegistry.isNoop()){ + return future.thenCompose(fn); + } + return future + .whenComplete((t, throwable) -> closeCurrentScope()) + .thenCompose(fn) + .whenComplete((u, throwable) -> openCurrentScope()); + } + // @formatter:on + + private void restorePreviousScope() { + logger.trace("Restoring previous scope"); + previousContextSnapshot.setThreadLocals(); + logger.debug("Previous scope restored successfully"); + } + + private void openCurrentScope() { + logger.trace("Opening scope"); + currentScope = currentContextSnapshot.setThreadLocals(); + logger.debug("Scope opened successfully"); + } + + private void closeCurrentScope() { + if (currentScope != null && observationRegistry.getCurrentObservationScope() != null) { + try { + logger.trace("Closing scope"); + currentScope.close(); + currentScope = null; + logger.debug("Scope closed successfully"); + } + catch (Exception e) { + logger.error("Failed to close scope", e); + } + } + else { + logger.trace("No scope to close"); + } + } + + private void logNoOpObservationRegistry() { + if (observationRegistry.isNoop()) { + logger.trace("ObservationRegistry is in No-Op mode"); + } + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/QueueAttributesResolver.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/QueueAttributesResolver.java index aca55e50b..58c6e3fac 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/QueueAttributesResolver.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/QueueAttributesResolver.java @@ -48,6 +48,7 @@ * @author Tomaz Fernandes * @author Agim Emruli * @author Adrian Stoelken + * @author Mariusz Sondecki * @since 3.0 * @see SqsContainerOptions#getQueueAttributeNames() * @see SqsContainerOptions#getQueueNotFoundStrategy() @@ -77,10 +78,14 @@ public static Builder builder() { // @formatter:off public CompletableFuture resolveQueueAttributes() { + return resolveQueueAttributes(null); + } + + public CompletableFuture resolveQueueAttributes(@Nullable ContextScopeManager scopeManager) { logger.debug("Resolving attributes for queue {}", this.queueName); - return CompletableFutures.exceptionallyCompose(resolveQueueUrl() - .thenCompose(queueUrl -> getQueueAttributes(queueUrl) - .thenApply(queueAttributes -> new QueueAttributes(this.queueName, queueUrl, queueAttributes))) + return CompletableFutures.exceptionallyCompose(resolveQueueUrl(scopeManager) + .thenCompose(queueUrl -> getQueueAttributes(queueUrl, scopeManager) + .thenApply(queueAttributes -> new QueueAttributes(this.queueName, queueUrl, queueAttributes))) , this::wrapException); } @@ -90,13 +95,13 @@ private CompletableFuture wrapException(Throwable t) { t instanceof CompletionException ? t.getCause() : t)); } - private CompletableFuture resolveQueueUrl() { + private CompletableFuture resolveQueueUrl(@Nullable ContextScopeManager scopeManager) { return isValidQueueUrl(this.queueName) ? CompletableFuture.completedFuture(this.queueName) - : doResolveQueueUrl(); + : doResolveQueueUrl(scopeManager); } - private CompletableFuture doResolveQueueUrl() { + private CompletableFuture doResolveQueueUrl(@Nullable ContextScopeManager scopeManager) { GetQueueUrlRequest.Builder getQueueUrlRequestBuilder = GetQueueUrlRequest.builder(); Arn arn = getQueueArnFromUrl(); if (arn != null) { @@ -106,9 +111,19 @@ private CompletableFuture doResolveQueueUrl() { else { getQueueUrlRequestBuilder.queueName(this.queueName); } - return CompletableFutures.exceptionallyCompose(this.sqsAsyncClient - .getQueueUrl(getQueueUrlRequestBuilder.build()).thenApply(GetQueueUrlResponse::queueUrl), - this::handleException); + + CompletableFuture getQueueUrlResponse; + if (scopeManager != null) { + getQueueUrlResponse = scopeManager.manageContextWhileComposing(CompletableFuture.completedFuture(null), + unused -> this.sqsAsyncClient + .getQueueUrl(getQueueUrlRequestBuilder.build())); + } else { + getQueueUrlResponse = this.sqsAsyncClient + .getQueueUrl(getQueueUrlRequestBuilder.build()); + } + return CompletableFutures.exceptionallyCompose(getQueueUrlResponse + .thenApply(GetQueueUrlResponse::queueUrl), + this::handleException); } private CompletableFuture handleException(Throwable t) { @@ -138,16 +153,25 @@ private void logCreateQueueResult(String v, Throwable t) { logger.debug("Created queue {} with url {}", this.queueName, v); } - private CompletableFuture> getQueueAttributes(String queueUrl) { + private CompletableFuture> getQueueAttributes(String queueUrl, @Nullable ContextScopeManager scopeManager) { return this.queueAttributeNames.isEmpty() ? CompletableFuture.completedFuture(Collections.emptyMap()) - : doGetAttributes(queueUrl); + : doGetAttributes(queueUrl, scopeManager); } - private CompletableFuture> doGetAttributes(String queueUrl) { + private CompletableFuture> doGetAttributes(String queueUrl, @Nullable ContextScopeManager scopeManager) { logger.debug("Resolving attributes {} for queue {}", this.queueAttributeNames, this.queueName); - return this.sqsAsyncClient - .getQueueAttributes(req -> req.queueUrl(queueUrl).attributeNames(this.queueAttributeNames)) + CompletableFuture queueAttributesResponse; + if (scopeManager != null) { + queueAttributesResponse = scopeManager.manageContextWhileComposing(CompletableFuture.completedFuture(null), + unused -> this.sqsAsyncClient + .getQueueAttributes(req -> req.queueUrl(queueUrl).attributeNames(this.queueAttributeNames))); + } else{ + queueAttributesResponse = this.sqsAsyncClient + .getQueueAttributes(req -> req.queueUrl(queueUrl).attributeNames(this.queueAttributeNames)); + } + + return queueAttributesResponse .thenApply(GetQueueAttributesResponse::attributes) .whenComplete((v, t) -> logger.debug("Attributes for queue {} resolved", this.queueName)); } diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/config/SqsMessageListenerContainerFactory.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/config/SqsMessageListenerContainerFactory.java index e2f9f4902..e5f00623f 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/config/SqsMessageListenerContainerFactory.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/config/SqsMessageListenerContainerFactory.java @@ -30,6 +30,7 @@ import io.awspring.cloud.sqs.listener.errorhandler.ErrorHandler; import io.awspring.cloud.sqs.listener.interceptor.AsyncMessageInterceptor; import io.awspring.cloud.sqs.listener.interceptor.MessageInterceptor; +import io.micrometer.observation.ObservationRegistry; import java.util.ArrayList; import java.util.Collection; import java.util.function.Consumer; @@ -123,6 +124,7 @@ * * @author Tomaz Fernandes * @author Joao Calassio + * @author Mariusz Sondecki * @since 3.0 * @see SqsMessageListenerContainer * @see ContainerOptions @@ -135,6 +137,8 @@ public class SqsMessageListenerContainerFactory extends private Supplier sqsAsyncClientSupplier; + private ObservationRegistry observationRegistry; + public SqsMessageListenerContainerFactory() { super(SqsContainerOptions.builder().build()); } @@ -146,7 +150,9 @@ protected SqsMessageListenerContainer createContainerInstance(Endpoint endpoi endpoint.getId() != null ? endpoint.getId() : endpoint.getLogicalNames()); Assert.notNull(this.sqsAsyncClientSupplier, "asyncClientSupplier not set"); SqsAsyncClient asyncClient = getSqsAsyncClientInstance(); - return new SqsMessageListenerContainer<>(asyncClient, containerOptions); + SqsMessageListenerContainer container = new SqsMessageListenerContainer<>(asyncClient, containerOptions); + ConfigUtils.INSTANCE.acceptIfNotNull(this.observationRegistry, container::setObservationRegistry); + return container; } protected SqsAsyncClient getSqsAsyncClientInstance() { @@ -178,6 +184,11 @@ public void setSqsAsyncClientSupplier(Supplier sqsAsyncClientSup this.sqsAsyncClientSupplier = sqsAsyncClientSupplier; } + public void setObservationRegistry(ObservationRegistry observationRegistry) { + Assert.notNull(observationRegistry, "observationRegistry cannot be null."); + this.observationRegistry = observationRegistry; + } + /** * Set the {@link SqsAsyncClient} instance to be shared by the containers. For high throughput scenarios the client * should be tuned for allowing higher maximum connections. @@ -200,6 +211,8 @@ public static class Builder { private Supplier sqsAsyncClientSupplier; + private ObservationRegistry observationRegistry; + private SqsAsyncClient sqsAsyncClient; private Collection> containerComponentFactories; @@ -229,6 +242,11 @@ public Builder sqsAsyncClient(SqsAsyncClient sqsAsyncClient) { return this; } + public Builder observationRegistry(ObservationRegistry observationRegistry) { + this.observationRegistry = observationRegistry; + return this; + } + /** * Set a supplier for {@link SqsAsyncClient} instances. A new instance will be used for each container created * by this factory. Useful for high throughput containers where sharing an {@link SqsAsyncClient} would be @@ -306,7 +324,8 @@ public SqsMessageListenerContainerFactory build() { .acceptIfNotNull(this.asyncAcknowledgementResultCallback, factory::setAcknowledgementResultCallback) .acceptIfNotNull(this.containerComponentFactories, factory::setContainerComponentFactories) .acceptIfNotNull(this.sqsAsyncClient, factory::setSqsAsyncClient) - .acceptIfNotNull(this.sqsAsyncClientSupplier, factory::setSqsAsyncClientSupplier); + .acceptIfNotNull(this.sqsAsyncClientSupplier, factory::setSqsAsyncClientSupplier) + .acceptIfNotNull(this.observationRegistry, factory::setObservationRegistry); this.messageInterceptors.forEach(factory::addMessageInterceptor); this.asyncMessageInterceptors.forEach(factory::addMessageInterceptor); factory.configure(this.optionsConsumer); diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/AbstractMessageListenerContainer.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/AbstractMessageListenerContainer.java index 9566fbb7a..613ce0ede 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/AbstractMessageListenerContainer.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/AbstractMessageListenerContainer.java @@ -21,6 +21,7 @@ import io.awspring.cloud.sqs.listener.errorhandler.ErrorHandler; import io.awspring.cloud.sqs.listener.interceptor.AsyncMessageInterceptor; import io.awspring.cloud.sqs.listener.interceptor.MessageInterceptor; +import io.micrometer.observation.ObservationRegistry; import java.util.ArrayList; import java.util.Arrays; import java.util.Collection; @@ -71,6 +72,8 @@ public abstract class AbstractMessageListenerContainer, B extends ContainerOptionsBuilder> @@ -246,6 +248,9 @@ protected TaskExecutor createTaskExecutor() { executor.setQueueCapacity(poolSize); executor.setAllowCoreThreadTimeOut(true); executor.setThreadFactory(createThreadFactory()); + if (!getObservationRegistry().isNoop()) { + executor.setTaskDecorator(new ContextPropagatingTaskDecorator()); + } executor.afterPropertiesSet(); return executor; } diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/ObservationRegistryAware.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/ObservationRegistryAware.java new file mode 100644 index 000000000..7dae2e247 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/ObservationRegistryAware.java @@ -0,0 +1,34 @@ +/* + * Copyright 2013-2022 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.listener; + +import io.micrometer.observation.ObservationRegistry; + +/** + * Implementations are enabled to receive a {@link ObservationRegistry} instance. + * + * @author Mariusz Sondecki + * @since 3.2 + */ +public interface ObservationRegistryAware { + + /** + * Set the {@link ObservationRegistry} instance. + * @param observationRegistry the instance. + */ + void setObservationRegistry(ObservationRegistry observationRegistry); + +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/SqsMessageListenerContainer.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/SqsMessageListenerContainer.java index 8e4404b7b..718c7112f 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/SqsMessageListenerContainer.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/SqsMessageListenerContainer.java @@ -25,6 +25,7 @@ import io.awspring.cloud.sqs.listener.interceptor.MessageInterceptor; import io.awspring.cloud.sqs.listener.sink.MessageSink; import io.awspring.cloud.sqs.listener.source.MessageSource; +import io.micrometer.observation.ObservationRegistry; import java.util.ArrayList; import java.util.Arrays; import java.util.Collection; @@ -101,6 +102,7 @@ * be used. * * @author Tomaz Fernandes + * @author Mariusz Sondecki * @since 3.0 */ public class SqsMessageListenerContainer @@ -141,12 +143,16 @@ private boolean isFifoQueue(String name) { protected void doConfigureMessageSources(Collection> messageSources) { ConfigUtils.INSTANCE.acceptManyIfInstance(messageSources, SqsAsyncClientAware.class, asca -> asca.setSqsAsyncClient(this.sqsAsyncClient)); + ConfigUtils.INSTANCE.acceptManyIfInstance(messageSources, ObservationRegistryAware.class, + asca -> asca.setObservationRegistry(this.getObservationRegistry())); } @Override protected void doConfigureMessageSink(MessageSink messageSink) { ConfigUtils.INSTANCE.acceptIfInstance(messageSink, SqsAsyncClientAware.class, asca -> asca.setSqsAsyncClient(this.sqsAsyncClient)); + ConfigUtils.INSTANCE.acceptIfInstance(messageSink, ObservationRegistryAware.class, + asca -> asca.setObservationRegistry(this.getObservationRegistry())); } public static Builder builder() { @@ -184,6 +190,8 @@ public static class Builder { private Integer phase; + private ObservationRegistry observationRegistry; + public Builder id(String id) { this.id = id; return this; @@ -262,6 +270,11 @@ public Builder phase(Integer phase) { return this; } + public Builder observationRegistry(ObservationRegistry observationRegistry) { + this.observationRegistry = observationRegistry; + return this; + } + // @formatter:off public SqsMessageListenerContainer build() { SqsMessageListenerContainer container = new SqsMessageListenerContainer<>(this.sqsAsyncClient); @@ -275,7 +288,8 @@ public SqsMessageListenerContainer build() { .acceptIfNotNull(this.asyncAcknowledgementResultCallback, container::setAcknowledgementResultCallback) .acceptIfNotNull(this.containerComponentFactories, container::setComponentFactories) .acceptIfNotEmpty(this.queueNames, container::setQueueNames) - .acceptIfNotNullOrElse(container::setPhase, this.phase, DEFAULT_PHASE); + .acceptIfNotNullOrElse(container::setPhase, this.phase, DEFAULT_PHASE) + .acceptIfNotNull(this.observationRegistry, container::setObservationRegistry); this.messageInterceptors.forEach(container::addMessageInterceptor); this.asyncMessageInterceptors.forEach(container::addMessageInterceptor); diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/acknowledgement/SqsAcknowledgementExecutor.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/acknowledgement/SqsAcknowledgementExecutor.java index dc99d180d..246ba98ba 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/acknowledgement/SqsAcknowledgementExecutor.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/acknowledgement/SqsAcknowledgementExecutor.java @@ -16,12 +16,13 @@ package io.awspring.cloud.sqs.listener.acknowledgement; import io.awspring.cloud.sqs.CompletableFutures; +import io.awspring.cloud.sqs.ContextScopeManager; import io.awspring.cloud.sqs.MessageHeaderUtils; import io.awspring.cloud.sqs.SqsAcknowledgementException; -import io.awspring.cloud.sqs.listener.QueueAttributes; -import io.awspring.cloud.sqs.listener.QueueAttributesAware; -import io.awspring.cloud.sqs.listener.SqsAsyncClientAware; -import io.awspring.cloud.sqs.listener.SqsHeaders; +import io.awspring.cloud.sqs.listener.*; +import io.micrometer.context.ContextRegistry; +import io.micrometer.context.ContextSnapshotFactory; +import io.micrometer.observation.ObservationRegistry; import java.util.Collection; import java.util.Collections; import java.util.UUID; @@ -46,7 +47,7 @@ * @see ExecutingAcknowledgementProcessor */ public class SqsAcknowledgementExecutor - implements AcknowledgementExecutor, SqsAsyncClientAware, QueueAttributesAware { + implements AcknowledgementExecutor, SqsAsyncClientAware, QueueAttributesAware, ObservationRegistryAware { private static final Logger logger = LoggerFactory.getLogger(SqsAcknowledgementExecutor.class); @@ -56,6 +57,11 @@ public class SqsAcknowledgementExecutor private String queueName; + private ObservationRegistry observationRegistry = ObservationRegistry.NOOP; + + private final ContextSnapshotFactory snapshotFactory = ContextSnapshotFactory.builder() + .contextRegistry(ContextRegistry.getInstance()).build(); + @Override public void setQueueAttributes(QueueAttributes queueAttributes) { Assert.notNull(queueAttributes, "queueAttributes cannot be null"); @@ -69,6 +75,12 @@ public void setSqsAsyncClient(SqsAsyncClient sqsAsyncClient) { this.sqsAsyncClient = sqsAsyncClient; } + @Override + public void setObservationRegistry(ObservationRegistry observationRegistry) { + Assert.notNull(observationRegistry, "observationRegistry cannot be null"); + this.observationRegistry = observationRegistry; + } + @Override public CompletableFuture execute(Collection> messagesToAck) { try { @@ -93,12 +105,15 @@ private CompletableFuture deleteMessages(Collection> messagesTo logger.trace("Acknowledging messages for queue {}: {}", this.queueName, MessageHeaderUtils.getId(messagesToAck)); StopWatch watch = new StopWatch(); + ContextScopeManager scopeManager = new ContextScopeManager(snapshotFactory.captureAll(), null, observationRegistry); watch.start(); - return CompletableFutures.exceptionallyCompose(this.sqsAsyncClient - .deleteMessageBatch(createDeleteMessageBatchRequest(messagesToAck)) + return CompletableFutures.exceptionallyCompose(scopeManager + .manageContextWhileComposing(CompletableFuture.completedFuture(null), unused -> this.sqsAsyncClient + .deleteMessageBatch(createDeleteMessageBatchRequest(messagesToAck))) .thenRun(() -> {}), t -> CompletableFutures.failedFuture(createAcknowledgementException(messagesToAck, t))) - .whenComplete((v, t) -> logAckResult(messagesToAck, t, watch)); + .whenComplete((v, t) -> logAckResult(messagesToAck, t, watch)) + .whenComplete((unused, throwable) -> scopeManager.restoreScope()); } private DeleteMessageBatchRequest createDeleteMessageBatchRequest(Collection> messagesToAck) { diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/AbstractMessageProcessingPipelineSink.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/AbstractMessageProcessingPipelineSink.java index bac3c116d..7f843c933 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/AbstractMessageProcessingPipelineSink.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/AbstractMessageProcessingPipelineSink.java @@ -17,10 +17,14 @@ import io.awspring.cloud.sqs.MessageHeaderUtils; import io.awspring.cloud.sqs.listener.MessageProcessingContext; +import io.awspring.cloud.sqs.listener.ObservationRegistryAware; import io.awspring.cloud.sqs.listener.TaskExecutorAware; import io.awspring.cloud.sqs.listener.pipeline.MessageProcessingPipeline; +import io.awspring.cloud.sqs.observation.*; +import io.micrometer.observation.ObservationRegistry; import java.util.Collection; import java.util.Collections; +import java.util.Objects; import java.util.UUID; import java.util.concurrent.CompletableFuture; import java.util.concurrent.Executor; @@ -41,13 +45,18 @@ * @param the {@link Message} payload type. * * @author Tomaz Fernandes + * @author Mariusz Sondecki * @since 3.0 */ public abstract class AbstractMessageProcessingPipelineSink - implements MessageProcessingPipelineSink, TaskExecutorAware { + implements MessageProcessingPipelineSink, TaskExecutorAware, ObservationRegistryAware { private static final Logger logger = LoggerFactory.getLogger(AbstractMessageProcessingPipelineSink.class); + private static final SingleMessagePollingProcessObservationConvention DEFAULT_SINGLE_MESSAGE_PROCESS_OBSERVATION_CONVENTION = new SingleMessagePollingProcessObservationConvention(); + + private static final BatchMessagePollingProcessObservationConvention DEFAULT_BATCH_MESSAGE_PROCESS_OBSERVATION_CONVENTION = new BatchMessagePollingProcessObservationConvention(); + private final Object lifecycleMonitor = new Object(); private volatile boolean running; @@ -58,6 +67,8 @@ public abstract class AbstractMessageProcessingPipelineSink private String id; + private ObservationRegistry observationRegistry = ObservationRegistry.NOOP; + @Override public void setMessagePipeline(MessageProcessingPipeline messageProcessingPipeline) { Assert.notNull(messageProcessingPipeline, "messageProcessingPipeline must not be null."); @@ -124,6 +135,24 @@ protected Void logError(Throwable t, Collection> msgs) { return null; } + protected CompletableFuture tryObservedCompletableFuture(Supplier> supplier, + Message msg) { + return Objects.requireNonNull(MessageObservationDocumentation.SINGLE_MESSAGE_POLLING_PROCESS.observation(null, + DEFAULT_SINGLE_MESSAGE_PROCESS_OBSERVATION_CONVENTION, + () -> new SingleMessagePollingProcessObservationContext(msg.getHeaders()), this.observationRegistry) + .observe(supplier)); + } + + protected CompletableFuture tryObservedCompletableFuture(Supplier> supplier, + Collection> msgs) { + return Objects.requireNonNull(MessageObservationDocumentation.BATCH_MESSAGE_POLLING_PROCESS + .observation(null, DEFAULT_BATCH_MESSAGE_PROCESS_OBSERVATION_CONVENTION, + () -> new BatchMessagePollingProcessObservationContext( + msgs.stream().map(Message::getHeaders).toList()), + this.observationRegistry) + .observe(supplier)); + } + private StopWatch getStartedWatch() { StopWatch watch = new StopWatch(); watch.start(); @@ -181,4 +210,8 @@ public boolean isRunning() { return this.running; } + @Override + public void setObservationRegistry(ObservationRegistry observationRegistry) { + this.observationRegistry = observationRegistry; + } } diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/BatchMessageSink.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/BatchMessageSink.java index bca81afbd..5673111be 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/BatchMessageSink.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/BatchMessageSink.java @@ -25,13 +25,15 @@ * {@link io.awspring.cloud.sqs.listener.pipeline.MessageProcessingPipeline}. * * @author Tomaz Fernandes + * @author Mariusz Sondecki * @since 3.0 */ public class BatchMessageSink extends AbstractMessageProcessingPipelineSink { @Override protected CompletableFuture doEmit(Collection> messages, MessageProcessingContext context) { - return execute(messages, context).exceptionally(t -> logError(t, messages)); + return tryObservedCompletableFuture(() -> execute(messages, context).exceptionally(t -> logError(t, messages)), + messages); } } diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/FanOutMessageSink.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/FanOutMessageSink.java index 83406f041..9d36d1c1c 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/FanOutMessageSink.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/FanOutMessageSink.java @@ -29,6 +29,7 @@ * @param the {@link Message} payload type. * * @author Tomaz Fernandes + * @author Mariusz Sondecki * @since 3.0 */ public class FanOutMessageSink extends AbstractMessageProcessingPipelineSink { @@ -38,9 +39,11 @@ public class FanOutMessageSink extends AbstractMessageProcessingPipelineSink< @Override protected CompletableFuture doEmit(Collection> messages, MessageProcessingContext context) { logger.trace("Emitting messages {}", MessageHeaderUtils.getId(messages)); - return CompletableFuture.allOf(messages.stream().map(msg -> execute(msg, context) - // Should log errors individually - no need to propagate upstream - .exceptionally(t -> logError(t, msg))).toArray(CompletableFuture[]::new)); + return CompletableFuture.allOf(messages.stream() + .map(msg -> tryObservedCompletableFuture(() -> execute(msg, context) + // Should log errors individually - no need to propagate upstream + .exceptionally(t -> logError(t, msg)), msg)) + .toArray(CompletableFuture[]::new)); } } diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/OrderedMessageSink.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/OrderedMessageSink.java index c498d4467..9f6f3ff06 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/OrderedMessageSink.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/OrderedMessageSink.java @@ -31,6 +31,7 @@ * @param the {@link Message} payload type. * * @author Tomaz Fernandes + * @author Mariusz Sondecki * @since 3.0 */ public class OrderedMessageSink extends AbstractMessageProcessingPipelineSink { @@ -41,14 +42,16 @@ public class OrderedMessageSink extends AbstractMessageProcessingPipelineSink protected CompletableFuture doEmit(Collection> messages, MessageProcessingContext context) { logger.trace("Emitting messages {}", MessageHeaderUtils.getId(messages)); CompletableFuture execution = messages.stream().reduce(CompletableFuture.completedFuture(null), - (resultFuture, msg) -> CompletableFutures.handleCompose(resultFuture, (v, t) -> { - if (t == null) { - return execute(msg, context).whenComplete(logIfError(msg)); - } - // Release backpressure from subsequent interrupted executions in case of errors. - context.runBackPressureReleaseCallback(); - return CompletableFutures.failedFuture(t); - }), (a, b) -> a); + (resultFuture, msg) -> tryObservedCompletableFuture( + () -> CompletableFutures.handleCompose(resultFuture, (v, t) -> { + if (t == null) { + return execute(msg, context).whenComplete(logIfError(msg)); + } + // Release backpressure from subsequent interrupted executions in case of errors. + context.runBackPressureReleaseCallback(); + return CompletableFutures.failedFuture(t); + }), msg), + (a, b) -> a); return execution.exceptionally(t -> null); } diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/adapter/AbstractDelegatingMessageListeningSinkAdapter.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/adapter/AbstractDelegatingMessageListeningSinkAdapter.java index 42f583dd9..47006d96d 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/adapter/AbstractDelegatingMessageListeningSinkAdapter.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/sink/adapter/AbstractDelegatingMessageListeningSinkAdapter.java @@ -18,11 +18,13 @@ import io.awspring.cloud.sqs.ConfigUtils; import io.awspring.cloud.sqs.LifecycleHandler; import io.awspring.cloud.sqs.listener.ContainerOptions; +import io.awspring.cloud.sqs.listener.ObservationRegistryAware; import io.awspring.cloud.sqs.listener.SqsAsyncClientAware; import io.awspring.cloud.sqs.listener.TaskExecutorAware; import io.awspring.cloud.sqs.listener.pipeline.MessageProcessingPipeline; import io.awspring.cloud.sqs.listener.sink.MessageProcessingPipelineSink; import io.awspring.cloud.sqs.listener.sink.MessageSink; +import io.micrometer.observation.ObservationRegistry; import org.springframework.core.task.TaskExecutor; import org.springframework.util.Assert; import software.amazon.awssdk.services.sqs.SqsAsyncClient; @@ -31,10 +33,11 @@ * {@link MessageProcessingPipelineSink} implementation that delegates method invocations to the provided delegate. * * @author Tomaz Fernandes + * @author Mariusz Sondecki * @since 3.0 */ public abstract class AbstractDelegatingMessageListeningSinkAdapter - implements MessageProcessingPipelineSink, TaskExecutorAware, SqsAsyncClientAware { + implements MessageProcessingPipelineSink, TaskExecutorAware, SqsAsyncClientAware, ObservationRegistryAware { private final MessageSink delegate; @@ -66,6 +69,12 @@ public void setSqsAsyncClient(SqsAsyncClient sqsAsyncClient) { saca -> saca.setSqsAsyncClient(sqsAsyncClient)); } + @Override + public void setObservationRegistry(ObservationRegistry observationRegistry) { + ConfigUtils.INSTANCE.acceptIfInstance(this.delegate, ObservationRegistryAware.class, + ea -> ea.setObservationRegistry(observationRegistry)); + } + @Override public void start() { LifecycleHandler.get().start(this.delegate); diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/source/AbstractSqsMessageSource.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/source/AbstractSqsMessageSource.java index 86f3e5ce1..d497b9c2a 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/source/AbstractSqsMessageSource.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/source/AbstractSqsMessageSource.java @@ -17,15 +17,11 @@ import io.awspring.cloud.sqs.ConfigUtils; import io.awspring.cloud.sqs.QueueAttributesResolver; -import io.awspring.cloud.sqs.listener.ContainerOptions; -import io.awspring.cloud.sqs.listener.QueueAttributes; -import io.awspring.cloud.sqs.listener.QueueAttributesAware; -import io.awspring.cloud.sqs.listener.QueueNotFoundStrategy; -import io.awspring.cloud.sqs.listener.SqsAsyncClientAware; -import io.awspring.cloud.sqs.listener.SqsContainerOptions; +import io.awspring.cloud.sqs.listener.*; import io.awspring.cloud.sqs.listener.acknowledgement.AcknowledgementExecutor; import io.awspring.cloud.sqs.listener.acknowledgement.ExecutingAcknowledgementProcessor; import io.awspring.cloud.sqs.listener.acknowledgement.SqsAcknowledgementExecutor; +import io.micrometer.observation.ObservationRegistry; import java.util.ArrayList; import java.util.Collection; import java.util.Collections; @@ -64,7 +60,7 @@ * @since 3.0 */ public abstract class AbstractSqsMessageSource extends AbstractPollingMessageSource - implements SqsAsyncClientAware { + implements SqsAsyncClientAware, ObservationRegistryAware { private static final Logger logger = LoggerFactory.getLogger(AbstractSqsMessageSource.class); @@ -88,12 +84,19 @@ public abstract class AbstractSqsMessageSource extends AbstractPollingMessage private int pollTimeout; + private ObservationRegistry observationRegistry; + @Override public void setSqsAsyncClient(SqsAsyncClient sqsAsyncClient) { Assert.notNull(sqsAsyncClient, "sqsAsyncClient cannot be null."); this.sqsAsyncClient = sqsAsyncClient; } + @Override + public void setObservationRegistry(ObservationRegistry observationRegistry) { + this.observationRegistry = observationRegistry; + } + @Override protected void doConfigure(ContainerOptions containerOptions) { Assert.isInstanceOf(SqsContainerOptions.class, containerOptions, @@ -146,7 +149,9 @@ protected AcknowledgementExecutor createAndConfigureAcknowledgementExecutor(Q ConfigUtils.INSTANCE .acceptIfInstance(executor, QueueAttributesAware.class, qaa -> qaa.setQueueAttributes(queueAttributes)) .acceptIfInstance(executor, SqsAsyncClientAware.class, - saca -> saca.setSqsAsyncClient(this.sqsAsyncClient)); + saca -> saca.setSqsAsyncClient(this.sqsAsyncClient)) + .acceptIfInstance(executor, ObservationRegistryAware.class, + ora -> ConfigUtils.INSTANCE.acceptIfNotNull(observationRegistry, ora::setObservationRegistry)); return executor; } diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessageManualProcessObservationConvention.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessageManualProcessObservationConvention.java new file mode 100644 index 000000000..614449a92 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessageManualProcessObservationConvention.java @@ -0,0 +1,29 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +/** + * Default implementation for {@link MessageObservationConvention} for the manual process of batch. + * + * @author Mariusz Sondecki + */ +public class BatchMessageManualProcessObservationConvention extends MessageManualProcessObservationConvention + implements MessageObservationConvention { + @Override + public MessagingOperationType getMessageType() { + return MessagingOperationType.BATCH_MANUAL_PROCESS; + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessagePollingProcessObservationContext.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessagePollingProcessObservationContext.java new file mode 100644 index 000000000..56256a24e --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessagePollingProcessObservationContext.java @@ -0,0 +1,39 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import io.micrometer.observation.transport.ReceiverContext; +import java.util.Collection; +import org.springframework.messaging.MessageHeaders; + +/** + * Context that holds information for observation metadata collection during the + * {@link MessageObservationDocumentation#BATCH_MESSAGE_POLLING_PROCESS processing of the whole received batch of SQS + * messages}. + *

+ * The inbound tracing information is propagated in the form of a list of {@link io.micrometer.tracing.Link} by looking + * it up in {@link MessageHeaders#get(Object, Class) incoming SQS message headers} of the entire received batch. + * + * @author Mariusz Sondecki + */ +public class BatchMessagePollingProcessObservationContext extends ReceiverContext> { + + public BatchMessagePollingProcessObservationContext(Collection messageHeaders) { + super((carrier, key) -> null); + setCarrier(messageHeaders); + } + +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessagePollingProcessObservationConvention.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessagePollingProcessObservationConvention.java new file mode 100644 index 000000000..5da6651f4 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessagePollingProcessObservationConvention.java @@ -0,0 +1,47 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import io.micrometer.observation.Observation; +import java.util.Objects; +import java.util.UUID; +import java.util.stream.Collectors; +import org.springframework.messaging.MessageHeaders; + +/** + * Default implementation for {@link MessageObservationConvention} for the polling process of batch. + * + * @author Mariusz Sondecki + */ +public class BatchMessagePollingProcessObservationConvention + implements MessageObservationConvention { + + @Override + public String getMessageId(BatchMessagePollingProcessObservationContext context) { + return context.getCarrier().stream().map(MessageHeaders::getId).filter(Objects::nonNull).map(UUID::toString) + .collect(Collectors.joining("; ")); + } + + @Override + public MessagingOperationType getMessageType() { + return MessagingOperationType.BATCH_POLLING_PROCESS; + } + + @Override + public boolean supportsContext(Observation.Context context) { + return context instanceof BatchMessagePollingProcessObservationContext; + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessageProcessTracingObservationHandler.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessageProcessTracingObservationHandler.java new file mode 100644 index 000000000..54831f6c3 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessageProcessTracingObservationHandler.java @@ -0,0 +1,60 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import io.micrometer.tracing.Link; +import io.micrometer.tracing.Span; +import io.micrometer.tracing.TraceContext; +import io.micrometer.tracing.Tracer; +import io.micrometer.tracing.handler.PropagatingReceiverTracingObservationHandler; +import io.micrometer.tracing.propagation.Propagator; +import java.util.function.Supplier; + +/** + * A TracingObservationHandler called when receiving occurred - e.g. batches of messages. + * + * @author Mariusz Sondecki + */ +public class BatchMessageProcessTracingObservationHandler + extends PropagatingReceiverTracingObservationHandler { + + private final Supplier propagatorSupplier; + + /** + * Creates a new instance of {@link BatchMessageProcessTracingObservationHandler}. + * + * @param tracer the tracer to use to record events + * @param propagator the mechanism to propagate tracing information from the carrier + */ + public BatchMessageProcessTracingObservationHandler(Tracer tracer, Propagator propagator) { + super(tracer, propagator); + this.propagatorSupplier = () -> propagator; + } + + @Override + public Span.Builder customizeExtractedSpan(BatchMessagePollingProcessObservationContext context, + Span.Builder builder) { + context.getCarrier().forEach(messageHeaders -> { + TraceContext traceContext = this.propagatorSupplier.get() + .extract(messageHeaders, (carrier, key) -> carrier.get(key, String.class)).start().context(); + String newParentSpanId = traceContext.parentId(); // newParentSpanId should be a span ID of received message + if (newParentSpanId != null) {// add only when received message contains tracing information + builder.addLink(new Link(traceContext)); + } + }); + return super.customizeExtractedSpan(context, builder); + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessagePublishObservationContext.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessagePublishObservationContext.java new file mode 100644 index 000000000..cabb5752a --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessagePublishObservationContext.java @@ -0,0 +1,50 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import io.micrometer.observation.transport.SenderContext; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.Map; +import org.springframework.messaging.MessageHeaders; + +/** + * Context that holds information for observation metadata collection during the + * {@link MessageObservationDocumentation#BATCH_MESSAGE_PUBLISH publication of the whole batch of SQS messages}. + *

+ * The outbound tracing information is propagated by setting it up in {@link Map#put(Object, Object) outgoing additional + * SQS message headers} of the entire received batch. + * + * @author Mariusz Sondecki + */ +public class BatchMessagePublishObservationContext extends SenderContext> { + + private Collection messageHeaders = Collections.emptyList(); + + public BatchMessagePublishObservationContext() { + super((carrier, key, value) -> carrier.put(key, value)); + setCarrier(new HashMap<>()); + } + + public void setMessageHeaders(Collection messageHeaders) { + this.messageHeaders = messageHeaders; + } + + public Collection getMessageHeaders() { + return messageHeaders; + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessagePublishObservationConvention.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessagePublishObservationConvention.java new file mode 100644 index 000000000..0362f6ed2 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/BatchMessagePublishObservationConvention.java @@ -0,0 +1,34 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import io.micrometer.observation.Observation; +import io.micrometer.observation.ObservationConvention; + +/** + * {@link ObservationConvention} interface for {@link MessageObservationDocumentation#BATCH_MESSAGE_PUBLISH SQS message + * publish} operations. + * + * @author Mariusz Sondecki + */ +public interface BatchMessagePublishObservationConvention + extends MessageObservationConvention { + + @Override + default boolean supportsContext(Observation.Context context) { + return context instanceof BatchMessagePublishObservationContext; + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/DefaultBatchMessagePublishObservationConvention.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/DefaultBatchMessagePublishObservationConvention.java new file mode 100644 index 000000000..03501bb4c --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/DefaultBatchMessagePublishObservationConvention.java @@ -0,0 +1,40 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import java.util.Objects; +import java.util.UUID; +import java.util.stream.Collectors; +import org.springframework.messaging.MessageHeaders; + +/** + * Default implementation for {@link BatchMessagePublishObservationConvention}. + * + * @author Mariusz Sondecki + */ +public class DefaultBatchMessagePublishObservationConvention implements BatchMessagePublishObservationConvention { + + @Override + public String getMessageId(BatchMessagePublishObservationContext context) { + return context.getMessageHeaders().stream().map(MessageHeaders::getId).filter(Objects::nonNull) + .map(UUID::toString).collect(Collectors.joining("; ")); + } + + @Override + public MessagingOperationType getMessageType() { + return MessagingOperationType.BATCH_PUBLISH; + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/DefaultSingleMessagePublishObservationConvention.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/DefaultSingleMessagePublishObservationConvention.java new file mode 100644 index 000000000..4a75a6d39 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/DefaultSingleMessagePublishObservationConvention.java @@ -0,0 +1,39 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import java.util.Optional; +import java.util.UUID; +import org.springframework.messaging.MessageHeaders; + +/** + * Default implementation for {@link SingleMessagePublishObservationConvention}. + * + * @author Mariusz Sondecki + */ +public class DefaultSingleMessagePublishObservationConvention implements SingleMessagePublishObservationConvention { + + @Override + public String getMessageId(SingleMessagePublishObservationContext context) { + return Optional.ofNullable(context.getMessageHeaders()).map(MessageHeaders::getId).map(UUID::toString) + .orElse(null); + } + + @Override + public MessagingOperationType getMessageType() { + return MessagingOperationType.SINGLE_PUBLISH; + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessageManualProcessObservationContext.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessageManualProcessObservationContext.java new file mode 100644 index 000000000..35dfdb323 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessageManualProcessObservationContext.java @@ -0,0 +1,39 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import io.micrometer.observation.Observation; +import java.util.Collection; +import java.util.Collections; +import org.springframework.messaging.MessageHeaders; + +/** + * Context that holds information for observation metadata collection during the manual process of SQS messages. + * + * @author Mariusz Sondecki + */ +public class MessageManualProcessObservationContext extends Observation.Context { + + private Collection messageHeaders = Collections.emptyList(); + + public Collection getMessageHeaders() { + return messageHeaders; + } + + public void setMessageHeaders(Collection messageHeaders) { + this.messageHeaders = messageHeaders; + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessageManualProcessObservationConvention.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessageManualProcessObservationConvention.java new file mode 100644 index 000000000..0c36cd96f --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessageManualProcessObservationConvention.java @@ -0,0 +1,42 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import io.micrometer.observation.Observation; +import java.util.Objects; +import java.util.UUID; +import java.util.stream.Collectors; +import org.springframework.messaging.MessageHeaders; + +/** + * {@link MessageObservationConvention} base class for manual SQS message operations. + * + * @author Mariusz Sondecki + */ +abstract class MessageManualProcessObservationConvention + implements MessageObservationConvention { + + @Override + public String getMessageId(MessageManualProcessObservationContext context) { + return context.getMessageHeaders().stream().map(MessageHeaders::getId).filter(Objects::nonNull) + .map(UUID::toString).collect(Collectors.joining("; ")); + } + + @Override + public boolean supportsContext(Observation.Context context) { + return context instanceof MessageManualProcessObservationContext; + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessageObservationConvention.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessageObservationConvention.java new file mode 100644 index 000000000..c51fcacf5 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessageObservationConvention.java @@ -0,0 +1,68 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import io.micrometer.common.KeyValue; +import io.micrometer.common.KeyValues; +import io.micrometer.observation.Observation; +import io.micrometer.observation.ObservationConvention; +import org.springframework.lang.Nullable; + +/** + * {@link ObservationConvention} interface for SQS message operations. + * + * @author Mariusz Sondecki + */ +public interface MessageObservationConvention extends ObservationConvention { + + @Nullable + String getMessageId(T context); + + MessagingOperationType getMessageType(); + + default String getOrDefault(String value) { + if (value == null) { + return KeyValue.NONE_VALUE; + } + return value; + } + + default KeyValue getId(T context) { + String messageId = getOrDefault(getMessageId(context)); + return KeyValue.of(MessageObservationDocumentation.HighCardinalityKeyNames.MESSAGE_ID, messageId); + } + + @Override + default String getContextualName(T context) { + return "%s %s".formatted(getId(context).getValue(), getMessageType().getValue()); + } + + @Override + default KeyValues getHighCardinalityKeyValues(T context) { + return KeyValues.of(getId(context)); + } + + @Override + default KeyValues getLowCardinalityKeyValues(T context) { + return KeyValues.of(KeyValue.of(MessageObservationDocumentation.LowCardinalityKeyNames.OPERATION, + getMessageType().getValue())); + } + + @Override + default String getName() { + return "sqs." + getMessageType().getValue().replace(" ", "."); + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessageObservationDocumentation.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessageObservationDocumentation.java new file mode 100644 index 000000000..06f7722e1 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessageObservationDocumentation.java @@ -0,0 +1,98 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import io.awspring.cloud.sqs.listener.sink.MessageProcessingPipelineSink; +import io.micrometer.common.docs.KeyName; +import io.micrometer.observation.Observation; +import io.micrometer.observation.ObservationConvention; +import io.micrometer.observation.docs.ObservationDocumentation; +import org.springframework.messaging.Message; + +/** + * Documented {@link io.micrometer.common.KeyValue KeyValues} for the observations on + * {@link MessageProcessingPipelineSink processing} of {@link Message SQS messages}. + * + * @author Mariusz Sondecki + */ +public enum MessageObservationDocumentation implements ObservationDocumentation { + + SINGLE_MESSAGE_POLLING_PROCESS { + @Override + public Class> getDefaultConvention() { + return SingleMessagePollingProcessObservationConvention.class; + } + }, + SINGLE_MESSAGE_MANUAL_PROCESS { + @Override + public Class> getDefaultConvention() { + return SingleMessageManualProcessObservationConvention.class; + } + }, + BATCH_MESSAGE_POLLING_PROCESS { + @Override + public Class> getDefaultConvention() { + return BatchMessagePollingProcessObservationConvention.class; + } + }, + BATCH_MESSAGE_MANUAL_PROCESS { + @Override + public Class> getDefaultConvention() { + return BatchMessageManualProcessObservationConvention.class; + } + }, + SINGLE_MESSAGE_PUBLISH { + @Override + public Class> getDefaultConvention() { + return DefaultSingleMessagePublishObservationConvention.class; + } + }, + BATCH_MESSAGE_PUBLISH { + @Override + public Class> getDefaultConvention() { + return DefaultBatchMessagePublishObservationConvention.class; + } + }; + + @Override + public KeyName[] getHighCardinalityKeyNames() { + return HighCardinalityKeyNames.values(); + } + + @Override + public KeyName[] getLowCardinalityKeyNames() { + return LowCardinalityKeyNames.values(); + } + + public enum HighCardinalityKeyNames implements KeyName { + + MESSAGE_ID { + @Override + public String asString() { + return "messaging.message.id"; + } + } + + } + + public enum LowCardinalityKeyNames implements KeyName { + OPERATION { + public String asString() { + return "messaging.operation"; + } + } + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessagingOperationType.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessagingOperationType.java new file mode 100644 index 000000000..129abc506 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/MessagingOperationType.java @@ -0,0 +1,40 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +/** + * @author Mariusz Sondecki + */ +public enum MessagingOperationType { + // @formatter:off + SINGLE_MANUAL_PROCESS("single message manual process"), + SINGLE_POLLING_PROCESS("single message polling process"), + BATCH_MANUAL_PROCESS("batch message manual process"), + BATCH_POLLING_PROCESS("batch message polling process"), + SINGLE_PUBLISH("single message publish"), + BATCH_PUBLISH("batch message publish"); + // @formatter:on + + private final String value; + + MessagingOperationType(String value) { + this.value = value; + } + + public String getValue() { + return value; + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessageManualProcessObservationConvention.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessageManualProcessObservationConvention.java new file mode 100644 index 000000000..8b8d94c56 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessageManualProcessObservationConvention.java @@ -0,0 +1,30 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +/** + * Default implementation for {@link MessageObservationConvention} for the manual process. + * + * @author Mariusz Sondecki + */ +public class SingleMessageManualProcessObservationConvention extends MessageManualProcessObservationConvention + implements MessageObservationConvention { + + @Override + public MessagingOperationType getMessageType() { + return MessagingOperationType.SINGLE_MANUAL_PROCESS; + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessagePollingProcessObservationContext.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessagePollingProcessObservationContext.java new file mode 100644 index 000000000..aa315953f --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessagePollingProcessObservationContext.java @@ -0,0 +1,37 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import io.micrometer.observation.transport.ReceiverContext; +import org.springframework.messaging.MessageHeaders; + +/** + * Context that holds information for observation metadata collection during the + * {@link MessageObservationDocumentation#SINGLE_MESSAGE_POLLING_PROCESS process of received SQS messages}. + *

+ * The inbound tracing information is propagated by looking it up in {@link MessageHeaders#get(Object, Class) incoming + * SQS message headers}. + * + * @author Mariusz Sondecki + */ +public class SingleMessagePollingProcessObservationContext extends ReceiverContext { + + public SingleMessagePollingProcessObservationContext(MessageHeaders messageHeaders) { + super((carrier, key) -> carrier.get(key, String.class)); + setCarrier(messageHeaders); + } + +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessagePollingProcessObservationConvention.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessagePollingProcessObservationConvention.java new file mode 100644 index 000000000..5fe8d3678 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessagePollingProcessObservationConvention.java @@ -0,0 +1,45 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import io.micrometer.observation.Observation; +import java.util.UUID; + +/** + * Default implementation for {@link MessageObservationConvention} for the polling process. + * + * @author Mariusz Sondecki + */ +public class SingleMessagePollingProcessObservationConvention + implements MessageObservationConvention { + + @Override + public String getMessageId(SingleMessagePollingProcessObservationContext context) { + UUID id = context.getCarrier().getId(); + return id != null ? id.toString() : null; + } + + @Override + public MessagingOperationType getMessageType() { + return MessagingOperationType.SINGLE_POLLING_PROCESS; + } + + @Override + public boolean supportsContext(Observation.Context context) { + return context instanceof SingleMessagePollingProcessObservationContext; + } + +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessagePublishObservationContext.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessagePublishObservationContext.java new file mode 100644 index 000000000..c3804e043 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessagePublishObservationContext.java @@ -0,0 +1,48 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import io.micrometer.observation.transport.SenderContext; +import java.util.HashMap; +import java.util.Map; +import org.springframework.messaging.MessageHeaders; + +/** + * Context that holds information for observation metadata collection during the + * {@link MessageObservationDocumentation#SINGLE_MESSAGE_PUBLISH publication of SQS messages}. + *

+ * The outbound tracing information is propagated by setting it up in {@link Map#put(Object, Object) outgoing additional + * SQS message headers}. + * + * @author Mariusz Sondecki + */ +public class SingleMessagePublishObservationContext extends SenderContext> { + + private MessageHeaders messageHeaders; + + public SingleMessagePublishObservationContext() { + super((carrier, key, value) -> carrier.put(key, value)); + setCarrier(new HashMap<>()); + } + + public MessageHeaders getMessageHeaders() { + return messageHeaders; + } + + public void setMessageHeaders(MessageHeaders messageHeaders) { + this.messageHeaders = messageHeaders; + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessagePublishObservationConvention.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessagePublishObservationConvention.java new file mode 100644 index 000000000..982f4c860 --- /dev/null +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/observation/SingleMessagePublishObservationConvention.java @@ -0,0 +1,34 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import io.micrometer.observation.Observation; +import io.micrometer.observation.ObservationConvention; + +/** + * {@link ObservationConvention} interface for {@link MessageObservationDocumentation#SINGLE_MESSAGE_PUBLISH SQS message + * publish} operations. + * + * @author Mariusz Sondecki + */ +public interface SingleMessagePublishObservationConvention + extends MessageObservationConvention { + + @Override + default boolean supportsContext(Observation.Context context) { + return context instanceof SingleMessagePublishObservationContext; + } +} diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/operations/AbstractMessagingTemplate.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/operations/AbstractMessagingTemplate.java index 28b2e9bb2..09e2f5a8b 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/operations/AbstractMessagingTemplate.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/operations/AbstractMessagingTemplate.java @@ -15,17 +15,19 @@ */ package io.awspring.cloud.sqs.operations; +import io.awspring.cloud.sqs.ContextScopeManager; import io.awspring.cloud.sqs.MessageHeaderUtils; +import io.awspring.cloud.sqs.observation.*; import io.awspring.cloud.sqs.support.converter.ContextAwareMessagingMessageConverter; import io.awspring.cloud.sqs.support.converter.MessageConversionContext; import io.awspring.cloud.sqs.support.converter.MessagingMessageConverter; +import io.micrometer.context.ContextRegistry; +import io.micrometer.context.ContextSnapshot; +import io.micrometer.context.ContextSnapshotFactory; +import io.micrometer.observation.Observation; +import io.micrometer.observation.ObservationRegistry; import java.time.Duration; -import java.util.Collection; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.Objects; -import java.util.Optional; +import java.util.*; import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletionException; import java.util.stream.Collectors; @@ -41,6 +43,7 @@ * @param the source message type for conversion * * @author Tomaz Fernandes + * @author Mariusz Sondecki * @since 3.0 */ public abstract class AbstractMessagingTemplate implements MessagingOperations, AsyncMessagingOperations { @@ -59,6 +62,17 @@ public abstract class AbstractMessagingTemplate implements MessagingOperation private static final int MAX_ONE_MESSAGE = 1; + private static final SingleMessageManualProcessObservationConvention DEFAULT_SINGLE_MESSAGE_PROCESS_MANUAL_OBSERVATION_CONVENTION = new SingleMessageManualProcessObservationConvention(); + + private static final BatchMessageManualProcessObservationConvention DEFAULT_BATCH_MESSAGE_PROCESS_MANUAL_OBSERVATION_CONVENTION = new BatchMessageManualProcessObservationConvention(); + + private static final DefaultSingleMessagePublishObservationConvention DEFAULT_SINGLE_MESSAGE_PUBLISH_OBSERVATION_CONVENTION = new DefaultSingleMessagePublishObservationConvention(); + + private static final DefaultBatchMessagePublishObservationConvention DEFAULT_BATCH_MESSAGE_PUBLISH_OBSERVATION_CONVENTION = new DefaultBatchMessagePublishObservationConvention(); + + private final ContextSnapshotFactory snapshotFactory = ContextSnapshotFactory.builder() + .contextRegistry(ContextRegistry.getInstance()).build(); + private final Map defaultAdditionalHeaders; private final Duration defaultPollTimeout; @@ -76,10 +90,13 @@ public abstract class AbstractMessagingTemplate implements MessagingOperation private final MessagingMessageConverter messageConverter; + private final ObservationRegistry observationRegistry; + protected AbstractMessagingTemplate(MessagingMessageConverter messageConverter, - AbstractMessagingTemplateOptions options) { + AbstractMessagingTemplateOptions options, ObservationRegistry observationRegistry) { Assert.notNull(messageConverter, "messageConverter must not be null"); Assert.notNull(options, "options must not be null"); + Assert.notNull(observationRegistry, "ObservationRegistry observationRegistry must not be null"); this.messageConverter = messageConverter; this.defaultAdditionalHeaders = options.defaultAdditionalHeaders; this.defaultMaxNumberOfMessages = options.defaultMaxNumberOfMessages; @@ -88,6 +105,7 @@ protected AbstractMessagingTemplate(MessagingMessageConverter messageConverte this.defaultEndpointName = options.defaultEndpointName; this.acknowledgementMode = options.acknowledgementMode; this.sendBatchFailureHandlingStrategy = options.sendBatchFailureHandlingStrategy; + this.observationRegistry = observationRegistry; } @Override @@ -154,24 +172,58 @@ public CompletableFuture>> receiveManyAsync(String que protected CompletableFuture>> receiveManyAsync(@Nullable String endpoint, @Nullable Class payloadClass, @Nullable Duration pollTimeout, @Nullable Integer maxNumberOfMessages, @Nullable Map additionalHeaders) { + MessageManualProcessObservationContext observationContext = new MessageManualProcessObservationContext(); String endpointToUse = getEndpointName(endpoint); - logger.trace("Receiving messages from endpoint {}", endpointToUse); Map headers = getAdditionalHeadersToReceive(endpointToUse, additionalHeaders); Duration pollTimeoutToUse = getOrDefault(pollTimeout, this.defaultPollTimeout, "pollTimeout"); Integer maxNumberOfMessagesToUse = getOrDefault(maxNumberOfMessages, this.defaultMaxNumberOfMessages, "defaultMaxNumberOfMessages"); - return doReceiveAsync(endpointToUse, pollTimeoutToUse, maxNumberOfMessagesToUse, headers) - .thenApply(messages -> convertReceivedMessages(endpointToUse, payloadClass, messages, headers)) - .thenCompose(messages -> handleAcknowledgement(endpointToUse, messages)) - .exceptionallyCompose(t -> CompletableFuture.failedFuture(new MessagingOperationFailedException( - "Message receive operation failed for endpoint %s".formatted(endpointToUse), endpointToUse, - t instanceof CompletionException ? t.getCause() : t))) - .whenComplete((v, t) -> logReceiveMessageResult(endpointToUse, v, t)); + Observation observation; + if (MAX_ONE_MESSAGE == maxNumberOfMessagesToUse) { + observation = MessageObservationDocumentation.SINGLE_MESSAGE_MANUAL_PROCESS + .observation(null, DEFAULT_SINGLE_MESSAGE_PROCESS_MANUAL_OBSERVATION_CONVENTION, + () -> observationContext, this.observationRegistry) + .start(); + } + else { + observation = MessageObservationDocumentation.BATCH_MESSAGE_MANUAL_PROCESS + .observation(null, DEFAULT_BATCH_MESSAGE_PROCESS_MANUAL_OBSERVATION_CONVENTION, + () -> observationContext, this.observationRegistry) + .start(); + } + + ContextSnapshot contextSnapshot = snapshotFactory.captureAll(); + try (Observation.Scope __ = observation.openScope()) { + ContextScopeManager scopeManager = new ContextScopeManager(snapshotFactory.captureAll(), contextSnapshot, + observationRegistry); + logger.trace("Receiving messages from endpoint {}", endpointToUse); + return doReceiveAsync(endpointToUse, pollTimeoutToUse, maxNumberOfMessagesToUse, headers, scopeManager) + .thenApply(messages -> convertReceivedMessages(endpointToUse, payloadClass, messages, headers, + scopeManager)) + .thenApply(messages -> addHeadersToContext(messages, observationContext)) + .thenCompose(messages -> handleAcknowledgement(endpointToUse, messages, scopeManager)) + .exceptionallyCompose(t -> CompletableFuture.failedFuture(new MessagingOperationFailedException( + "Message receive operation failed for endpoint %s".formatted(endpointToUse), endpointToUse, + t instanceof CompletionException ? t.getCause() : t))) + .whenComplete((v, t) -> { + logReceiveMessageResult(endpointToUse, v, t); + scopeManager.restoreScope(); + logger.trace("close observation {} of receiveManyAsync()", observation); + observation.stop(); + }); + } } protected abstract Map preProcessHeadersForReceive(String endpointToUse, Map additionalHeaders); + private Collection> addHeadersToContext( + Collection> messages, + MessageManualProcessObservationContext observationContext) { + observationContext.setMessageHeaders(messages.stream().map(Message::getHeaders).toList()); + return messages; + } + private Map getAdditionalHeadersToReceive(String endpointToUse, @Nullable Map additionalHeaders) { Map headers = new HashMap<>(this.defaultAdditionalHeaders); @@ -182,10 +234,11 @@ private Map getAdditionalHeadersToReceive(String endpointToUse, } private Collection> convertReceivedMessages(String endpoint, @Nullable Class payloadClass, - Collection messages, Map additionalHeaders) { + Collection messages, Map additionalHeaders, ContextScopeManager scopeManager) { + logger.info("convertReceivedMessages {}", observationRegistry.getCurrentObservationScope() != null); return messages.stream() .map(message -> convertReceivedMessage(getEndpointName(endpoint), message, - payloadClass != null ? payloadClass : this.defaultPayloadClass)) + payloadClass != null ? payloadClass : this.defaultPayloadClass, scopeManager)) .map(message -> addAdditionalHeaders(message, additionalHeaders)).collect(Collectors.toList()); } @@ -197,14 +250,15 @@ protected Message addAdditionalHeaders(Message message, Map handleAdditionalHeaders(Map additionalHeaders); private CompletableFuture>> handleAcknowledgement(@Nullable String endpoint, - Collection> messages) { + Collection> messages, ContextScopeManager scopeManager) { return TemplateAcknowledgementMode.ACKNOWLEDGE.equals(this.acknowledgementMode) && !messages.isEmpty() - ? doAcknowledgeMessages(getEndpointName(endpoint), messages).thenApply(theVoid -> messages) + ? doAcknowledgeMessages(getEndpointName(endpoint), messages, scopeManager) + .thenApply(theVoid -> messages) : CompletableFuture.completedFuture(messages); } protected abstract CompletableFuture doAcknowledgeMessages(String endpointName, - Collection> messages); + Collection> messages, ContextScopeManager scopeManager); private String getEndpointName(@Nullable String endpoint) { String endpointName = getOrDefault(endpoint, this.defaultEndpointName, "endpointName"); @@ -217,10 +271,11 @@ private V getOrDefault(@Nullable V value, V defaultValue, String valueName) valueName + " not set and no default value provided"); } - private Message convertReceivedMessage(String endpoint, S message, @Nullable Class payloadClass) { + private Message convertReceivedMessage(String endpoint, S message, @Nullable Class payloadClass, + ContextScopeManager scopeManager) { return this.messageConverter instanceof ContextAwareMessagingMessageConverter contextConverter ? contextConverter.toMessagingMessage(message, - getReceiveMessageConversionContext(endpoint, payloadClass)) + getReceiveMessageConversionContext(endpoint, payloadClass, scopeManager)) : this.messageConverter.toMessagingMessage(message); } @@ -241,7 +296,7 @@ protected Collection> addTypeToMessages(Collection> m } protected abstract CompletableFuture> doReceiveAsync(String endpointName, Duration pollTimeout, - Integer maxNumberOfMessages, Map additionalHeaders); + Integer maxNumberOfMessages, Map additionalHeaders, ContextScopeManager scopeManager); @Override public SendResult send(T payload) { @@ -277,45 +332,94 @@ public CompletableFuture> sendAsync(@Nullable String endpointN @Override public CompletableFuture> sendAsync(@Nullable String endpointName, Message message) { - String endpointToUse = getEndpointName(endpointName); - logger.trace("Sending message {} to endpoint {}", MessageHeaderUtils.getId(message), endpointName); - return preProcessMessageForSendAsync(endpointToUse, message).thenCompose( - messageToUse -> doSendAsync(endpointToUse, convertMessageToSend(messageToUse), messageToUse) - .exceptionallyCompose( - t -> CompletableFuture.failedFuture(new MessagingOperationFailedException( - "Message send operation failed for message %s to endpoint %s" - .formatted(MessageHeaderUtils.getId(message), endpointToUse), - endpointToUse, message, t))) - .whenComplete((v, t) -> logSendMessageResult(endpointToUse, message, t))); + SingleMessagePublishObservationContext observationContext = new SingleMessagePublishObservationContext(); + ContextSnapshot contextSnapshot = snapshotFactory.captureAll(); + Observation observation = MessageObservationDocumentation.SINGLE_MESSAGE_PUBLISH + .observation(null, DEFAULT_SINGLE_MESSAGE_PUBLISH_OBSERVATION_CONVENTION, () -> observationContext, + this.observationRegistry) + .start(); + try (Observation.Scope __ = observation.openScope()) { + ContextScopeManager scopeManager = new ContextScopeManager(snapshotFactory.captureAll(), contextSnapshot, + observationRegistry); + String endpointToUse = getEndpointName(endpointName); + logger.trace("Sending message {} to endpoint {}", MessageHeaderUtils.getId(message), endpointName); + return preProcessMessageForSendAsync(endpointToUse, message, scopeManager) + .thenApply(messageToUse -> maybeAddObservedSendHeaders(messageToUse, observationContext)) + .thenCompose(messageToUse -> doSendAsync(endpointToUse, convertMessageToSend(messageToUse), + messageToUse, scopeManager) + .exceptionallyCompose( + t -> CompletableFuture.failedFuture(new MessagingOperationFailedException( + "Message send operation failed for message %s to endpoint %s" + .formatted(MessageHeaderUtils.getId(message), endpointToUse), + endpointToUse, message, t))) + .whenComplete((v, t) -> { + logSendMessageResult(endpointToUse, message, t); + scopeManager.restoreScope(); + logger.trace("close observation {} of sendAsync()", observation); + observation.stop(); + })); + } } - protected abstract Message preProcessMessageForSend(String endpointToUse, Message message); + protected abstract Message preProcessMessageForSend(String endpointToUse, Message message, + ContextScopeManager scopeManager); - protected CompletableFuture> preProcessMessageForSendAsync(String endpointToUse, - Message message) { - return CompletableFuture.completedFuture(preProcessMessageForSend(endpointToUse, message)); + protected CompletableFuture> preProcessMessageForSendAsync(String endpointToUse, Message message, + ContextScopeManager scopeManager) { + return CompletableFuture.completedFuture(preProcessMessageForSend(endpointToUse, message, scopeManager)); } @Override public CompletableFuture> sendManyAsync(@Nullable String endpointName, Collection> messages) { - logger.trace("Sending messages {} to endpoint {}", MessageHeaderUtils.getId(messages), endpointName); - String endpointToUse = getEndpointName(endpointName); - return preProcessMessagesForSendAsync(endpointToUse, messages).thenCompose( - messagesToUse -> doSendBatchAsync(endpointToUse, convertMessagesToSend(messagesToUse), messagesToUse) - .exceptionallyCompose(t -> wrapSendException(messagesToUse, endpointToUse, t)) - .thenCompose(result -> handleFailedMessages(endpointToUse, result)) - .whenComplete((v, t) -> logSendMessageBatchResult(endpointToUse, messagesToUse, t))); + BatchMessagePublishObservationContext observationContext = new BatchMessagePublishObservationContext(); + ContextSnapshot contextSnapshot = snapshotFactory.captureAll(); + Observation observation = MessageObservationDocumentation.BATCH_MESSAGE_PUBLISH + .observation(null, DEFAULT_BATCH_MESSAGE_PUBLISH_OBSERVATION_CONVENTION, () -> observationContext, + this.observationRegistry) + .start(); + try (Observation.Scope __ = observation.openScope()) { + ContextScopeManager scopeManager = new ContextScopeManager(snapshotFactory.captureAll(), contextSnapshot, + observationRegistry); + logger.trace("Sending messages {} to endpoint {}", MessageHeaderUtils.getId(messages), endpointName); + String endpointToUse = getEndpointName(endpointName); + return preProcessMessagesForSendAsync(endpointToUse, messages, scopeManager) + .thenApply(messagesToUse -> maybeAddObservedSendHeaders(messagesToUse, observationContext)) + .thenCompose(messagesToUse -> doSendBatchAsync(endpointToUse, convertMessagesToSend(messagesToUse), + messagesToUse, scopeManager) + .exceptionallyCompose(t -> wrapSendException(messagesToUse, endpointToUse, t)) + .thenCompose(result -> handleFailedMessages(endpointToUse, result)).whenComplete((v, t) -> { + logSendMessageBatchResult(endpointToUse, messagesToUse, t); + scopeManager.restoreScope(); + logger.trace("close observation {} of sendManyAsync()", observation); + observation.stop(); + })); + } } protected abstract Collection> preProcessMessagesForSend(String endpointToUse, Collection> messages); protected CompletableFuture>> preProcessMessagesForSendAsync(String endpointToUse, - Collection> messages) { + Collection> messages, ContextScopeManager scopeManager) { return CompletableFuture.completedFuture(preProcessMessagesForSend(endpointToUse, messages)); } + private org.springframework.messaging.Message maybeAddObservedSendHeaders( + org.springframework.messaging.Message message, + SingleMessagePublishObservationContext observationContext) { + observationContext.setMessageHeaders(message.getHeaders()); + return MessageHeaderUtils.addHeadersIfAbsent(message, Objects.requireNonNull(observationContext.getCarrier())); + } + + private Collection> maybeAddObservedSendHeaders( + Collection> messages, + BatchMessagePublishObservationContext observationContext) { + observationContext.setMessageHeaders(messages.stream().map(Message::getHeaders).toList()); + return messages.stream().map(message -> MessageHeaderUtils.addHeadersIfAbsent(message, + Objects.requireNonNull(observationContext.getCarrier()))).toList(); + } + private CompletableFuture> handleFailedMessages(String endpointToUse, SendResult.Batch result) { return !result.failed().isEmpty() @@ -347,14 +451,14 @@ private S convertMessageToSend(Message message) { } protected abstract CompletableFuture> doSendAsync(String endpointName, S message, - Message originalMessage); + Message originalMessage, ContextScopeManager scopeManager); protected abstract CompletableFuture> doSendBatchAsync(String endpointName, - Collection messages, Collection> originalMessages); + Collection messages, Collection> originalMessages, ContextScopeManager scopeManager); @Nullable protected MessageConversionContext getReceiveMessageConversionContext(String endpointName, - @Nullable Class payloadClass) { + @Nullable Class payloadClass, ContextScopeManager scopeManager) { // Subclasses can override this method to return a context return null; } diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/operations/SqsTemplate.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/operations/SqsTemplate.java index 4c6ff0a23..37e718d7d 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/operations/SqsTemplate.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/operations/SqsTemplate.java @@ -15,10 +15,7 @@ */ package io.awspring.cloud.sqs.operations; -import io.awspring.cloud.sqs.FifoUtils; -import io.awspring.cloud.sqs.MessageHeaderUtils; -import io.awspring.cloud.sqs.QueueAttributesResolver; -import io.awspring.cloud.sqs.SqsAcknowledgementException; +import io.awspring.cloud.sqs.*; import io.awspring.cloud.sqs.listener.QueueAttributes; import io.awspring.cloud.sqs.listener.QueueNotFoundStrategy; import io.awspring.cloud.sqs.listener.SqsHeaders; @@ -29,6 +26,7 @@ import io.awspring.cloud.sqs.support.converter.MessagingMessageConverter; import io.awspring.cloud.sqs.support.converter.SqsMessageConversionContext; import io.awspring.cloud.sqs.support.converter.SqsMessagingMessageConverter; +import io.micrometer.observation.ObservationRegistry; import java.time.Duration; import java.util.Collection; import java.util.Collections; @@ -72,6 +70,7 @@ * * @author Tomaz Fernandes * @author Zhong Xi Lu + * @author Mariusz Sondecki * * @since 3.0 */ @@ -96,7 +95,7 @@ public class SqsTemplate extends AbstractMessagingTemplate implements S private final TemplateContentBasedDeduplication contentBasedDeduplication; private SqsTemplate(SqsTemplateBuilderImpl builder) { - super(builder.messageConverter, builder.options); + super(builder.messageConverter, builder.options, builder.observationRegistry); SqsTemplateOptionsImpl options = builder.options; this.sqsAsyncClient = builder.sqsAsyncClient; this.messageAttributeNames = options.messageAttributeNames; @@ -257,7 +256,7 @@ private Map addAdditionalReceiveHeaders(SqsReceiveOptionsImpl op @Override protected org.springframework.messaging.Message preProcessMessageForSend(String endpointToUse, - org.springframework.messaging.Message message) { + org.springframework.messaging.Message message, ContextScopeManager scopeManager) { return message; } @@ -269,9 +268,9 @@ protected Collection> preProcessMes @Override protected CompletableFuture> preProcessMessageForSendAsync( - String endpointToUse, org.springframework.messaging.Message message) { + String endpointToUse, org.springframework.messaging.Message message, ContextScopeManager scopeManager) { return FifoUtils.isFifo(endpointToUse) - ? endpointHasContentBasedDeduplicationEnabled(endpointToUse) + ? endpointHasContentBasedDeduplicationEnabled(endpointToUse, scopeManager) .thenApply(enabled -> enabled ? addMissingFifoSendHeaders(message, Map.of()) : addMissingFifoSendHeaders(message, getRandomDeduplicationIdHeader())) : CompletableFuture.completedFuture(message); @@ -279,9 +278,11 @@ protected CompletableFuture> prePro @Override protected CompletableFuture>> preProcessMessagesForSendAsync( - String endpointToUse, Collection> messages) { + String endpointToUse, Collection> messages, + ContextScopeManager scopeManager) { return FifoUtils.isFifo(endpointToUse) - ? endpointHasContentBasedDeduplicationEnabled(endpointToUse).thenApply(enabled -> messages.stream() + ? endpointHasContentBasedDeduplicationEnabled(endpointToUse, scopeManager).thenApply(enabled -> messages + .stream() .map(message -> enabled ? addMissingFifoSendHeaders(message, Map.of()) : addMissingFifoSendHeaders(message, getRandomDeduplicationIdHeader())) .toList()) @@ -304,22 +305,25 @@ private Map getRandomDeduplicationIdHeader() { return Map.of(MessageSystemAttributes.SQS_MESSAGE_DEDUPLICATION_ID_HEADER, UUID.randomUUID().toString()); } - private CompletableFuture endpointHasContentBasedDeduplicationEnabled(String endpointName) { + private CompletableFuture endpointHasContentBasedDeduplicationEnabled(String endpointName, + ContextScopeManager scopeManager) { return TemplateContentBasedDeduplication.AUTO.equals(this.contentBasedDeduplication) - ? handleAutoDeduplication(endpointName) + ? handleAutoDeduplication(endpointName, scopeManager) : CompletableFuture .completedFuture(contentBasedDeduplication.equals(TemplateContentBasedDeduplication.ENABLED)); } - private CompletableFuture handleAutoDeduplication(String endpointName) { - return getQueueAttributes(endpointName).thenApply(attributes -> Boolean + private CompletableFuture handleAutoDeduplication(String endpointName, ContextScopeManager scopeManager) { + return getQueueAttributes(endpointName, scopeManager).thenApply(attributes -> Boolean .parseBoolean(attributes.getQueueAttribute(QueueAttributeName.CONTENT_BASED_DEDUPLICATION))); } @Override protected CompletableFuture> doSendAsync(String endpointName, Message message, - org.springframework.messaging.Message originalMessage) { - return createSendMessageRequest(endpointName, message).thenCompose(this.sqsAsyncClient::sendMessage) + org.springframework.messaging.Message originalMessage, ContextScopeManager scopeManager) { + return scopeManager + .manageContextWhileComposing(createSendMessageRequest(endpointName, message, scopeManager), + this.sqsAsyncClient::sendMessage) .thenApply(response -> createSendResult(UUID.fromString(response.messageId()), response.sequenceNumber(), endpointName, originalMessage)); } @@ -332,8 +336,9 @@ private SendResult createSendResult(UUID messageId, @Nullable String sequ : Collections.emptyMap()); } - private CompletableFuture createSendMessageRequest(String endpointName, Message message) { - return getQueueAttributes(endpointName) + private CompletableFuture createSendMessageRequest(String endpointName, Message message, + ContextScopeManager scopeManager) { + return getQueueAttributes(endpointName, scopeManager) .thenApply(queueAttributes -> doCreateSendMessageRequest(message, queueAttributes)); } @@ -348,7 +353,8 @@ private SendMessageRequest doCreateSendMessageRequest(Message message, QueueAttr @Override protected CompletableFuture> doSendBatchAsync(String endpointName, - Collection messages, Collection> originalMessages) { + Collection messages, Collection> originalMessages, + ContextScopeManager scopeManager) { logger.debug("Sending messages {} to endpoint {}", messages, endpointName); return createSendMessageBatchRequest(endpointName, messages).thenCompose(this.sqsAsyncClient::sendMessageBatch) .thenApply(response -> createSendResultBatch(response, endpointName, @@ -391,13 +397,13 @@ private org.springframework.messaging.Message getOriginalMessage( @Nullable @Override protected MessageConversionContext getReceiveMessageConversionContext(String endpointName, - @Nullable Class payloadClass) { + @Nullable Class payloadClass, ContextScopeManager scopeManager) { return this.conversionContextCache.computeIfAbsent(endpointName, - newEndpoint -> doGetSqsMessageConversionContext(endpointName, payloadClass)); + newEndpoint -> doGetSqsMessageConversionContext(endpointName, payloadClass, scopeManager)); } private SqsMessageConversionContext doGetSqsMessageConversionContext(String newEndpoint, - @Nullable Class payloadClass) { + @Nullable Class payloadClass, ContextScopeManager scopeManager) { SqsMessageConversionContext conversionContext = new SqsMessageConversionContext(); conversionContext.setSqsAsyncClient(this.sqsAsyncClient); // At this point we'll already have retrieved and cached the queue attributes @@ -405,7 +411,7 @@ private SqsMessageConversionContext doGetSqsMessageConversionContext(String if (payloadClass != null) { conversionContext.setPayloadClass(payloadClass); } - conversionContext.setAcknowledgementCallback(new TemplateAcknowledgementCallback()); + conversionContext.setAcknowledgementCallback(new TemplateAcknowledgementCallback(scopeManager)); return conversionContext; } @@ -465,15 +471,21 @@ private boolean isSkipAttribute(MessageSystemAttributeName name) { } private CompletableFuture getQueueAttributes(String endpointName) { + return getQueueAttributes(endpointName, null); + } + + private CompletableFuture getQueueAttributes(String endpointName, + @Nullable ContextScopeManager scopeManager) { return this.queueAttributesCache.computeIfAbsent(endpointName, - newName -> doGetQueueAttributes(endpointName, newName)); + newName -> doGetQueueAttributes(endpointName, newName, scopeManager)); } - private CompletableFuture doGetQueueAttributes(String endpointName, String newName) { + private CompletableFuture doGetQueueAttributes(String endpointName, String newName, + @Nullable ContextScopeManager scopeManager) { return QueueAttributesResolver.builder().sqsAsyncClient(this.sqsAsyncClient).queueName(newName) .queueNotFoundStrategy(this.queueNotFoundStrategy) .queueAttributeNames(maybeAddContentBasedDeduplicationAttribute(endpointName)).build() - .resolveQueueAttributes(); + .resolveQueueAttributes(scopeManager); } private Collection maybeAddContentBasedDeduplicationAttribute(String endpointName) { @@ -494,18 +506,20 @@ protected Map handleAdditionalHeaders(Map additi @Override protected CompletableFuture doAcknowledgeMessages(String endpointName, - Collection> messages) { - return deleteMessages(endpointName, messages); + Collection> messages, ContextScopeManager scopeManager) { + return deleteMessages(endpointName, messages, scopeManager); } @Override protected CompletableFuture> doReceiveAsync(String endpointName, Duration pollTimeout, - Integer maxNumberOfMessages, Map additionalHeaders) { + Integer maxNumberOfMessages, Map additionalHeaders, ContextScopeManager scopeManager) { logger.trace( "Receiving messages with settings: endpointName - {}, pollTimeout - {}, maxNumberOfMessages - {}, additionalHeaders - {}", endpointName, pollTimeout, maxNumberOfMessages, additionalHeaders); - return createReceiveMessageRequest(endpointName, pollTimeout, maxNumberOfMessages, additionalHeaders) - .thenCompose(this.sqsAsyncClient::receiveMessage).thenApply(ReceiveMessageResponse::messages); + return scopeManager + .manageContextWhileComposing(createReceiveMessageRequest(endpointName, pollTimeout, maxNumberOfMessages, + additionalHeaders, scopeManager), this.sqsAsyncClient::receiveMessage) + .thenApply(ReceiveMessageResponse::messages); } @Override @@ -519,11 +533,11 @@ private Map addMissingFifoReceiveHeaders(Map hea } private CompletableFuture deleteMessages(String endpointName, - Collection> messages) { + Collection> messages, ContextScopeManager scopeManager) { logger.trace("Acknowledging in queue {} messages {}", endpointName, MessageHeaderUtils.getId(addTypeToMessages(messages))); - return getQueueAttributes(endpointName) - .thenCompose(attributes -> this.sqsAsyncClient.deleteMessageBatch(DeleteMessageBatchRequest.builder() + return scopeManager.manageContextWhileComposing(getQueueAttributes(endpointName, scopeManager), + attributes -> this.sqsAsyncClient.deleteMessageBatch(DeleteMessageBatchRequest.builder() .queueUrl(attributes.getQueueUrl()).entries(createDeleteMessageEntries(messages)).build())) .exceptionallyCompose( t -> createAcknowledgementException(endpointName, Collections.emptyList(), messages, t)) @@ -592,9 +606,11 @@ private Collection createDeleteMessageEntries( } private CompletableFuture createReceiveMessageRequest(String endpointName, - Duration pollTimeout, Integer maxNumberOfMessages, Map additionalHeaders) { - return getQueueAttributes(endpointName).thenApply(attributes -> doCreateReceiveMessageRequest(pollTimeout, - maxNumberOfMessages, attributes, additionalHeaders)); + Duration pollTimeout, Integer maxNumberOfMessages, Map additionalHeaders, + ContextScopeManager scopeManager) { + return getQueueAttributes(endpointName, scopeManager) + .thenApply(attributes -> doCreateReceiveMessageRequest(pollTimeout, maxNumberOfMessages, attributes, + additionalHeaders)); } private ReceiveMessageRequest doCreateReceiveMessageRequest(Duration pollTimeout, Integer maxNumberOfMessages, @@ -688,6 +704,8 @@ private static class SqsTemplateBuilderImpl implements SqsTemplateBuilder { private MessagingMessageConverter messageConverter; + private ObservationRegistry observationRegistry; + private SqsTemplateBuilderImpl() { this.options = new SqsTemplateOptionsImpl(); } @@ -725,12 +743,22 @@ public SqsTemplateBuilder configure(Consumer options) { return this; } + @Override + public SqsTemplateBuilder observationRegistry(ObservationRegistry observationRegistry) { + Assert.notNull(observationRegistry, "observationRegistry must not be null"); + this.observationRegistry = observationRegistry; + return this; + } + @Override public SqsTemplate build() { Assert.notNull(this.sqsAsyncClient, "no sqsAsyncClient set"); if (this.messageConverter == null) { this.messageConverter = createDefaultMessageConverter(); } + if (this.observationRegistry == null) { + this.observationRegistry = ObservationRegistry.NOOP; + } return new SqsTemplate(this); } @@ -892,10 +920,16 @@ public SqsReceiveOptionsImpl receiveRequestAttemptId(UUID receiveRequestAttemptI private class TemplateAcknowledgementCallback implements AcknowledgementCallback { + private final ContextScopeManager scopeManager; + + TemplateAcknowledgementCallback(ContextScopeManager scopeManager) { + this.scopeManager = scopeManager; + } + @Override public CompletableFuture onAcknowledge(org.springframework.messaging.Message message) { return deleteMessages(MessageHeaderUtils.getHeaderAsString(message, SqsHeaders.SQS_QUEUE_NAME_HEADER), - Collections.singletonList(message)); + Collections.singletonList(message), scopeManager); } @Override @@ -905,7 +939,8 @@ public CompletableFuture onAcknowledge(Collection (org.springframework.messaging.Message) msg) - .collect(Collectors.toList())); + .collect(Collectors.toList()), + scopeManager); } } diff --git a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/operations/SqsTemplateBuilder.java b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/operations/SqsTemplateBuilder.java index 0c2e924d6..b1c782220 100644 --- a/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/operations/SqsTemplateBuilder.java +++ b/spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/operations/SqsTemplateBuilder.java @@ -17,6 +17,7 @@ import io.awspring.cloud.sqs.support.converter.MessagingMessageConverter; import io.awspring.cloud.sqs.support.converter.SqsMessagingMessageConverter; +import io.micrometer.observation.ObservationRegistry; import java.util.function.Consumer; import software.amazon.awssdk.services.sqs.SqsAsyncClient; import software.amazon.awssdk.services.sqs.model.Message; @@ -82,4 +83,11 @@ public interface SqsTemplateBuilder { */ SqsOperations buildSyncTemplate(); + /** + * Set the {@link ObservationRegistry} to be used by the {@link SqsTemplate}. + * + * @param observationRegistry the instance. + * @return the builder. + */ + SqsTemplateBuilder observationRegistry(ObservationRegistry observationRegistry); } diff --git a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/config/SqsMessageListenerContainerFactoryTests.java b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/config/SqsMessageListenerContainerFactoryTests.java index 27bb9293d..b1d0cac6a 100644 --- a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/config/SqsMessageListenerContainerFactoryTests.java +++ b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/config/SqsMessageListenerContainerFactoryTests.java @@ -32,6 +32,8 @@ import io.awspring.cloud.sqs.listener.errorhandler.ErrorHandler; import io.awspring.cloud.sqs.listener.interceptor.AsyncMessageInterceptor; import io.awspring.cloud.sqs.listener.interceptor.MessageInterceptor; +import io.micrometer.observation.ObservationRegistry; +import io.micrometer.observation.tck.TestObservationRegistry; import java.time.Duration; import java.util.Collections; import java.util.List; @@ -73,6 +75,7 @@ void shouldCreateContainerFromEndpointWithOptionsDefaults() { assertThat(container.getId()).isEqualTo(id); assertThat(container.getQueueNames()).containsExactlyElementsOf(queueNames); + assertThat(container.getObservationRegistry()).isEqualTo(ObservationRegistry.NOOP); } @@ -82,6 +85,7 @@ void shouldCreateContainerFromEndpointOverridingOptions() { String id = "test-id"; SqsAsyncClient client = mock(SqsAsyncClient.class); SqsEndpoint endpoint = mock(SqsEndpoint.class); + TestObservationRegistry observationRegistry = TestObservationRegistry.create(); int inflight = 9; int messagesPerPoll = 7; Duration pollTimeout = Duration.ofSeconds(6); @@ -95,6 +99,7 @@ void shouldCreateContainerFromEndpointOverridingOptions() { SqsMessageListenerContainerFactory factory = new SqsMessageListenerContainerFactory<>(); factory.setSqsAsyncClient(client); + factory.setObservationRegistry(observationRegistry); SqsMessageListenerContainer container = factory.createContainer(endpoint); assertThat(container.getContainerOptions()).isInstanceOfSatisfying(SqsContainerOptions.class, options -> { @@ -106,6 +111,7 @@ void shouldCreateContainerFromEndpointOverridingOptions() { assertThat(container.getId()).isEqualTo(id); assertThat(container.getQueueNames()).containsExactlyElementsOf(queueNames); + assertThat(container.getObservationRegistry()).isEqualTo(observationRegistry); } @@ -118,13 +124,15 @@ void shouldCreateFromBuilderWithBlockingComponents() { MessageInterceptor interceptor2 = mock(MessageInterceptor.class); AcknowledgementResultCallback callback = mock(AcknowledgementResultCallback.class); ContainerComponentFactory componentFactory = mock(ContainerComponentFactory.class); + TestObservationRegistry observationRegistry = TestObservationRegistry.create(); List> componentFactories = Collections .singletonList(componentFactory); SqsMessageListenerContainerFactory factory = SqsMessageListenerContainerFactory.builder() .messageListener(listener).sqsAsyncClient(client).errorHandler(errorHandler) .containerComponentFactories(componentFactories).acknowledgementResultCallback(callback) - .messageInterceptor(interceptor1).messageInterceptor(interceptor2).build(); + .messageInterceptor(interceptor1).messageInterceptor(interceptor2) + .observationRegistry(observationRegistry).build(); assertThat(factory).extracting("messageListener").isEqualTo(listener); assertThat(factory).extracting("errorHandler").isEqualTo(errorHandler); @@ -134,6 +142,7 @@ void shouldCreateFromBuilderWithBlockingComponents() { assertThat(factory).extracting("containerComponentFactories").isEqualTo(componentFactories); assertThat(factory).extracting("sqsAsyncClientSupplier").asInstanceOf(type(Supplier.class)) .extracting(Supplier::get).isEqualTo(client); + assertThat(factory).extracting("observationRegistry").isEqualTo(observationRegistry); } @Test @@ -145,19 +154,22 @@ void shouldCreateFromBuilderWithAsyncComponents() { AsyncMessageInterceptor interceptor2 = mock(AsyncMessageInterceptor.class); AsyncAcknowledgementResultCallback callback = mock(AsyncAcknowledgementResultCallback.class); ContainerComponentFactory componentFactory = mock(ContainerComponentFactory.class); + TestObservationRegistry observationRegistry = TestObservationRegistry.create(); List> componentFactories = Collections .singletonList(componentFactory); SqsMessageListenerContainerFactory container = SqsMessageListenerContainerFactory.builder() .asyncMessageListener(listener).sqsAsyncClient(client).errorHandler(errorHandler) .containerComponentFactories(componentFactories).acknowledgementResultCallback(callback) - .messageInterceptor(interceptor1).messageInterceptor(interceptor2).build(); + .messageInterceptor(interceptor1).messageInterceptor(interceptor2) + .observationRegistry(observationRegistry).build(); assertThat(container).extracting("asyncMessageListener").isEqualTo(listener); assertThat(container).extracting("asyncErrorHandler").isEqualTo(errorHandler); assertThat(container).extracting("asyncAcknowledgementResultCallback").isEqualTo(callback); assertThat(container).extracting("asyncMessageInterceptors") .asInstanceOf(collection(AsyncMessageInterceptor.class)).containsExactly(interceptor1, interceptor2); + assertThat(container).extracting("observationRegistry").isEqualTo(observationRegistry); } } diff --git a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/integration/SqsIntegrationTests.java b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/integration/SqsIntegrationTests.java index 50bded839..88abd158f 100644 --- a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/integration/SqsIntegrationTests.java +++ b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/integration/SqsIntegrationTests.java @@ -49,6 +49,11 @@ import io.awspring.cloud.sqs.listener.source.AbstractSqsMessageSource; import io.awspring.cloud.sqs.listener.source.MessageSource; import io.awspring.cloud.sqs.operations.SqsTemplate; +import io.micrometer.observation.Observation; +import io.micrometer.observation.ObservationRegistry; +import io.micrometer.observation.tck.ObservationContextAssert; +import io.micrometer.observation.tck.TestObservationRegistry; +import io.micrometer.observation.tck.TestObservationRegistryAssert; import java.lang.reflect.Method; import java.time.Duration; import java.util.ArrayList; @@ -93,6 +98,7 @@ * @author Mikhail Strokov * @author Michael Sosa * @author gustavomonarin + * @author Mariusz Sondecki */ @SpringBootTest @TestPropertySource(properties = { "property.one=1", "property.five.seconds=5s", @@ -130,6 +136,8 @@ class SqsIntegrationTests extends BaseSqsIntegrationTest { static final String MAX_CONCURRENT_MESSAGES_QUEUE_NAME = "max_concurrent_messages_test_queue"; + static final String OBSERVED_MESSAGE_QUEUE_NAME = "observed_message_test_queue"; + static final String LOW_RESOURCE_FACTORY = "lowResourceFactory"; static final String MANUAL_ACK_FACTORY = "manualAcknowledgementFactory"; @@ -138,6 +146,8 @@ class SqsIntegrationTests extends BaseSqsIntegrationTest { static final String ACK_AFTER_SECOND_ERROR_FACTORY = "ackAfterSecondErrorFactory"; + static final String OBSERVED_MESSAGE_FACTORY = "observedMessageFactory"; + @BeforeAll static void beforeTests() { SqsAsyncClient client = createAsyncClient(); @@ -174,6 +184,9 @@ static void beforeTests() { @Qualifier("inactiveContainer") MessageListenerContainer inactiveMessageListenerContainer; + @Autowired + TestObservationRegistry observationRegistry; + @Test void receivesMessage() throws Exception { String messageBody = "receivesMessage-payload"; @@ -349,6 +362,24 @@ void maxConcurrentMessages() { assertDoesNotThrow(() -> latchContainer.maxConcurrentMessagesBarrier.await(10, TimeUnit.SECONDS)); } + @Test + void receivesObservedMessage() throws Exception { + observationRegistry.clear(); + Message message = MessageBuilder.withPayload("receivesObservedMessage-payload").build(); + sqsTemplate.send(OBSERVED_MESSAGE_QUEUE_NAME, message); + logger.debug("Sent message to queue {} with message {}", OBSERVED_MESSAGE_QUEUE_NAME, message); + assertThat(latchContainer.observedMessageLatch.await(10, TimeUnit.SECONDS)).isTrue(); + + TestObservationRegistryAssert.then(observationRegistry).hasNumberOfObservationsEqualTo(2) + .hasHandledContextsThatSatisfy(contexts -> { + ObservationContextAssert.then(contexts.get(0)).hasNameEqualTo("sqs.single.message.polling.process") + .doesNotHaveParentObservation(); + ObservationContextAssert.then(contexts.get(1)).hasNameEqualTo("listener.process") + .hasParentObservationContextMatching( + contextView -> contextView.getName().equals("sqs.single.message.polling.process")); + }); + } + static class ReceivesMessageListener { @Autowired @@ -497,6 +528,23 @@ void listen(String message) throws BrokenBarrierException, InterruptedException } } + static class ObservedMessageListener { + + @Autowired + LatchContainer latchContainer; + + @Autowired + ObservationRegistry observationRegistry; + + @SqsListener(queueNames = OBSERVED_MESSAGE_QUEUE_NAME, factory = OBSERVED_MESSAGE_FACTORY, id = "observedMessageContainer") + void listen(String message) { + Observation.createNotStarted("listener.process", observationRegistry).observe(() -> { + logger.debug("Observed message in Listener Method: {}", message); + latchContainer.observedMessageLatch.countDown(); + }); + } + } + static class LatchContainer { final CountDownLatch receivesMessageLatch = new CountDownLatch(1); @@ -521,6 +569,7 @@ static class LatchContainer { final CountDownLatch acknowledgementCallbackErrorLatch = new CountDownLatch(1); final CountDownLatch manuallyInactiveCreatedContainerLatch = new CountDownLatch(1); final CyclicBarrier maxConcurrentMessagesBarrier = new CyclicBarrier(21); + final CountDownLatch observedMessageLatch = new CountDownLatch(1); } @@ -542,6 +591,18 @@ public SqsMessageListenerContainerFactory defaultSqsListenerContainerFac .build(); } + @Bean(OBSERVED_MESSAGE_FACTORY) + public SqsMessageListenerContainerFactory observedMessageFactory() { + return SqsMessageListenerContainerFactory + .builder() + .sqsAsyncClientSupplier(BaseSqsIntegrationTest::createAsyncClient) + .observationRegistry(observationRegistry()) + .configure(options -> options + .maxDelayBetweenPolls(Duration.ofSeconds(5)) + .pollTimeout(Duration.ofSeconds(5))) + .build(); + } + @Bean(name = LOW_RESOURCE_FACTORY) public SqsMessageListenerContainerFactory lowResourceFactory() { return SqsMessageListenerContainerFactory @@ -758,6 +819,11 @@ MaxConcurrentMessagesListener maxConcurrentMessagesListener() { return new MaxConcurrentMessagesListener(); } + @Bean + ObservedMessageListener observedMessageListener() { + return new ObservedMessageListener(); + } + @Bean SqsListenerConfigurer customizer() { return registrar -> { @@ -786,6 +852,11 @@ SqsTemplate sqsTemplate() { return SqsTemplate.builder().sqsAsyncClient(BaseSqsIntegrationTest.createAsyncClient()).build(); } + @Bean + TestObservationRegistry observationRegistry() { + return TestObservationRegistry.create(); + } + private AsyncMessageInterceptor testInterceptor() { return new AsyncMessageInterceptor() { @Override diff --git a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/integration/SqsObservationIntegrationTests.java b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/integration/SqsObservationIntegrationTests.java new file mode 100644 index 000000000..19f779a26 --- /dev/null +++ b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/integration/SqsObservationIntegrationTests.java @@ -0,0 +1,358 @@ +/* + * Copyright 2013-2022 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.integration; + +import static org.assertj.core.api.Assertions.assertThat; + +import io.awspring.cloud.sqs.CompletableFutures; +import io.awspring.cloud.sqs.MessageHeaderUtils; +import io.awspring.cloud.sqs.annotation.SqsListener; +import io.awspring.cloud.sqs.config.SqsBootstrapConfiguration; +import io.awspring.cloud.sqs.config.SqsMessageListenerContainerFactory; +import io.awspring.cloud.sqs.listener.SqsHeaders; +import io.awspring.cloud.sqs.listener.StandardSqsComponentFactory; +import io.awspring.cloud.sqs.listener.acknowledgement.BatchingAcknowledgementProcessor; +import io.awspring.cloud.sqs.listener.acknowledgement.handler.AcknowledgementMode; +import io.awspring.cloud.sqs.listener.errorhandler.ErrorHandler; +import io.awspring.cloud.sqs.listener.interceptor.AsyncMessageInterceptor; +import io.awspring.cloud.sqs.listener.interceptor.MessageInterceptor; +import io.awspring.cloud.sqs.operations.SqsTemplate; +import io.awspring.cloud.sqs.support.converter.MessagingMessageHeaders; +import io.micrometer.observation.Observation; +import io.micrometer.observation.ObservationRegistry; +import io.micrometer.observation.tck.ObservationContextAssert; +import io.micrometer.observation.tck.TestObservationRegistry; +import io.micrometer.observation.tck.TestObservationRegistryAssert; +import java.time.Duration; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.UUID; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.TimeUnit; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; +import org.springframework.messaging.Message; +import org.springframework.messaging.handler.annotation.Header; +import org.springframework.messaging.support.MessageBuilder; +import software.amazon.awssdk.services.sqs.SqsAsyncClient; +import software.amazon.awssdk.services.sqs.model.QueueAttributeName; + +/** + * Integration tests for the SQS Observation API. + * + * @author Mariusz Sondecki + */ +@SpringBootTest +class SqsObservationIntegrationTests extends BaseSqsIntegrationTest { + + private static final Logger logger = LoggerFactory.getLogger(SqsObservationIntegrationTests.class); + + static final String RECEIVES_CHANGED_MESSAGE_ON_COMPONENTS_QUEUE_NAME = "observability_on_components_test_queue"; + + static final String RECEIVES_CHANGED_MESSAGE_ON_ERROR_QUEUE_NAME = "observability_on_error_test_queue"; + + static final String SHOULD_CHANGE_PAYLOAD = "should-change-payload"; + + private static final String CHANGED_PAYLOAD = "Changed payload"; + + private static final UUID CHANGED_ID = UUID.fromString("0009f2c8-1dc4-e211-cc99-9f9c62b5e66b"); + + @Autowired + LatchContainer latchContainer; + + @Autowired + SqsTemplate sqsTemplate; + + @Autowired(required = false) + ReceivesChangedPayloadOnErrorListener receivesChangedPayloadOnErrorListener; + + @Autowired + TestObservationRegistry observationRegistry; + + @BeforeAll + static void beforeTests() { + SqsAsyncClient client = createAsyncClient(); + CompletableFuture.allOf(createQueue(client, RECEIVES_CHANGED_MESSAGE_ON_COMPONENTS_QUEUE_NAME), + createQueue(client, RECEIVES_CHANGED_MESSAGE_ON_ERROR_QUEUE_NAME)).join(); + } + + @Test + @DisplayName("Should correctly instrument and propagate observations across different threads") + void shouldInstrumentObservationsAcrossThreads() throws Exception { + sqsTemplate.send(RECEIVES_CHANGED_MESSAGE_ON_ERROR_QUEUE_NAME, SHOULD_CHANGE_PAYLOAD); + assertThat(latchContainer.receivesChangedMessageOnErrorLatch.await(10, TimeUnit.SECONDS)).isTrue(); + assertThat(receivesChangedPayloadOnErrorListener.receivedMessages).containsExactly(CHANGED_PAYLOAD); + assertThat(latchContainer.receivesChangedMessageLatch.await(10, TimeUnit.SECONDS)).isTrue(); + TestObservationRegistryAssert.then(observationRegistry).hasNumberOfObservationsEqualTo(9) + .hasHandledContextsThatSatisfy(contexts -> { + ObservationContextAssert.then(contexts.get(0)).hasNameEqualTo("sqs.single.message.publish") + .doesNotHaveParentObservation(); + + ObservationContextAssert.then(contexts.get(1)).hasNameEqualTo("sqs.single.message.polling.process") + .doesNotHaveParentObservation(); + ObservationContextAssert.then(contexts.get(2)).hasNameEqualTo("blockingInterceptor.intercept") + .hasParentObservationContextMatching( + contextView -> contextView.getName().equals("sqs.single.message.polling.process")); + ObservationContextAssert.then(contexts.get(3)).hasNameEqualTo("listener.listen") + .hasParentObservationContextMatching( + contextView -> contextView.getName().equals("sqs.single.message.polling.process")); + ObservationContextAssert.then(contexts.get(4)).hasNameEqualTo("sqs.single.message.publish") + .hasParentObservationContextMatching( + contextView -> contextView.getName().equals("sqs.single.message.polling.process")); + ObservationContextAssert.then(contexts.get(5)).hasNameEqualTo("errorHandler.handle") + .hasParentObservationContextMatching( + contextView -> contextView.getName().equals("sqs.single.message.polling.process")); + + ObservationContextAssert.then(contexts.get(6)).hasNameEqualTo("asyncInterceptor1.afterProcessing") + .hasParentObservationContextMatching( + contextView -> contextView.getName().equals("sqs.single.message.polling.process")); + ObservationContextAssert.then(contexts.get(7)).hasNameEqualTo("sqs.single.message.manual.process") + .hasParentObservationContextMatching( + contextView -> contextView.getName().equals("asyncInterceptor1.afterProcessing")); + ObservationContextAssert.then(contexts.get(8)).hasNameEqualTo("asyncInterceptor2.afterProcessing") + .hasParentObservationContextMatching( + contextView -> contextView.getName().equals("asyncInterceptor1.afterProcessing")); + }); + } + + static class ReceivesChangedPayloadOnErrorListener { + + @Autowired + LatchContainer latchContainer; + + @Autowired + SqsTemplate sqsTemplate; + + @Autowired + ObservationRegistry observationRegistry; + + Collection receivedMessages = Collections.synchronizedList(new ArrayList<>()); + + @SqsListener(queueNames = RECEIVES_CHANGED_MESSAGE_ON_ERROR_QUEUE_NAME, id = "receives-changed-payload-on-error") + CompletableFuture listen(Message message, + @Header(SqsHeaders.SQS_QUEUE_NAME_HEADER) String queueName) { + Observation.createNotStarted("listener.listen", observationRegistry).observe(() -> { + logger.debug("Received message {} with id {} from queue {}", message.getPayload(), + MessageHeaderUtils.getId(message), queueName); + if (isChangedPayload(message)) { + receivedMessages.add(message.getPayload()); + latchContainer.receivesChangedMessageOnErrorLatch.countDown(); + } + }); + return sqsTemplate.sendAsync(RECEIVES_CHANGED_MESSAGE_ON_COMPONENTS_QUEUE_NAME, message.getPayload()) + .thenAccept(result -> logger.debug("Sent message by SqsTemplate#sendAsync: {}", result.messageId())) + .thenCompose(result -> CompletableFutures.failedFuture( + new RuntimeException("Expected exception from receives-changed-payload-on-error"))); + } + } + + static class BlockingErrorHandler implements ErrorHandler { + + @Autowired + LatchContainer latchContainer; + + @Autowired + ObservationRegistry observationRegistry; + + @Override + public void handle(Message message, Throwable t) { + Observation.createNotStarted("errorHandler.handle", observationRegistry).observe(() -> { + logger.debug("BlockingErrorHandler - handle: {}", MessageHeaderUtils.getId(message)); + if (isChangedPayload(message)) { + latchContainer.receivesChangedMessageOnErrorLatch.countDown(); + } + }); + } + } + + static class ReceivesChangedMessageInterceptor implements AsyncMessageInterceptor { + + @Autowired + LatchContainer latchContainer; + + @Autowired + SqsTemplate sqsTemplate; + + @Autowired + ObservationRegistry observationRegistry; + + @Override + public CompletableFuture afterProcessing(Message message, Throwable t) { + return Observation.createNotStarted("asyncInterceptor1.afterProcessing", observationRegistry) + .observe(() -> { + logger.debug("ReceivesChangedMessageInterceptor - afterProcessing: {}", + MessageHeaderUtils.getId(message)); + return sqsTemplate.receiveAsync(RECEIVES_CHANGED_MESSAGE_ON_COMPONENTS_QUEUE_NAME, String.class) + .thenAccept(result -> { + logger.debug("Received message with SqsTemplate#receiveAsync: {}", + result.map(MessageHeaderUtils::getId).orElse("empty")); + result.ifPresent(msg -> latchContainer.receivesChangedMessageLatch.countDown()); + }); + }); + } + } + + static class ReceivesChangedPayloadOnErrorInterceptor implements AsyncMessageInterceptor { + + @Autowired + LatchContainer latchContainer; + + @Autowired + ObservationRegistry observationRegistry; + + @Override + public CompletableFuture afterProcessing(Message message, Throwable t) { + return Observation.createNotStarted("asyncInterceptor2.afterProcessing", observationRegistry) + .observe(() -> { + logger.debug("ReceivesChangedPayloadOnErrorInterceptor - afterProcessing: {}", + MessageHeaderUtils.getId(message)); + if (isChangedPayload(message)) { + latchContainer.receivesChangedMessageOnErrorLatch.countDown(); + } + return CompletableFuture.completedFuture(null); + }); + } + } + + static class BlockingInterceptor implements MessageInterceptor { + + @Autowired + ObservationRegistry observationRegistry; + + @Override + public Message intercept(Message message) { + return Observation.createNotStarted("blockingInterceptor.intercept", observationRegistry).observe(() -> { + logger.debug("BlockingInterceptor - intercept: {}", MessageHeaderUtils.getId(message)); + if (message.getPayload().equals(SHOULD_CHANGE_PAYLOAD)) { + MessagingMessageHeaders headers = new MessagingMessageHeaders(message.getHeaders(), CHANGED_ID); + return MessageBuilder.createMessage(CHANGED_PAYLOAD, headers); + } + return message; + }); + } + } + + static class LatchContainer { + + final CountDownLatch receivesChangedMessageLatch = new CountDownLatch(1); + + final CountDownLatch receivesChangedMessageOnErrorLatch = new CountDownLatch(3); + + } + + @Import(SqsBootstrapConfiguration.class) + @Configuration + static class SQSConfiguration { + + LatchContainer latchContainer = new LatchContainer(); + + // @formatter:off + @Bean + SqsMessageListenerContainerFactory defaultSqsListenerContainerFactory() { + SqsMessageListenerContainerFactory factory = new SqsMessageListenerContainerFactory<>(); + factory.configure(options -> options + .maxDelayBetweenPolls(Duration.ofSeconds(1)) + .queueAttributeNames(Collections.singletonList(QueueAttributeName.QUEUE_ARN)) + .acknowledgementMode(AcknowledgementMode.ALWAYS) + .pollTimeout(Duration.ofSeconds(3))); + factory.setSqsAsyncClientSupplier(BaseSqsIntegrationTest::createAsyncClient); + factory.setObservationRegistry(observationRegistry()); + factory.addMessageInterceptor(asyncInterceptor1()); + factory.addMessageInterceptor(asyncInterceptor2()); + factory.addMessageInterceptor(blockingInterceptor()); + factory.setErrorHandler(blockErrorHandler()); + factory.setContainerComponentFactories(Collections.singletonList(getContainerComponentFactory())); + return factory; + } + // @formatter:on + + @Bean + BlockingErrorHandler blockErrorHandler() { + return new BlockingErrorHandler(); + } + + @Bean + ReceivesChangedPayloadOnErrorListener receivesChangedPayloadOnErrorListener() { + return new ReceivesChangedPayloadOnErrorListener(); + } + + @Bean + LatchContainer latchContainer() { + return this.latchContainer; + } + + @Bean + TestObservationRegistry observationRegistry() { + return TestObservationRegistry.create(); + } + + @Bean + SqsTemplate sqsTemplate() { + return SqsTemplate.builder().observationRegistry(observationRegistry()) + .sqsAsyncClient(BaseSqsIntegrationTest.createAsyncClient()).build(); + } + + @Bean + ReceivesChangedMessageInterceptor asyncInterceptor1() { + return new ReceivesChangedMessageInterceptor(); + } + + @Bean + ReceivesChangedPayloadOnErrorInterceptor asyncInterceptor2() { + return new ReceivesChangedPayloadOnErrorInterceptor(); + } + + @Bean + BlockingInterceptor blockingInterceptor() { + return new BlockingInterceptor(); + } + + private StandardSqsComponentFactory getContainerComponentFactory() { + return new StandardSqsComponentFactory<>() { + @Override + protected BatchingAcknowledgementProcessor createBatchingProcessorInstance() { + return new BatchingAcknowledgementProcessor<>() { + @Override + protected CompletableFuture sendToExecutor(Collection> messagesToAck) { + return super.sendToExecutor(messagesToAck).whenComplete((v, t) -> { + if (messagesToAck.stream().allMatch(SqsObservationIntegrationTests::isChangedPayload)) { + latchContainer.receivesChangedMessageOnErrorLatch.countDown(); + } + }); + } + }; + } + }; + } + + } + + private static boolean isChangedPayload(Message message) { + return message != null && message.getPayload().equals(CHANGED_PAYLOAD) + && MessageHeaderUtils.getId(message).equals(CHANGED_ID.toString()); + } + +} diff --git a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/integration/SqsTemplateIntegrationTests.java b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/integration/SqsTemplateIntegrationTests.java index 9ed76639a..82f5819da 100644 --- a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/integration/SqsTemplateIntegrationTests.java +++ b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/integration/SqsTemplateIntegrationTests.java @@ -17,10 +17,23 @@ import static org.assertj.core.api.Assertions.assertThat; +import brave.Tracing; +import brave.handler.MutableSpan; +import brave.test.TestSpanHandler; import io.awspring.cloud.sqs.listener.SqsHeaders; import io.awspring.cloud.sqs.listener.acknowledgement.Acknowledgement; import io.awspring.cloud.sqs.operations.*; import io.awspring.cloud.sqs.support.converter.AbstractMessagingMessageConverter; +import io.micrometer.context.ContextRegistry; +import io.micrometer.observation.Observation; +import io.micrometer.observation.ObservationRegistry; +import io.micrometer.tracing.brave.bridge.BraveBaggageManager; +import io.micrometer.tracing.brave.bridge.BraveCurrentTraceContext; +import io.micrometer.tracing.brave.bridge.BravePropagator; +import io.micrometer.tracing.brave.bridge.BraveTracer; +import io.micrometer.tracing.contextpropagation.ObservationAwareSpanThreadLocalAccessor; +import io.micrometer.tracing.handler.DefaultTracingObservationHandler; +import io.micrometer.tracing.handler.PropagatingSenderTracingObservationHandler; import java.time.Duration; import java.util.*; import java.util.concurrent.CompletableFuture; @@ -42,6 +55,7 @@ /** * @author Tomaz Fernandes * @author Dongha Kim + * @author Mariusz Sondecki */ @SpringBootTest public class SqsTemplateIntegrationTests extends BaseSqsIntegrationTest { @@ -162,6 +176,54 @@ void shouldSendAndReceiveMessageWithHeaders() { .containsValues(myCustomValue, myCustomValue2, myCustomValue3); } + @Test + void shouldSendAndReceiveMessageWithTracingContext() { + SampleRecord testRecord = new SampleRecord("Hello world!", + "From shouldSendAndReceiveMessageWithTracingContext!"); + String parentSpanName = "Parent span"; + + ObservationRegistry registry = ObservationRegistry.create(); + SqsOperations template = SqsTemplate.builder().sqsAsyncClient(this.asyncClient).observationRegistry(registry) + .build(); + TestSpanHandler testSpanHandler = new TestSpanHandler(); + Tracing tracing = Tracing.newBuilder().addSpanHandler(testSpanHandler).build(); + io.micrometer.tracing.Tracer tracer = new BraveTracer(tracing.tracer(), + new BraveCurrentTraceContext(tracing.currentTraceContext()), new BraveBaggageManager()); + ContextRegistry.getInstance().loadThreadLocalAccessors() + .registerThreadLocalAccessor(new ObservationAwareSpanThreadLocalAccessor(tracer)); + BravePropagator propagator = new BravePropagator(tracing); + registry.observationConfig() + .observationHandler(new PropagatingSenderTracingObservationHandler<>(tracer, propagator)); + + template.send(to -> to.queue(SENDS_AND_RECEIVES_WITH_HEADERS_QUEUE_NAME).payload(testRecord)); + + registry.observationConfig().observationHandler(new DefaultTracingObservationHandler(tracer)); + + Optional> receivedMessage = Observation.createNotStarted(parentSpanName, registry) + .observe(() -> template.receive(from -> from.queue(SENDS_AND_RECEIVES_WITH_HEADERS_QUEUE_NAME), + SampleRecord.class)); + + List spans = testSpanHandler.spans(); + assertThat(spans).hasSize(3); + String publishTraceId = spans.get(0).traceId(); + String publishSpanId = spans.get(0).id(); + String processTraceId = spans.get(1).traceId(); + String processSpanId = spans.get(1).id(); + String processParentSpanId = spans.get(1).parentId(); + String parentTraceId = spans.get(2).traceId(); + String parentSpanId = spans.get(2).id(); + String parentParentSpanId = spans.get(2).parentId(); + + assertThat(receivedMessage).isPresent().get().extracting(Message::getHeaders) + .asInstanceOf(InstanceOfAssertFactories.MAP).containsEntry("X-B3-TraceId", publishTraceId) + .containsEntry("X-B3-SpanId", publishSpanId); + + assertThat(processTraceId).isNotEqualTo(publishTraceId).isEqualTo(parentTraceId); + assertThat(processSpanId).isNotEqualTo(publishSpanId); + assertThat(processParentSpanId).isEqualTo(parentSpanId); + assertThat(parentParentSpanId).isNull(); + } + @Test void shouldSendAndReceiveWithManualAcknowledgement() { SqsTemplate template = SqsTemplate.builder().sqsAsyncClient(this.asyncClient) diff --git a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/AbstractMessageListenerContainerTests.java b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/AbstractMessageListenerContainerTests.java index f7e450286..436920d2a 100644 --- a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/AbstractMessageListenerContainerTests.java +++ b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/AbstractMessageListenerContainerTests.java @@ -24,6 +24,7 @@ import io.awspring.cloud.sqs.listener.errorhandler.ErrorHandler; import io.awspring.cloud.sqs.listener.interceptor.AsyncMessageInterceptor; import io.awspring.cloud.sqs.listener.interceptor.MessageInterceptor; +import io.micrometer.observation.ObservationRegistry; import java.util.Collections; import java.util.List; import org.junit.jupiter.api.Test; @@ -74,6 +75,8 @@ void shouldAdaptBlockingComponents() { .extracting("blockingMessageInterceptor").isEqualTo(interceptor); assertThat(container.getPhase()).isEqualTo(MessageListenerContainer.DEFAULT_PHASE); + + assertThat(container.getObservationRegistry()).isEqualTo(ObservationRegistry.NOOP); } @Test @@ -103,6 +106,7 @@ void shouldSetAsyncComponents() { assertThat(container.getContainerComponentFactories()).containsExactlyElementsOf(componentFactories); assertThat(container.getMessageInterceptors()).containsExactly(interceptor); assertThat(container.getPhase()).isEqualTo(MessageListenerContainer.DEFAULT_PHASE); + assertThat(container.getObservationRegistry()).isEqualTo(ObservationRegistry.NOOP); } diff --git a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/SqsMessageListenerContainerTests.java b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/SqsMessageListenerContainerTests.java index c0b7e99ed..9fddff17a 100644 --- a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/SqsMessageListenerContainerTests.java +++ b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/SqsMessageListenerContainerTests.java @@ -30,6 +30,7 @@ import io.awspring.cloud.sqs.listener.interceptor.AsyncMessageInterceptor; import io.awspring.cloud.sqs.listener.interceptor.MessageInterceptor; import io.awspring.cloud.sqs.listener.source.MessageSource; +import io.micrometer.observation.ObservationRegistry; import java.util.Arrays; import java.util.Collections; import java.util.List; @@ -62,11 +63,13 @@ void shouldCreateFromBuilderWithBlockingComponents() { .singletonList(componentFactory); List queueNames = Arrays.asList("test-queue-name-1", "test-queue-name-2"); Integer phase = 2; + ObservationRegistry observationRegistry = mock(ObservationRegistry.class); SqsMessageListenerContainer container = SqsMessageListenerContainer.builder().messageListener(listener) .sqsAsyncClient(client).errorHandler(errorHandler).componentFactories(componentFactories) .acknowledgementResultCallback(callback).messageInterceptor(interceptor1) - .messageInterceptor(interceptor2).queueNames(queueNames).phase(phase).build(); + .messageInterceptor(interceptor2).queueNames(queueNames).phase(phase) + .observationRegistry(observationRegistry).build(); assertThat(container.getMessageListener()) .isInstanceOf(AsyncComponentAdapters.AbstractThreadingComponentAdapter.class) @@ -93,6 +96,8 @@ void shouldCreateFromBuilderWithBlockingComponents() { assertThat(container.getQueueNames()).containsExactlyElementsOf(queueNames); assertThat(container.getPhase()).isEqualTo(phase); + + assertThat(container.getObservationRegistry()).isSameAs(observationRegistry); } @Test @@ -106,18 +111,22 @@ void shouldCreateFromBuilderWithAsyncComponents() { AsyncMessageInterceptor interceptor2 = mock(AsyncMessageInterceptor.class); AsyncAcknowledgementResultCallback callback = mock(AsyncAcknowledgementResultCallback.class); ContainerComponentFactory componentFactory = mock(ContainerComponentFactory.class); + ObservationRegistry observationRegistry = mock(ObservationRegistry.class); + List> componentFactories = Collections .singletonList(componentFactory); SqsMessageListenerContainer container = SqsMessageListenerContainer.builder() .asyncMessageListener(listener).sqsAsyncClient(client).errorHandler(errorHandler) .componentFactories(componentFactories).acknowledgementResultCallback(callback) - .messageInterceptor(interceptor1).messageInterceptor(interceptor2).queueNames(queueName).id(id).build(); + .messageInterceptor(interceptor1).messageInterceptor(interceptor2).queueNames(queueName).id(id) + .observationRegistry(observationRegistry).build(); assertThat(container.getMessageListener()).isEqualTo(listener); assertThat(container.getErrorHandler()).isEqualTo(errorHandler); assertThat(container.getAcknowledgementResultCallback()).isEqualTo(callback); assertThat(container.getMessageInterceptors()).containsExactly(interceptor1, interceptor2); assertThat(container.getPhase()).isEqualTo(MessageListenerContainer.DEFAULT_PHASE); + assertThat(container.getObservationRegistry()).isSameAs(observationRegistry); } @Test diff --git a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/BatchMessageListeningSinkTests.java b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/BatchMessageListeningSinkTests.java new file mode 100644 index 000000000..be4ea1dea --- /dev/null +++ b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/BatchMessageListeningSinkTests.java @@ -0,0 +1,81 @@ +/* + * Copyright 2013-2022 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.listener.sink; + +import static java.util.stream.Collectors.toList; +import static org.assertj.core.api.Assertions.assertThat; + +import io.awspring.cloud.sqs.listener.MessageProcessingContext; +import io.awspring.cloud.sqs.listener.pipeline.MessageProcessingPipeline; +import io.awspring.cloud.sqs.observation.MessageObservationDocumentation; +import io.awspring.cloud.sqs.observation.MessageObservationDocumentation.HighCardinalityKeyNames; +import io.awspring.cloud.sqs.observation.MessagingOperationType; +import io.micrometer.observation.tck.TestObservationRegistry; +import io.micrometer.observation.tck.TestObservationRegistryAssert; +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; +import java.util.concurrent.CompletableFuture; +import java.util.stream.IntStream; +import org.junit.jupiter.api.Test; +import org.springframework.messaging.Message; +import org.springframework.messaging.support.MessageBuilder; + +/** + * Tests for {@link BatchMessageSink}. + * + * @author Mariusz Sondecki + */ +class BatchMessageListeningSinkTests { + + @Test + void shouldEmitBatchMessages() { + TestObservationRegistry registry = TestObservationRegistry.create(); + int numberOfMessagesToEmit = 10; + List> messagesToEmit = IntStream.range(0, numberOfMessagesToEmit) + .mapToObj(index -> MessageBuilder.withPayload(index).build()).collect(toList()); + List> received = new ArrayList<>(numberOfMessagesToEmit); + AbstractMessageProcessingPipelineSink sink = new BatchMessageSink<>(); + sink.setObservationRegistry(registry); + sink.setTaskExecutor(Runnable::run); + sink.setMessagePipeline(getMessageProcessingPipeline(received)); + sink.start(); + sink.emit(messagesToEmit, MessageProcessingContext.create()).join(); + sink.stop(); + assertThat(received).containsExactlyElementsOf(messagesToEmit); + TestObservationRegistryAssert.assertThat(registry) + .hasNumberOfObservationsWithNameEqualTo("sqs.batch.message.polling.process", 1) + .forAllObservationsWithNameEqualTo("sqs.batch.message.polling.process", + observationContextAssert -> observationContextAssert + .hasHighCardinalityKeyValueWithKey(HighCardinalityKeyNames.MESSAGE_ID.asString()) + .hasLowCardinalityKeyValue( + MessageObservationDocumentation.LowCardinalityKeyNames.OPERATION.asString(), + MessagingOperationType.BATCH_POLLING_PROCESS.getValue())); + } + + private MessageProcessingPipeline getMessageProcessingPipeline(List> received) { + return new MessageProcessingPipeline<>() { + + @Override + public CompletableFuture>> process(Collection> messages, + MessageProcessingContext context) { + received.addAll(messages); + return CompletableFuture.completedFuture(messages); + } + }; + } + +} diff --git a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/FanOutMessageListeningSinkTests.java b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/FanOutMessageListeningSinkTests.java new file mode 100644 index 000000000..b39c3258e --- /dev/null +++ b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/FanOutMessageListeningSinkTests.java @@ -0,0 +1,215 @@ +/* + * Copyright 2013-2022 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.listener.sink; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.fail; + +import brave.Tracing; +import brave.handler.MutableSpan; +import brave.test.TestSpanHandler; +import io.awspring.cloud.sqs.listener.AsyncMessageListener; +import io.awspring.cloud.sqs.listener.MessageProcessingContext; +import io.awspring.cloud.sqs.listener.acknowledgement.handler.AcknowledgementHandler; +import io.awspring.cloud.sqs.listener.interceptor.AsyncMessageInterceptor; +import io.awspring.cloud.sqs.listener.pipeline.*; +import io.awspring.cloud.sqs.observation.MessageObservationDocumentation; +import io.awspring.cloud.sqs.observation.MessagingOperationType; +import io.micrometer.observation.Observation; +import io.micrometer.observation.ObservationRegistry; +import io.micrometer.observation.tck.ObservationContextAssert; +import io.micrometer.observation.tck.TestObservationRegistry; +import io.micrometer.observation.tck.TestObservationRegistryAssert; +import io.micrometer.tracing.Span; +import io.micrometer.tracing.TraceContext; +import io.micrometer.tracing.brave.bridge.BraveBaggageManager; +import io.micrometer.tracing.brave.bridge.BraveCurrentTraceContext; +import io.micrometer.tracing.brave.bridge.BravePropagator; +import io.micrometer.tracing.brave.bridge.BraveTracer; +import io.micrometer.tracing.handler.PropagatingReceiverTracingObservationHandler; +import java.util.*; +import java.util.concurrent.CompletableFuture; +import org.junit.jupiter.api.Test; +import org.springframework.core.task.SimpleAsyncTaskExecutor; +import org.springframework.core.task.TaskExecutor; +import org.springframework.core.task.support.ContextPropagatingTaskDecorator; +import org.springframework.messaging.Message; +import org.springframework.messaging.support.MessageBuilder; + +/** + * Tests for {@link FanOutMessageSink}. + * + * @author Mariusz Sondecki + */ +class FanOutMessageListeningSinkTests { + + private static final String TRACE_ID_HEADER = "X-B3-TraceId"; + + private static final String SPAN_ID_HEADER = "X-B3-SpanId"; + + private static final String PARENT_SPAN_ID_HEADER = "X-B3-ParentSpanId"; + + @Test + void shouldEmitInNestedObservation() { + // GIVEN + TestObservationRegistry registry = TestObservationRegistry.create(); + Message messageToEmit = MessageBuilder.withPayload("foo").build(); + List> received = new ArrayList<>(1); + + MessageProcessingConfiguration.Builder configuration = MessageProcessingConfiguration. builder() + .interceptors(List.of(getInterceptor(registry, Collections.emptyList(), null))) + .messageListener(getListener(received, registry)); + + // WHEN + emitMessage(registry, Runnable::run, messageToEmit, configuration); + + // THEN + assertThat(received).containsExactly(messageToEmit); + TestObservationRegistryAssert.then(registry).hasNumberOfObservationsEqualTo(3) + .hasHandledContextsThatSatisfy(contexts -> { + ObservationContextAssert.then(contexts.get(0)).hasNameEqualTo("sqs.single.message.polling.process") + .hasHighCardinalityKeyValueWithKey( + MessageObservationDocumentation.HighCardinalityKeyNames.MESSAGE_ID.asString()) + .hasLowCardinalityKeyValue( + MessageObservationDocumentation.LowCardinalityKeyNames.OPERATION.asString(), + MessagingOperationType.SINGLE_POLLING_PROCESS.getValue()) + .doesNotHaveParentObservation(); + + ObservationContextAssert.then(contexts.get(1)).hasNameEqualTo("sqs.interceptor.process") + .hasParentObservationContextMatching( + contextView -> contextView.getName().equals("sqs.single.message.polling.process")); + + ObservationContextAssert.then(contexts.get(2)).hasNameEqualTo("sqs.listener.process") + .hasHighCardinalityKeyValue("payload", "foo").hasParentObservationContextMatching( + contextView -> contextView.getName().equals("sqs.interceptor.process")); + }); + } + + @Test + void shouldEmitWithTracingContextFromMessage() { + // GIVEN + SimpleAsyncTaskExecutor taskExecutor = new SimpleAsyncTaskExecutor(); + taskExecutor.setTaskDecorator(new ContextPropagatingTaskDecorator()); + + TestSpanHandler testSpanHandler = new TestSpanHandler(); + Tracing tracing = Tracing.newBuilder().addSpanHandler(testSpanHandler).build(); + io.micrometer.tracing.Tracer tracer = new BraveTracer(tracing.tracer(), + new BraveCurrentTraceContext(tracing.currentTraceContext()), new BraveBaggageManager()); + + ObservationRegistry registry = ObservationRegistry.create(); + registry.observationConfig().observationHandler( + new PropagatingReceiverTracingObservationHandler<>(tracer, new BravePropagator(tracing))); + + TraceContext b3TraceContext = tracer.nextSpan().context(); + String traceId = b3TraceContext.traceId(); + String spanId = b3TraceContext.spanId(); + List> contexts = new ArrayList<>(); + Message messageToEmit = MessageBuilder.withPayload("") + .copyHeaders(Map.of(TRACE_ID_HEADER, traceId, SPAN_ID_HEADER, spanId, PARENT_SPAN_ID_HEADER, spanId)) + .build(); + + MessageProcessingConfiguration.Builder configuration = MessageProcessingConfiguration. builder() + .interceptors(List.of(getInterceptor(registry, contexts, tracer))) + .messageListener(getListener(contexts, registry, tracer)); + + // WHEN + emitMessage(registry, taskExecutor, messageToEmit, configuration); + + // THEN + MutableSpan finishedSpan = testSpanHandler.get(0); + Map context = contexts.get(0); + assertThat(finishedSpan.traceId()).isEqualTo(traceId).isEqualTo(context.get(TRACE_ID_HEADER)); + assertThat(finishedSpan.id()).isNotEqualTo(spanId).isEqualTo(context.get(SPAN_ID_HEADER)); + assertThat(finishedSpan.parentId()).isEqualTo(spanId).isEqualTo(context.get(PARENT_SPAN_ID_HEADER)); + } + + private void emitMessage(ObservationRegistry registry, TaskExecutor taskExecutor, Message messageToEmit, + MessageProcessingConfiguration.Builder configuration) { + AcknowledgementHandler dummyAcknowledgementHandler = new AcknowledgementHandler<>() { + }; + + MessageProcessingPipeline messageProcessingPipeline = MessageProcessingPipelineBuilder + . first(BeforeProcessingInterceptorExecutionStage::new).then(MessageListenerExecutionStage::new) + .thenInTheFuture(AfterProcessingInterceptorExecutionStage::new) + .build(configuration.ackHandler(dummyAcknowledgementHandler).build()); + + AbstractMessageProcessingPipelineSink sink = new FanOutMessageSink<>(); + sink.setObservationRegistry(registry); + sink.setTaskExecutor(taskExecutor); + sink.setMessagePipeline(messageProcessingPipeline); + sink.start(); + sink.emit(List.of(messageToEmit), MessageProcessingContext.create()).join(); + sink.stop(); + } + + private AsyncMessageInterceptor getInterceptor(ObservationRegistry registry, + List> contexts, io.micrometer.tracing.Tracer tracer) { + return new AsyncMessageInterceptor<>() { + @Override + public CompletableFuture> intercept(Message message) { + Observation.createNotStarted("sqs.interceptor.process", registry).start().openScope(); + if (tracer != null) { + Span span = tracer.currentSpan(); + if (span != null) { + TraceContext traceContext = span.context(); + contexts.add(Map.of(TRACE_ID_HEADER, traceContext.traceId(), SPAN_ID_HEADER, + traceContext.spanId(), PARENT_SPAN_ID_HEADER, traceContext.parentId())); + } + } + return AsyncMessageInterceptor.super.intercept(message); + } + + @Override + public CompletableFuture afterProcessing(Message message, Throwable t) { + Observation observation = registry.getCurrentObservation(); + if (observation != null) { + Observation.Scope currentScope = observation.getEnclosingScope(); + if (currentScope != null) { + currentScope.close(); + } + else { + fail("A scope for current observation expected"); + } + observation.stop(); + } + return AsyncMessageInterceptor.super.afterProcessing(message, t); + } + }; + } + + private AsyncMessageListener getListener(List> received, ObservationRegistry registry) { + return message -> Objects.requireNonNull(Observation.createNotStarted("sqs.listener.process", registry) + .highCardinalityKeyValue("payload", message.getPayload()).observe(() -> { + received.add(message); + return CompletableFuture.completedFuture(null); + })); + } + + private AsyncMessageListener getListener(List> contexts, ObservationRegistry registry, + io.micrometer.tracing.Tracer tracer) { + return message -> Objects + .requireNonNull(Observation.createNotStarted("sqs.listener.process", registry).observe(() -> { + Span span = tracer.currentSpan(); + if (span != null) { + TraceContext traceContext = span.context(); + contexts.add(Map.of(TRACE_ID_HEADER, traceContext.traceId(), SPAN_ID_HEADER, + traceContext.spanId(), PARENT_SPAN_ID_HEADER, traceContext.parentId())); + } + return CompletableFuture.completedFuture(null); + })); + } + +} diff --git a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/MessageGroupingSinkTests.java b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/MessageGroupingSinkTests.java index ff286ed63..6aadf0da7 100644 --- a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/MessageGroupingSinkTests.java +++ b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/MessageGroupingSinkTests.java @@ -23,6 +23,10 @@ import io.awspring.cloud.sqs.listener.SqsHeaders; import io.awspring.cloud.sqs.listener.pipeline.MessageProcessingPipeline; import io.awspring.cloud.sqs.listener.sink.adapter.MessageGroupingSinkAdapter; +import io.awspring.cloud.sqs.observation.MessageObservationDocumentation; +import io.awspring.cloud.sqs.observation.MessagingOperationType; +import io.micrometer.observation.tck.TestObservationRegistry; +import io.micrometer.observation.tck.TestObservationRegistryAssert; import java.util.ArrayList; import java.util.Collections; import java.util.List; @@ -41,6 +45,7 @@ * Tests for {@link MessageGroupingSinkAdapter}. * * @author Tomaz Fernandes + * @author Mariusz Sondecki */ class MessageGroupingSinkTests { @@ -62,11 +67,13 @@ void maintainsOrderWithinEachGroup() { messagesToEmit.addAll(thirdMessageGroupMessages); List> received = Collections.synchronizedList(new ArrayList<>()); + TestObservationRegistry registry = TestObservationRegistry.create(); MessageGroupingSinkAdapter sinkAdapter = new MessageGroupingSinkAdapter<>(new OrderedMessageSink<>(), message -> message.getHeaders().get(header, String.class)); + sinkAdapter.setObservationRegistry(registry); sinkAdapter.setTaskExecutor(new SimpleAsyncTaskExecutor()); - sinkAdapter.setMessagePipeline(new MessageProcessingPipeline() { + sinkAdapter.setMessagePipeline(new MessageProcessingPipeline<>() { @Override public CompletableFuture> process(Message message, MessageProcessingContext context) { @@ -84,10 +91,20 @@ public CompletableFuture> process(Message message, sinkAdapter.emit(messagesToEmit, MessageProcessingContext.create()).join(); Map>> receivedMessages = received.stream() .collect(groupingBy(message -> (String) message.getHeaders().get(header))); + sinkAdapter.stop(); assertThat(receivedMessages.get(firstMessageGroupId)).containsExactlyElementsOf(firstMessageGroupMessages); assertThat(receivedMessages.get(secondMessageGroupId)).containsExactlyElementsOf(secondMessageGroupMessages); assertThat(receivedMessages.get(thirdMessageGroupId)).containsExactlyElementsOf(thirdMessageGroupMessages); + TestObservationRegistryAssert.assertThat(registry) + .hasNumberOfObservationsWithNameEqualTo("sqs.single.message.polling.process", 30) + .forAllObservationsWithNameEqualTo("sqs.single.message.polling.process", + observationContextAssert -> observationContextAssert + .hasHighCardinalityKeyValueWithKey( + MessageObservationDocumentation.HighCardinalityKeyNames.MESSAGE_ID.asString()) + .hasLowCardinalityKeyValue( + MessageObservationDocumentation.LowCardinalityKeyNames.OPERATION.asString(), + MessagingOperationType.SINGLE_POLLING_PROCESS.getValue())); } @NotNull diff --git a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/OrderedMessageListeningSinkTests.java b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/OrderedMessageListeningSinkTests.java index 6f01f59f1..81f72faf0 100644 --- a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/OrderedMessageListeningSinkTests.java +++ b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/listener/sink/OrderedMessageListeningSinkTests.java @@ -20,6 +20,11 @@ import io.awspring.cloud.sqs.listener.MessageProcessingContext; import io.awspring.cloud.sqs.listener.pipeline.MessageProcessingPipeline; +import io.awspring.cloud.sqs.observation.MessageObservationDocumentation; +import io.awspring.cloud.sqs.observation.MessageObservationDocumentation.HighCardinalityKeyNames; +import io.awspring.cloud.sqs.observation.MessagingOperationType; +import io.micrometer.observation.tck.TestObservationRegistry; +import io.micrometer.observation.tck.TestObservationRegistryAssert; import java.util.ArrayList; import java.util.List; import java.util.concurrent.CompletableFuture; @@ -32,26 +37,37 @@ * Tests for {@link OrderedMessageSink}. * * @author Tomaz Fernandes + * @author Mariusz Sondecki */ class OrderedMessageListeningSinkTests { @Test void shouldEmitInOrder() { + TestObservationRegistry registry = TestObservationRegistry.create(); int numberOfMessagesToEmit = 1000; List> messagesToEmit = IntStream.range(0, numberOfMessagesToEmit) .mapToObj(index -> MessageBuilder.withPayload(index).build()).collect(toList()); List> received = new ArrayList<>(numberOfMessagesToEmit); AbstractMessageProcessingPipelineSink sink = new OrderedMessageSink<>(); + sink.setObservationRegistry(registry); sink.setTaskExecutor(Runnable::run); sink.setMessagePipeline(getMessageProcessingPipeline(received)); sink.start(); sink.emit(messagesToEmit, MessageProcessingContext.create()).join(); sink.stop(); assertThat(received).containsSequence(messagesToEmit); + TestObservationRegistryAssert.assertThat(registry) + .hasNumberOfObservationsWithNameEqualTo("sqs.single.message.polling.process", numberOfMessagesToEmit) + .forAllObservationsWithNameEqualTo("sqs.single.message.polling.process", + observationContextAssert -> observationContextAssert + .hasHighCardinalityKeyValueWithKey(HighCardinalityKeyNames.MESSAGE_ID.asString()) + .hasLowCardinalityKeyValue( + MessageObservationDocumentation.LowCardinalityKeyNames.OPERATION.asString(), + MessagingOperationType.SINGLE_POLLING_PROCESS.getValue())); } private MessageProcessingPipeline getMessageProcessingPipeline(List> received) { - return new MessageProcessingPipeline() { + return new MessageProcessingPipeline<>() { @Override public CompletableFuture> process(Message message, MessageProcessingContext context) { diff --git a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/observation/BatchMessageProcessTracingObservationHandlerTest.java b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/observation/BatchMessageProcessTracingObservationHandlerTest.java new file mode 100644 index 000000000..08dba3391 --- /dev/null +++ b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/observation/BatchMessageProcessTracingObservationHandlerTest.java @@ -0,0 +1,91 @@ +/* + * Copyright 2013-2024 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package io.awspring.cloud.sqs.observation; + +import static org.assertj.core.api.Assertions.assertThat; + +import brave.Tracing; +import brave.handler.MutableSpan; +import brave.test.TestSpanHandler; +import io.micrometer.tracing.TraceContext; +import io.micrometer.tracing.brave.bridge.BraveBaggageManager; +import io.micrometer.tracing.brave.bridge.BraveCurrentTraceContext; +import io.micrometer.tracing.brave.bridge.BravePropagator; +import io.micrometer.tracing.brave.bridge.BraveTracer; +import java.util.Collection; +import java.util.List; +import java.util.Map; +import org.junit.jupiter.api.Test; +import org.springframework.messaging.MessageHeaders; + +/** + * Tests for {@link BatchMessageProcessTracingObservationHandler}. + * + * @author Mariusz Sondecki + */ +class BatchMessageProcessTracingObservationHandlerTest { + + private static final String TRACE_ID_HEADER = "X-B3-TraceId"; + + private static final String SPAN_ID_HEADER = "X-B3-SpanId"; + + private static final String PARENT_SPAN_ID_HEADER = "X-B3-ParentSpanId"; + + private final TestSpanHandler testSpanHandler = new TestSpanHandler(); + + private final Tracing tracing = Tracing.newBuilder().addSpanHandler(testSpanHandler).build(); + + private final io.micrometer.tracing.Tracer tracer = new BraveTracer(tracing.tracer(), + new BraveCurrentTraceContext(tracing.currentTraceContext()), new BraveBaggageManager()); + + private final BatchMessageProcessTracingObservationHandler handler = new BatchMessageProcessTracingObservationHandler( + tracer, new BravePropagator(tracing)); + + @Test + void shouldCreateLinksForAllMessages() { + // GIVEN + TraceContext traceContext1 = tracer.nextSpan().context(); + TraceContext traceContext2 = tracer.nextSpan().context(); + String traceId1 = traceContext1.traceId(); + String spanId1 = traceContext1.spanId(); + String traceId2 = traceContext2.traceId(); + String spanId2 = traceContext2.spanId(); + Collection receivedMessageHeaders = List.of( + new MessageHeaders( + Map.of(TRACE_ID_HEADER, traceId1, SPAN_ID_HEADER, spanId1, PARENT_SPAN_ID_HEADER, spanId1)), + new MessageHeaders( + Map.of(TRACE_ID_HEADER, traceId2, SPAN_ID_HEADER, spanId2, PARENT_SPAN_ID_HEADER, spanId2))); + BatchMessagePollingProcessObservationContext context = new BatchMessagePollingProcessObservationContext( + receivedMessageHeaders); + + // WHEN + handler.onStart(context); + handler.onStop(context); + + // THEN + MutableSpan finishedSpan = testSpanHandler.get(0); + assertThat(traceId1).isNotEqualTo(traceId2); + assertThat(finishedSpan.traceId()).isNotEmpty().withFailMessage( + "A trace ID of the process of the whole batch should differ from trace IDs of received messages") + .isNotIn(traceId1, traceId2); + assertThat(finishedSpan.tags()).containsEntry("links[0].traceId", traceId1) + .containsEntry("links[1].traceId", traceId2) + .withFailMessage( + """ + "links[*].spanId" tags should differ from span IDs of received messages (which will be parent span IDs)""") + .doesNotContainEntry("links[0].spanId", spanId1).doesNotContainEntry("links[1].spanId", spanId2); + } +} diff --git a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/operations/SqsTemplateTests.java b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/operations/SqsTemplateTests.java index f9ab5cec7..b53e887da 100644 --- a/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/operations/SqsTemplateTests.java +++ b/spring-cloud-aws-sqs/src/test/java/io/awspring/cloud/sqs/operations/SqsTemplateTests.java @@ -28,7 +28,13 @@ import io.awspring.cloud.sqs.SqsAcknowledgementException; import io.awspring.cloud.sqs.listener.QueueNotFoundStrategy; import io.awspring.cloud.sqs.listener.SqsHeaders; +import io.awspring.cloud.sqs.observation.MessageObservationDocumentation; +import io.awspring.cloud.sqs.observation.MessagingOperationType; import io.awspring.cloud.sqs.support.converter.ContextAwareMessagingMessageConverter; +import io.micrometer.observation.Observation; +import io.micrometer.observation.tck.ObservationContextAssert; +import io.micrometer.observation.tck.TestObservationRegistry; +import io.micrometer.observation.tck.TestObservationRegistryAssert; import java.time.Duration; import java.util.Collection; import java.util.Collections; @@ -47,26 +53,11 @@ import org.springframework.messaging.MessageHeaders; import org.springframework.messaging.support.MessageBuilder; import software.amazon.awssdk.services.sqs.SqsAsyncClient; -import software.amazon.awssdk.services.sqs.model.CreateQueueResponse; -import software.amazon.awssdk.services.sqs.model.DeleteMessageBatchRequest; -import software.amazon.awssdk.services.sqs.model.DeleteMessageBatchResponse; -import software.amazon.awssdk.services.sqs.model.GetQueueAttributesRequest; -import software.amazon.awssdk.services.sqs.model.GetQueueAttributesResponse; -import software.amazon.awssdk.services.sqs.model.GetQueueUrlRequest; -import software.amazon.awssdk.services.sqs.model.GetQueueUrlResponse; -import software.amazon.awssdk.services.sqs.model.MessageSystemAttributeName; -import software.amazon.awssdk.services.sqs.model.QueueAttributeName; -import software.amazon.awssdk.services.sqs.model.QueueDoesNotExistException; -import software.amazon.awssdk.services.sqs.model.ReceiveMessageRequest; -import software.amazon.awssdk.services.sqs.model.ReceiveMessageResponse; -import software.amazon.awssdk.services.sqs.model.SendMessageBatchRequest; -import software.amazon.awssdk.services.sqs.model.SendMessageBatchRequestEntry; -import software.amazon.awssdk.services.sqs.model.SendMessageBatchResponse; -import software.amazon.awssdk.services.sqs.model.SendMessageRequest; -import software.amazon.awssdk.services.sqs.model.SendMessageResponse; +import software.amazon.awssdk.services.sqs.model.*; /** * @author Tomaz Fernandes + * @author Mariusz Sondecki */ @SuppressWarnings("unchecked") class SqsTemplateTests { @@ -456,6 +447,42 @@ void shouldSendWithQueueAndMessageAndHeaders() { assertThat(capturedRequest.messageAttributes().get(headerName).stringValue()).isEqualTo(headerValue); } + @Test + void shouldSendInNestedObservation() { + String queue = "test-queue"; + String payload = "test-payload"; + String parentSpanName = "Parent span"; + + TestObservationRegistry registry = TestObservationRegistry.create(); + GetQueueUrlResponse urlResponse = GetQueueUrlResponse.builder().queueUrl(queue).build(); + given(mockClient.getQueueUrl(any(GetQueueUrlRequest.class))) + .willReturn(CompletableFuture.completedFuture(urlResponse)); + mockQueueAttributes(mockClient, Map.of()); + + UUID uuid = UUID.randomUUID(); + String sequenceNumber = "1234"; + SendMessageResponse response = SendMessageResponse.builder().messageId(uuid.toString()) + .sequenceNumber(sequenceNumber).build(); + given(mockClient.sendMessage(any(SendMessageRequest.class))) + .willReturn(CompletableFuture.completedFuture(response)); + + SqsOperations template = SqsTemplate.builder().sqsAsyncClient(mockClient).observationRegistry(registry) + .buildSyncTemplate(); + Observation.createNotStarted(parentSpanName, registry) + .observe(() -> template.send(to -> to.queue(queue).payload(payload))); + + TestObservationRegistryAssert.then(registry).hasNumberOfObservationsEqualTo(2) + .hasHandledContextsThatSatisfy(contexts -> ObservationContextAssert.then(contexts.get(1)) + .hasNameEqualTo("sqs.single.message.publish") + .hasHighCardinalityKeyValueWithKey( + MessageObservationDocumentation.HighCardinalityKeyNames.MESSAGE_ID.asString()) + .hasLowCardinalityKeyValue( + MessageObservationDocumentation.LowCardinalityKeyNames.OPERATION.asString(), + MessagingOperationType.SINGLE_PUBLISH.getValue()) + .hasParentObservationContextMatching( + contextView -> contextView.getName().equals(parentSpanName))); + } + @Test void shouldSendBatch() { String queue = "test-queue"; @@ -503,6 +530,54 @@ void shouldSendBatch() { assertThat(secondEntry.messageAttributes().get(headerName2).stringValue()).isEqualTo(headerValue2); } + @Test + void shouldSendBatchInNestedObservation() { + String queue = "test-queue"; + String payload1 = "test-payload-1"; + String payload2 = "test-payload-2"; + String headerName1 = "headerName"; + String headerValue1 = "headerValue"; + String headerName2 = "headerName2"; + String headerValue2 = "headerValue2"; + String parentSpanName = "Parent span"; + + TestObservationRegistry registry = TestObservationRegistry.create(); + + Message message1 = MessageBuilder.withPayload(payload1).setHeader(headerName1, headerValue1).build(); + Message message2 = MessageBuilder.withPayload(payload2).setHeader(headerName2, headerValue2).build(); + List> messages = List.of(message1, message2); + + GetQueueUrlResponse urlResponse = GetQueueUrlResponse.builder().queueUrl(queue).build(); + given(mockClient.getQueueUrl(any(GetQueueUrlRequest.class))) + .willReturn(CompletableFuture.completedFuture(urlResponse)); + mockQueueAttributes(mockClient, Map.of()); + + SendMessageBatchResponse response = SendMessageBatchResponse.builder().successful( + builder -> builder.id(message1.getHeaders().getId().toString()).messageId(UUID.randomUUID().toString()), + builder -> builder.id(message2.getHeaders().getId().toString()).messageId(UUID.randomUUID().toString())) + .build(); + given(mockClient.sendMessageBatch(any(SendMessageBatchRequest.class))) + .willReturn(CompletableFuture.completedFuture(response)); + + SqsOperations template = SqsTemplate.builder().sqsAsyncClient(mockClient).observationRegistry(registry) + .buildSyncTemplate(); + SendResult.Batch results = Observation.createNotStarted(parentSpanName, registry) + .observe(() -> template.sendMany(queue, messages)); + + TestObservationRegistryAssert.then(registry).hasNumberOfObservationsEqualTo(2) + .hasHandledContextsThatSatisfy(contexts -> ObservationContextAssert.then(contexts.get(1)) + .hasNameEqualTo("sqs.batch.message.publish") + .hasHighCardinalityKeyValueWithKey( + MessageObservationDocumentation.HighCardinalityKeyNames.MESSAGE_ID.asString()) + .hasLowCardinalityKeyValue( + MessageObservationDocumentation.LowCardinalityKeyNames.OPERATION.asString(), + MessagingOperationType.BATCH_PUBLISH.getValue()) + .hasParentObservationContextMatching( + contextView -> contextView.getName().equals(parentSpanName))); + + assertThat(results.successful()).hasSize(2); + } + @Test void shouldAddFailedToTheBatchResult() { String queue = "test-queue"; @@ -1033,6 +1108,46 @@ void shouldReceiveFifoWithRandomAttemptId() { assertThat(request.maxNumberOfMessages()).isEqualTo(1); } + @Test + void shouldReceiveInNestedObservation() { + String queue = "test-queue"; + String payload = "test-payload"; + String parentSpanName = "Parent span"; + + TestObservationRegistry registry = TestObservationRegistry.create(); + GetQueueUrlResponse urlResponse = GetQueueUrlResponse.builder().queueUrl(queue).build(); + given(mockClient.getQueueUrl(any(GetQueueUrlRequest.class))) + .willReturn(CompletableFuture.completedFuture(urlResponse)); + ReceiveMessageResponse receiveMessageResponse = ReceiveMessageResponse.builder().messages(builder -> builder + .messageId(UUID.randomUUID().toString()).receiptHandle("test-receipt-handle").body(payload).build()) + .build(); + given(mockClient.receiveMessage(any(ReceiveMessageRequest.class))) + .willReturn(CompletableFuture.completedFuture(receiveMessageResponse)); + DeleteMessageBatchResponse deleteResponse = DeleteMessageBatchResponse.builder() + .successful(builder -> builder.id(UUID.randomUUID().toString())).build(); + given(mockClient.deleteMessageBatch(any(DeleteMessageBatchRequest.class))) + .willReturn(CompletableFuture.completedFuture(deleteResponse)); + + SqsOperations template = SqsTemplate.builder().sqsAsyncClient(mockClient).observationRegistry(registry) + .configure(options -> options.defaultQueue(queue)).buildSyncTemplate(); + Optional> receivedMessage = Observation.createNotStarted(parentSpanName, registry) + .observe(() -> template.receive()); + + TestObservationRegistryAssert.then(registry).hasNumberOfObservationsEqualTo(2) + .hasHandledContextsThatSatisfy(contexts -> ObservationContextAssert.then(contexts.get(1)) + .hasNameEqualTo("sqs.single.message.manual.process") + .hasHighCardinalityKeyValueWithKey( + MessageObservationDocumentation.HighCardinalityKeyNames.MESSAGE_ID.asString()) + .hasLowCardinalityKeyValue( + MessageObservationDocumentation.LowCardinalityKeyNames.OPERATION.asString(), + MessagingOperationType.SINGLE_MANUAL_PROCESS.getValue()) + .hasParentObservationContextMatching( + contextView -> contextView.getName().equals(parentSpanName))); + assertThat(receivedMessage).isPresent() + .hasValueSatisfying(message -> assertThat(message.getPayload()).isEqualTo(payload)); + + } + @Test void shouldReceiveBatchWithDefaultValues() { String queue = "test-queue"; @@ -1208,4 +1323,45 @@ void shouldReceiveBatchFifo() { } + @Test + void shouldReceiveBatchInNestedObservation() { + String parentSpanName = "Parent span"; + + TestObservationRegistry registry = TestObservationRegistry.create(); + GetQueueUrlResponse urlResponse = GetQueueUrlResponse.builder().queueUrl("test-queue").build(); + given(mockClient.getQueueUrl(any(GetQueueUrlRequest.class))) + .willReturn(CompletableFuture.completedFuture(urlResponse)); + mockQueueAttributes(mockClient, Map.of()); + ReceiveMessageResponse receiveMessageResponse = ReceiveMessageResponse.builder() + .messages( + builder -> builder.messageId(UUID.randomUUID().toString()) + .receiptHandle("test-receipt-handle-1").body("test-payload").build(), + builder -> builder.messageId(UUID.randomUUID().toString()) + .receiptHandle("test-receipt-handle-2").body("test-payload").build()) + .build(); + given(mockClient.receiveMessage(any(ReceiveMessageRequest.class))) + .willReturn(CompletableFuture.completedFuture(receiveMessageResponse)); + DeleteMessageBatchResponse deleteResponse = DeleteMessageBatchResponse.builder() + .successful(builder -> builder.id(UUID.randomUUID().toString())).build(); + given(mockClient.deleteMessageBatch(any(DeleteMessageBatchRequest.class))) + .willReturn(CompletableFuture.completedFuture(deleteResponse)); + + SqsOperations template = SqsTemplate.builder().sqsAsyncClient(mockClient).observationRegistry(registry) + .configure(options -> options.defaultQueue("test-queue")).buildSyncTemplate(); + Collection> receivedMessages = Observation.createNotStarted(parentSpanName, registry) + .observe(() -> template.receiveMany()); + + TestObservationRegistryAssert.then(registry).hasNumberOfObservationsEqualTo(2) + .hasHandledContextsThatSatisfy(contexts -> ObservationContextAssert.then(contexts.get(1)) + .hasNameEqualTo("sqs.batch.message.manual.process") + .hasHighCardinalityKeyValueWithKey( + MessageObservationDocumentation.HighCardinalityKeyNames.MESSAGE_ID.asString()) + .hasLowCardinalityKeyValue( + MessageObservationDocumentation.LowCardinalityKeyNames.OPERATION.asString(), + MessagingOperationType.BATCH_MANUAL_PROCESS.getValue()) + .hasParentObservationContextMatching( + contextView -> contextView.getName().equals(parentSpanName))); + assertThat(receivedMessages).hasSize(2); + } + } diff --git a/spring-cloud-aws-sqs/src/test/resources/logback.xml b/spring-cloud-aws-sqs/src/test/resources/logback.xml index 99b1679d4..d8e5c2c1b 100644 --- a/spring-cloud-aws-sqs/src/test/resources/logback.xml +++ b/spring-cloud-aws-sqs/src/test/resources/logback.xml @@ -46,6 +46,7 @@ +