Map SNS notification message as object in SqsListener

[lenlaurito]

Type: Feature

Is your feature request related to a problem? Please describe.
Is there a way to automatically map SNS notification messages in @SqsListener? If the SQS queue is subscribed to a topic, the listener for the queue would process the SNS Notification. How can we automatically map the SNS Notification body as an object in the Sqs Listener?

Describe the solution you'd like
Ideally we can use the @NotificationSubject and @NotificationMessage in the method for the SqsListener. This worked for earlier versions of spring cloud (2.x). But I can't use it for spring cloud 3. Is this possible?

  @SqsListener(value = QUEUE_NAME)
  public void consumeMessage(
      @NotificationSubject String subject,
      @NotificationMessage SomeObject messageAsObject) {
    log.info(subject);
    log.info(messageAsObject)
  }

[dbouclier]

@lenlaurito did you tried setup the SNS topic with this attribute => https://docs.aws.amazon.com/sns/latest/dg/sns-large-payload-raw-message-delivery.html

more explanation here: #886

[lenlaurito]

@dbouclier Yes, we are also manually mapping the "Message" in the SNS notification to an object-- similar to your comment. We're looking for an easier way of doing this if possible. Like having the "Message" mapped to an object automatically.

In previous versions of spring cloud (2.x), we were able to achieve this using @NotificationMessage annotation. But it doesn't seem to be working with spring cloud 3?

[dbouclier]

it's working actually, I was able to automatically de-serialize object in my @SqsListener, like that:

@SqsListener(queueNames = "some_queue.fifo")
    public void onEvent(final Message<MyCustomeType> event) {
      .....
    }

OR 

@SqsListener(queueNames = "some_queue.fifo")
    public void onEvent(final MyCustomeType event) {
      .....
    }

but in the subscription you have to enable "RawMessageDelivery" attribute

[lenlaurito]

@dbouclier We'd still like to be able to access the other fields of the notification though, like the subject.

[tomazfernandes]

Hi @lenlaurito and @dbouclier.

SCAWS 3.0 SQS integration doesn't provide any particular integration with SNS OOTB.

You should be able to have the raw message as a POJO and work your way through it with all available information in the original message using RawMessageDelivery as mentioned.

But I think it'd be helpful to provide the functionality similar to 2.0.

Would you be willing to write a PR for it?

Thanks.

[dapriett]

Having the same issue with trying to upgrade. I believe previous version of argument resolvers had a NotificationMessageArgumentResolver, where as the current version does not.

[dapriett]

For the current version, what's the recommended way to customize the message converter to add the NotificationSubjectHandlerMethodArgumentResolver and NotificationMessageHandlerMethodArgumentResolver, but still keep all the other defaults?

[tomazfernandes]

Hi @dapriett, you can use a SqsListenerConfigurer to manage the converters.

We're looking for a PR to add these resolvers, if you'd like to write one I'd be happy to review it.

Thanks.

[msosa]

I was able to get @NotficationMessage to work locally by adding the missing classes to my project and adding it to the registrar.

@tomazfernandes i don't mind opening up a PR to add these, I was able to copy NotificationMessageArgumentResolver and NotificationRequestConverter over with no changes(I don't think). So let me know if there is something I am missing, otherwise I can create a PR

[tomazfernandes]

Hi @msosa, sure, sounds good, if you can please open a PR.

It'd be great if you could add an integration test and / or a Sample using this, though that's optional and won't block the PR.

Thanks.

[EstelaGarcia]

Is there any progress on this ticket? We use this library, and SNS->SQS pipes, and want to update to SpringBoot 3. We are waiting for this issue to be fixed so that we can do the update without having to update our entire infrastructure to deliver raw messages.

[tomazfernandes]

Is there any progress on this ticket? We use this library, and SNS->SQS pipes, and want to update to SpringBoot 3. We are waiting for this issue to be fixed so that we can do the update without having to update our entire infrastructure to deliver raw messages.

We should be able to take a look into that soon - in the meantime, there's a workaround by creating a NotificationMessageArgumentResolver just like the PR opened for this issue.

Would that work for you?

[EstelaGarcia]

We have 20+ projects currently using this, so rolling our own changes is not exactly something we would like to do. Particularly when the library change is in the pipeline. Hopefully it won't be too long before the library fix is in place :)

[tomazfernandes]

We now support @SnsMessage for single message and batch listeners.

I'll close this issue, but if there's anything else we can reopen it.

[lenlaurito]

@tomazfernandes

Thank you for the update. We've seen the SnsNotificationMessage annotation that can be used to get the Message payload and map it to an object.

Is there also a way for us to get the other details, like the sns notification subject in the the queue listeners? In older versions, we were able to use NotificationSubject annotation in the listeners for this.