(session handling): No response from SUBSCRIBE when client resets

[rodmaz]

Describe the bug
Whenever our Greengrass client devices reset, they are able to SUBSCRIBE but receive no response as Moquette throws exception.

To Reproduce
Use clean-session=true and reset Greengrass client device after first SUBSCRIPTIONs.

Expected behavior
Greengrass client can reset at any time and subscriptions should work smoothly.

Actual behavior
Moquette throws Java exception:

2023-09-14T00:30:58.039Z [DEBUG] (Session Executor 0) io.moquette.broker.Authorizator: Client will be subscribed to the topic username: -----BEGIN CERTIFICATE-----
MIIDW...==
-----END CERTIFICATE-----
, messageId: 1, topic: $aws/things/my-thing/shadow/name/my-shadow/get/accepted. {}
2023-09-14T00:30:58.041Z [INFO] (Session Executor 0) io.moquette.broker.SessionEventLoop: SessionEventLoop Session Executor 0 reached exception in processing command. {}
java.util.concurrent.ExecutionException: java.lang.NullPointerException
        at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
        at io.moquette.broker.SessionEventLoop.executeTask(SessionEventLoop.java:49)
        at io.moquette.broker.SessionEventLoop.run(SessionEventLoop.java:34)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NullPointerException
        at io.moquette.broker.PostOffice.subscribeClientToTopics(PostOffice.java:236)
        at io.moquette.broker.MQTTConnection.lambda$processSubscribe$6(MQTTConnection.java:402)
        at io.moquette.broker.SessionCommand.execute(SessionCommand.java:23)
        at io.moquette.broker.PostOffice.lambda$routeCommand$5(PostOffice.java:635)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at io.moquette.broker.SessionEventLoop.executeTask(SessionEventLoop.java:46)
        ... 2 more

Environment

• OS: Fedora
• JDK version: openjdk 11.0.20 2023-07-18
• Nucleus version: 2.11.2
• Moquette MQTT version: 2.3.4
• MQTT client: FreeRTOS (CoreMQTT)

Additional context
This seems related to the issue reported in 2022 in Moquette.

Workaround
Using clean-session=false seems to be a valid workaround.

[MikeDombo]

@rodmaz
Yes, you're correct this is a bug in moquette due to persistent session handling.

If you retry the subscribe request, does it succeed? This should not fail consistently, but only when a client disconnects and very quickly reconnects and attempts to subscribe.

Do you have retries on your subscribe request?

[rodmaz]

Currently we don't have retries in our subscription logic (something we probably have to add).
Do you have any rule of thumb of how long we should wait to circumvent this issue?

[MikeDombo]

I'm not sure at this time. I'm not entirely sure that a retry would fix this problem, but it is something that I'd request you to try.

Is your client actually calling subscribe after re-connecting? Because if you have a persistent session, you shouldn't have any need to resubscribe to any topics.

Moquette claims to have fixed the problem, but we are already using a version which includes the fix, so I'm not very confident it is actually resolved upstream.

[MikeDombo]

#121 may address your problem. If you are interested, you could take that pull request and create a private version of the Greengrass Moquette component to see if it solves your problem.

You can do this using the Greengrass Development Kit: https://github.com/aws-greengrass/aws-greengrass-gdk-cli using the gdk-config.json file which is in this repository you can edit that file to set the proper AWS region. Running gdk component build then gdk component publish should create this component in your account and region. Note that if you do this, the public version of the component will not be available anymore as long as a private version of the component exists.

[rodmaz]

@MikeDombo #121 is a rebase from 0.16, right? According to Moquette, this is fixed in 0.17.
Chances are the issue will remain in place, no?

[MikeDombo]

Yes, that is a rebase on 0.16 which had additional changes to session handling done to it. 0.17 has made even more changes to the session handling. PR #122 has moquette 0.17 now.

[rodmaz]

@MikeDombo Thanks for the swift action on this matter!

[MikeDombo]

@rodmaz Is this something that you'd be willing/able to try out the pre-release? I'm not positive when we'd be able to release this change publicly; at least 2 weeks at this point.

[rodmaz]

@MikeDombo Not needed at this time. We can wait a couple of weeks. Important that a fix is on its way. Thanks!

[MikeDombo]

New component version is now released which uses Moquette 0.17. The new version is 2.3.5, please try it out when you're able to see if it addresses this issue for you.

[MikeDombo]

Closing now, please reopen as needed.