Support for QNX (performance issues)

[fschmitz925]

Hi,

we use the mqtt client lib on QNX (via the C++ async_client class). Compared to builds for Windows/Linux, it turned out that on QNX pushing is extremely slow (with payloads typically between 500 KB and 1 MB).

Using MQTT_C_CLIENT_TRACE, we figured out that the problem are partial writes, so we increased the system's net.inet.tcp.sendspace from 22528 to 180224, which is about the max value we should be using.

This helped somewhat, but the push performance was still quite low. So we looked into the source code and identified four possible changes, which have effects independently, and together dramatically improve push performance:

- In : 3rdParty\paho.mqtt.cpp\src\externals\paho-mqtt-c\src\MQTTAsyncUtils.c		   

	- In : thread_return_type WINAPI MQTTAsync_sendThread(void* n)
		Remove (line 1814) 
				timeout = 1000; // 1 second for follow on waits
		to leave the to at 10 [ms]

	- In : thread_return_type WINAPI MQTTAsync_receiveThread(void* n)
		Remove (line 2035) 
				timeout = 1000L;
		to leave the to at 10 [ms]

	- In : static MQTTPacket* MQTTAsync_cycle(SOCKET* sock, unsigned long timeout, int* rc)
		Replace line (3023) 
			MQTTAsync_sleep(100L);
		with 
			MQTTAsync_sleep(5L);

- In : 3rdParty\paho.mqtt.cpp\src\externals\paho-mqtt-c\src\Socket.h

		#define USE_SELECT

While this works fine for us (no excessive CPU load or threading or stability issues observed so far), we are a hesitant to change the 3rd party code, for fear of losing compatibility with future releases, and, to be honest, because we don't fully understand all implications / possible consequences of these changes. And, of course, the legalese involved in changing 3rd party code ...

So we would greatly appreciate if someone with a deeper understanding of the matter could review these changes, and if a future release could take them somehow into account, e.g. by using #defines for the literal to/sleep values, so we may use cmake add_compile_definitions to inject working values - If still applicable, this is, because I notice that in other threads the sleeps/timeouts are already being discussed.

Thanks a lot and best regards,
Frank Schmitz
( PROMESS Montage- und Prüfsysteme GmbH, Berlin, Germany )

[speedy-h]

Hello,

I noticed the same performance issue when sending packets on a FreeBSD system and narrowed it down to partial writes and the same 1s delay you've highlighted.

I think the root cause is after a partial write, if the socket becomes available for more data, nothing is notified. We instead wait for the full timeout to happen before running the MQTTAsync_processCommand in the MQTTAsync_sendThread loop. Based on this I don't think reducing the timeout is the best solution, but rather send a trigger to the send thread when there's a pending write that could continue.

I managed to get this to work by making the send_cond variable visible in the Socket.c file. It get triggered in theSocket_getReadySocket after the select function call if a partial write happened. I introduced a new event in the Sockets structure to help with this.

I did this on version 1.3.9 and it looks like it could work for master.
If the current maintainers think this is allong the right track to get this performance improvement I can try to get my change working on master and open a PR.

[fschmitz925]

Sounds promising, thx