Error collecting status: MQGET: MQCC = MQCC_FAILED [2] MQRC = MQRC_NO_MSG_AVAILABLE [2033]

[AntonioTrindade]

I am using the InfluxDB exporter and there are some queue managers where the exporter aborts with the following in the log file:

time="2023-06-12T15:37:59+01:00" level=error msg="Error collecting cluster status: MQGET: MQCC = MQCC_FAILED [2] MQRC = MQRC_NO_MSG_AVAILABLE [2033]"
time="2023-06-12T15:38:02+01:00" level=error msg="Error collecting queue manager status: MQGET: MQCC = MQCC_FAILED [2] MQRC = MQRC_NO_MSG_AVAILABLE [2033]"
time="2023-06-12T15:38:02+01:00" level=fatal msg="Error collecting status: MQGET: MQCC = MQCC_FAILED [2] MQRC = MQRC_NO_MSG_AVAILABLE [2033]"```

Is it possible to make the exporter ignore the 2033 RC?
What could possibly go wrong if this RC is ignored?

Thanks.

[ibmmqmet]

The most likely issue is that your qmgr is not responding quickly enough to the status requests - the collection has a 3 second timeout to get returned messages, which should be plenty of time. But perhaps your system is too busy. The quoted error log has 3 seconds between the first and second entry, which suggests that particular timeout - it's not configurable, but you can easily change it in the source code for status.go - look for WaitInterval and rebuild it. Setting logLevel=trace would show more detail of the flows to confirm my guess.

But if it's taking that long to respond to simple status queries, then other metrics could well be delayed/out-of-date. And other monitoring requests will queue up. So it's not a good idea to ignore the problem. Better would be to work out why your qmgr is taking that long.

[ravixsinghAtPIMCO]

I am getting the same issue with the JSON exporter. Does that have a WaitInterval as well? If so, where?

[AntonioTrindade]

@ibmmqmet

I put logLevel=trace and found that, before each error, the collector get an RC=2085 (MQRC_UNKNOWN_OBJECT_NAME), followed by CC=1 RC=3008 (MQRCCF_COMMAND_FAILED).

The relevant log entries are:
time="2023-06-16T11:42:21+01:00" level=debug msg="Collected all cluster status"
time="2023-06-16T11:42:21+01:00" level=trace msg="> [CollectQueueManagerStatus]"
time="2023-06-16T11:42:21+01:00" level=trace msg="> [QueueManagerInitAttributes]"
time="2023-06-16T11:42:21+01:00" level=trace msg="< [QueueManagerInitAttributes] rp: 1"
time="2023-06-16T11:42:21+01:00" level=trace msg="> [collectQueueManagerStatus]"
time="2023-06-16T11:42:21+01:00" level=trace msg="> [statusClearReplyQ]"
time="2023-06-16T11:42:21+01:00" level=trace msg="< [statusClearReplyQ] rp: 0"
time="2023-06-16T11:42:21+01:00" level=trace msg="> [statusSetCommandHeaders]"
time="2023-06-16T11:42:21+01:00" level=trace msg="< [statusSetCommandHeaders] rp: 0"
time="2023-06-16T11:42:21+01:00" level=trace msg="> [statusGetReply]"
time="2023-06-16T11:42:21+01:00" level=trace msg="< [statusGetReply] rp: 1 CFH: &{Type:2 StrucLength
:36 Version:1 Command:41 MsgSeqNumber:1 Control:0 CompCode:2 Reason:2085 ParameterCount:0} Error: "
time="2023-06-16T11:42:21+01:00" level=trace msg="> [statusGetReply]"
time="2023-06-16T11:42:21+01:00" level=trace msg="< [statusGetReply] rp: 1 CFH: &{Type:2 StrucLength:36 Version:1 Command:41 MsgSeqNumber:2 Control:1 CompCode:2 Reason:3008 ParameterCount:0} Error: "
time="2023-06-16T11:42:21+01:00" level=trace msg="< [collectQueueManagerStatus] rp: 0 Error: nil"
time="2023-06-16T11:42:21+01:00" level=trace msg="< [CollectQueueManagerStatus] rp: 0 Error: nil"
time="2023-06-16T11:42:21+01:00" level=debug msg="Collected all queue manager status"
time="2023-06-16T11:42:21+01:00" level=fatal msg="Error collecting status: MQGET: MQCC = MQCC_FAILED [2] MQRC = MQRC_NO_MSG_AVAILABLE [2033]"

[ibmmqmet]

The CFH response shown here corresponds to an earlier CollectQueueStatus operation Command:41 Which even more strongly suggests that your queue manager is simply not responding fast enough. So as I said before, you might prefer to bump the WaitInterval from 3 seconds in the source code.

[AntonioTrindade]

Mark, perhaps a future release can have this parameter as a command-line option... That would be great, IMO.

[ravixsinghAtPIMCO]

I am getting the same issue with the JSON exporter. Does that have a WaitInterval as well? If so, where?

You can ignore my issue - I was testing the MQ system event error logging in JSON by killing an MQ Process ( I think it was the command server) but forgot to restart the queue manager to restart the killed process. Once I did, the mq JSON collector worked just fine. Sorry for the noise.