Node runs into an exception failed_to_deny_deprecated_features on boot when try to deactivate deprecated feature

[fernandomacho]

Describe the bug

Hi, I migrate my rabbit cluster from 4.0.3 to 4.0.4.
Worlk fine but on gui show this error:

I try to disable the transient_nonexcl_queues feature adding on /etc/rabbitmq/rabbitmq.conf this line

deprecated_features.permit.transient_nonexcl_queues = false

when restart the node get this error:

Nov 24 10:48:51 localhost rabbitmq-server[381810]:   Starting broker...
Nov 24 10:48:51 localhost rabbitmq-server[381810]: BOOT FAILED
Nov 24 10:48:51 localhost rabbitmq-server[381810]: ===========
Nov 24 10:48:51 localhost rabbitmq-server[381810]: Error during startup: {error,
Nov 24 10:48:51 localhost rabbitmq-server[381810]:                           {failed_to_deny_deprecated_features,
Nov 24 10:48:51 localhost rabbitmq-server[381810]:                               [transient_nonexcl_queues]}}
Nov 24 10:48:52 localhost rabbitmq-server[381810]: Runtime terminating during boot (terminating)
Nov 24 10:48:52 localhost rabbitmq-server[381810]: #015

Is the procedure I am following to disable the deprecated feature correct?

I attach dump
erl_crash.dump.zip

Reproduction steps

Make rolling upgrade from 4.0.3 to 4.0.4

Expected behavior

Rabbitmq start correctly

Additional context

No response

[michaelklishin]

@fernandomacho

I cannot reproduce:

# deprecated.features.disable.conf
deprecated_features.permit.transient_nonexcl_queues = false

RABBITMQ_CONFIG_FILE=/path/to/deprecated.features.disable.conf ./sbin/rabbitmq-server

Evidence of effectively applied settings:

rabbitmq-diagnostics environment | grep permit_deprecated
% => {permit_deprecated_features,#{transient_nonexcl_queues => false}}

rabbitmq-diagnostics status
# => /path/to/configs/deprecated.features.disable.conf

This is not an upgrade.

The error, failed_to_deny_deprecated_features, only appears in a couple of places, specifically
when a deprecated feature is in used and you try to disable it. Disaling a deprecated feature won't be possible if the feature is currently in use, that is, in this case you have classic non-exclusive queues.

[fernandomacho]

Hi, I not use classic queues on my config has:
classic_queue.default_version = 2

This param disable creation of classic queues, correct?

How to show classic non-exclusive queues on cluster?

Thanks

[fernandomacho]

I review all queues on my cluster and all queues are quorum queues or rabbit_mqtt_qos0_queue ...
Sorry, but I'm a little lost.

[michaelklishin]

@fernandomacho I cannot suggest much without a definition import file and logs at debug level.

classic_queue.default_version is not relevant. It is classic durable (non-exclusive) queues that are relevant if you prohibit the use of deprecated_features.permit.transient_nonexcl_queues

Assuming that there are some durable CQs, the node works as expected: it detects that a feature you have disabled is used, and the node refuses to boot. This is by design.

So the only open ended question is whether the "feature in use" detection works correctly. We cannot answer that question without a definition file (you are welcome to edit out users and alter sensitive names in it, of course).

[michaelklishin]

In case it is not clear, the mechanism that allows you to disable deprecated features is supposed to be used like so:

1. You use CLI tools or any other option to detect a case of, say, classic queues with certain characteristics (mirroring in 3.13.x, or transient non-exclusive queues in 4.x, or something else)
2. You work towards removing the use of said feature
3. When you are done, you disable its use so that, say, certain declarations or policy definitions will be rejected. This also means that if a node detects deprecated feature used on boot, it will fail to start for maximum visibility

So this route towards being deprecated feature-free is opinionated and won't be suitable for every environment.

[fernandomacho]

Hi,
I review queues via api end point /api/queues/detailed and not show any use of classic queues
I review all definitions via api end point /api/definitions and not show any use of classic queues

I attach file. This file contain:
Output of /api/queues/detailed and /api/definitions
Broker export (via gui)
log on debug mode when restart node with param deprecated_features.permit.transient_nonexcl_queues = false
erl_crash.dump on this failed restart

Looking at the log, I can't really find why I'm using the transient_nonexcl_queues flag.
Reading the documentation, I understand that this feature flag is only used in the case of classic queues, which apparently is not my case.
As I was saying, all queues are quorum or MQTT type and in the definition of the vhosts they are declared to use quorum as the default queue.

Thanks for your help!

Archive.zip

[michaelklishin]

Looks like that feature flag filters out queues with the wrong characteristic. Ignore the warning and remove deprecated_features.permit.transient_nonexcl_queues until it is corrected.

[michaelklishin]

Addressed in #12840 by @mkuratczyk.