Support different numeric types specified in AMQP arguments

[Ayanda-D]

Describe the bug

We've noticed that client applications specifying numeric arguments packed in different bit-lengths results in the broker encoding to different internal representation types, which can cause what may appear like duplicate AMQP elements from a user perspective, but internally just a different representation. For example using RMQ C++ client, the following code (note, the difference in int32_t and int64_t arg types):

    rmqt::ExchangeHandle exch = topology.addExchange("X1", rmqt::ExchangeType::DIRECT);
    rmqt::QueueHandle queue   = topology.addQueue("Q1");

    rmqt::FieldTable args1{{"x-test-arg", rmqt::FieldValue(bsl::string("any"))},
                           {"group", rmqt::FieldValue(bsl::int32_t(123456))}};

    rmqt::FieldTable args2{{"x-test-arg", rmqt::FieldValue(bsl::string("any"))},
                           {"group", rmqt::FieldValue(bsl::int64_t(123456))}};

    topology.bind(exch, queue, "RK-1", args1);
    topology.bind(exch, queue, "RK-1", args2);

Results in:

On tracing the command assembler when executing the snippet above, you can see the 2 consecutive queue.bind resulting in {<<"group">>,signedint,123456} and {<<"group">>,long,123456} :

17:5:28.251513 <0.7952.0> rabbit_command_assembler:process({method,'queue.bind',
        <<0,0,2,81,49,2,88,49,4,82,75,45,49,0,0,0,0,30,5,103,114,111,117,112,
          73,0,1,226,64,10,120,45,116,101,115,116,45,97,114,103,83,0,0,0,3,97,
          110,121>>}, {method,rabbit_framing_amqp_0_9_1})

17:5:28.251716 <0.7952.0> rabbit_command_assembler:process/2 --> {ok,{'queue.bind',0,<<"Q1">>,<<"X1">>,<<"RK-1">>,false,
                  [{<<"group">>,signedint,123456},
                   {<<"x-test-arg">>,longstr,<<"any">>}]},
    {method,rabbit_framing_amqp_0_9_1}}

17:5:28.251896 <0.7952.0> rabbit_command_assembler:process({method,'queue.bind',
        <<0,0,2,81,49,2,88,49,4,82,75,45,49,0,0,0,0,34,5,103,114,111,117,112,
          108,0,0,0,0,0,1,226,64,10,120,45,116,101,115,116,45,97,114,103,83,0,
          0,0,3,97,110,121>>}, {method,rabbit_framing_amqp_0_9_1})

17:5:28.252096 <0.7952.0> rabbit_command_assembler:process/2 --> {ok,{'queue.bind',0,<<"Q1">>,<<"X1">>,<<"RK-1">>,false,
                  [{<<"group">>,long,123456},
                   {<<"x-test-arg">>,longstr,<<"any">>}]},
    {method,rabbit_framing_amqp_0_9_1}}

Reproduction steps

NOTE: You can also easily reproduce this by creating 2 bindings, one from with Pika and the other with Elixir Client (which abstract types):

Pika

    queue_name = "Q1"
    exchange_name = "X1"
    routing_key = "RK-1"
    arguments = {"group": 123456, "x-test-arg": "any"}
    channel.queue_bind(
        queue=queue_name,
        exchange=exchange_name,
        routing_key=routing_key,
        arguments=arguments,
    )

Elixir

iex(1)> {:ok, conn} = AMQP.Connection.open( host: "localhost", port: 5672, virtual_host: "abc",  username: "abc", password: "abc" )
{:ok, %AMQP.Connection{pid: #PID<0.215.0>}}
iex(2)> {:ok, chan} = AMQP.Channel.open(conn)
{:ok,
 %AMQP.Channel{
   conn: %AMQP.Connection{pid: #PID<0.215.0>},
   pid: #PID<0.226.0>,
   custom_consumer: {AMQP.SelectiveConsumer, #PID<0.212.0>}
 }}
iex(3)> AMQP.Queue.bind(chan, "Q1", "X1", routing_key: "RK-1", arguments: [{"group",123456}, {"x-test-arg", "any"}] )
:ok

Should give the same what appears a duplicate binding as above.

Expected behavior

We expect RabbitMQ to use same internal type for the numeric arg e.g. long for both, regardless the client and the bit-length packing, OR to support both types and not result in things that appear like duplicate bindings to the user/client apps. Seems like a change that touch the protocol aspects which can potentially affect a couple of things - hence leaving this for you folks to fix (we also dont have the time).

Also note, the Management-Plugin assigns type long for numeric arguments on e.g. bindings created on the UI, which it sets here (there's no signedint):
https://github.com/rabbitmq/rabbitmq-server/blob/v3.12.13/deps/rabbit_common/src/rabbit_misc.erl#L388

Additional context

No response

[michaelklishin]

@Ayanda-D that would be a potentially breaking change that will get angry exactly the audience you try to advocate for: users of statically typed languages.

C++ does not consider 100 as i32 and 100 as u32 or 100 as i64 to be the same value, why should RabbitMQ?

I'm afraid this is a very common binary protocol problem that has no universally accepted solution.

[michaelklishin]

@Ayanda-D I'm sorry but I don't buy the argument about the management UI. Any Web UI does not have much choice in what type it uses: 99.999% of Web apps use the most generic value possible because that's how JavaScript and JSON work.

Not only the management UI can use a value different from C++, so can a Java app coded differently, so will Python, JS, Elixir, and any other dynamically typed language where numbers autopromote.

RabbitMQ cannot and should not organize that chaos. Your apps must agree on the values they use.
This can be painful when you have a C++ app and a Python app, but so is the case with all other
data services. They don't all degenerate down to two numerical types (i64 and f64). They ask the apps to agree on something.

Sorry, this is not something realistic for a 17 year old tool with 20+ client libraries.

[michaelklishin]

I would understand if RabbitMQ itself used inconsistent types for what it stores or declares internally. But to my knowledge, in the few places where it does declare a topology, the numerical types are always longs (in the AMQP 0-9-1 type system).

Adopting a rule that requires all C++ clients to use an equivalent of a long for bindings would avoid the problem, and work well with Python clients, JS clients, Elixir clients, and RabbitMQ's plugins that declare topologies.

[Ayanda-D]

But to my knowledge, in the few places where it does declare a topology, the numerical types are always longs (in the AMQP 0-9-1 type system).

That's what we've seen as well, hence highlighting what the Management-Plugin also does - assigning long. Whether a language is statically or dynamically typed, can RabbitMQ use fixed long and for all inbound numeric arguments. What numeric types external clients assign shouldnt matter from broker perspective. Also, this is mainly from the underlying Erlang implementation. Erlang doesnt really care whether signedint or long - numerics are either an integer number or a float https://www.erlang.org/doc/reference_manual/data_types#number

& similarly in other clients like Java, e.g. whether a client specifies an "L" suffix or not, RabbitMQ should simply treat these same (decode to long), these two calls shouldnt cause duplicate bindings:

            Map<String, Object> args = new HashMap<String, Object>();
            args.put("x-test-arg", "any");
            args.put("group", 123456);
            channel.queueBind(QUEUE_NAME, EXCHANGE, ROUTING_KEY, args);

Or,

            Map<String, Object> args = new HashMap<String, Object>();
            args.put("x-test-arg", "any");
            args.put("group", 123456L);
            channel.queueBind(QUEUE_NAME, EXCHANGE, ROUTING_KEY, args);

We feel RabbitMQ should be translating/handling numeric types just as its underlying implementation language, Erlang, does - to a generic number type (which can be referred to as a long). We dont see any benefit of differentiating numeric types in the broker side (except floats, which we also dont use for such protocol layer purposes). Current behaviour seems to cause a bit more confusion to users when they see things like duplicate bindings, from different data types (which can also be assigned by client libraries in an abstracted manner, without users knowledge. e.g. Elixir which always, assigns long https://github.com/pma/amqp/blob/main/lib/amqp/utils.ex#L36-L38 ). We dont have to change external clients, but just use common type for all numerics within the broker. The signedint & long seperation internally in the broker (and in Erlang) seems to be of no use from our assessment. Just bouncing this off your team to make final decision (your call). As users, I'm sure we can take manual means/measures on the client end to avoid these confusing scenarios. (although we feel Rabbit should simply be handling numeric types as underlying Erlang/OTP does). 👍