Function to get size of useful payload in hg_proc's buffer

[mdorier]

The scenario is as follows: a sender sends an RPC with some data. In the HG_Get_input function, when calling the hg_proc_my_type function to deserialize the input, i see this:

• hg_proc_get_size(proc) returns 4080
• hg_proc_get_size_left(proc) returns 4080
• hg_proc_get_size_used(proc) returns 0

To my knowledge, there is no API to obtain how much bytes of useful data the buffer actually holds.

If I serialize 100, 200, 300 bytes, I always get these same values. When I serialize more than 4080 bytes, I see these values increase accordingly, but (1) under 4080 I can't know what is "useful" data, and above 4080 I can't just hope that Mercury has reallocated the buffer to only the size needed.

Could you add an API function to get the payload size?

My use case is that of an RPC handler that has no knowledge of the semantics of the data, it simply gets access to the Mercury buffer and forwards the data to another software component via a non-Mercury channel (ZMQ). Since the final receiver (the other side of the ZMQ channel) knows the semantics, I can afford to just send the full buffer, but it's a waste of bandwidth.

[soumagne]

Hmm I'm a bit confused, hg_proc_get_size_used() is supposed to do just that? if you look there: https://github.com/mercury-hpc/mercury/blob/master/src/mercury.c#L789 this is what we already do. Are you saying it's not working at all?

[mdorier]

The code you're pointing to is on the sender side, I think. On the receiver side, in the HG_DECODE part of a deserializer, hg_proc_get_size_used() returns 0 (or rather, it seems to return the number of bytes that have been deserialized already).

[soumagne]

ah ok yeah it would do that. OK maybe we should just have a completely different routine then, outside of the proc stuff ?

[mdorier]

A function outside the proc stuff would work too, and would actually be more generic. I just remembered I had a feature requested about that: #691, for monitoring purpose. So that's two use-cases that could use such a function.

[soumagne]

I just added the new routines in #753, have a look and let me know if that works for you.

[mdorier]

The API looks good to me, thanks!