Virtio Ring Buffer based DDS

[parai]

For study purpose, I created this vdds project, that by utilizing the virtio ring buffer to implement a DDS for inter-process communication.

For more details, check this https://github.com/autoas/vdds/blob/master/README.md

This a concept project, and it can also be expand to support linux dma_buf as the shared memory between certain hardware and process applicaiton.

[elfenpiff]

@parai very interesting and nicely done!

If you have a look at iceoryx2 you will see that we have adopted the virtio idea there but with some subtle and important difference. We do not use ref counting at all, otherwise one process destroys the ref count and everything goes to waste. Every connection has its own return channel (in your terms avail ring).

Btw. iceoryx2 works without a central daemon. But getting rid of a central daemon comes with huge challenges.

I think your initial idea is a very good first step, but soon you have to answer questions like:

• What happens when one of the subscriber dies/crashes, does it mean the publisher leaks the memory in the descriptor ring?
• What happens when a crashed subscriber rejoins or joins late? How can you deliver all the older data so that it is up to speed?
• What happens when the publisher crashes? Is the subscriber at least functional?

These are the issues RouDi (our central daemon) addresses and iceoryx2 has here decentralized but much more complex algorithms in place to handle this.

In iceoryx2 the used ring and available ring is implemented by the zero copy connection concept, we call the descriptor ring the data segment of the publisher which is implemented by the shared memory concept

Btw. we founded a company called ekxide.io to provide services around the iceoryx eco-system. So if you would like to have a deep-dive, just mail us: "info@ekxide.io".

[parai]

Hi @elfenpiff :
Nice to hear your reply so quickly, I do spent some time to research iceoryx and try run some examples, but I notice that it seems that iceoryx dynamicly allocates a memory from the segment heap when call API to load a buffer, it's not like the virtio ring that when it was created the shared buffers are fixed to a certain numbers. And for the iceoryx, the publisher has no way to know that has all the consumer done the use of the shared buffer. In certain case, the publisher may wish to know that the shared buffer are released by all the consumers.

And for the questions you point out, I have noticed that's really a big problem and in my current VDDS, I have some monitor strategy to detect those issue such as a DESC is hold for a long time by consumers, but it's true that can't tell is that because of the consumer applicaiton crash or stuck. This VDDS is a concept project, not to compete with iceoryx, I just share the idea here that in case if possible that iceoryx can take such idea and this may help others, certain user may wish this features.

1. A fixed set of buffers used for the communication
2. The producer has a way to know the life cycle of the buffer, better that callback to notify the application that all the consumers release the buffer. (Now in VDDS, if the publish call load to get the sample buffer again, then we can know that the sample buffer is free now as released by all the consumer)
3. The set of buffers can be customized to be allocated by the users, such as allocate from linux dma_buf.

And for the iceoryx2 you shared, first time to notice this new rust project, let me check the design concept of how the virtio idea was used.

Thanks!