Pass server->client command over a separate socket pair #762
Conversation
mnissler-rivos
force-pushed
the
cmd_socket
branch
from
ccddadf
to
925b2f6
Compare
This change adds support for a separate socket to carry commands in the server-to-client direction. It has proven problematic to send commands in both directions over a single socket, since matching replies to commands can become non-trivial when both sides send commands at the same time and adds significant complexity. See issue nutanix#279 for details. To set up the reverse communication channel, the client indicates support for it via a new capability flag in the version message. The server will then create a fresh pair of sockets and pass one end to the client in its version reply. When the server wishes to send commands to the client at a later point, it now uses its end of the new socket pair rather than the main socket. Corresponding replies are also passed back over the new socket pair. Signed-off-by: Mattias Nissler <mnissler@rivosinc.com>
Many tests have a `sock` global holding the client socket. In order to allow additional client context to be managed, introduce a Client class and replace the `sock` global with `client` global object. Signed-off-by: Mattias Nissler <mnissler@rivosinc.com>
The new tests verify behavior of vfu_sgl_{read,write} for success and
error cases.
Signed-off-by: Mattias Nissler <mnissler@rivosinc.com>
Include rationale and operation of separate server-to-client command channel in the protocol description and indicate which commands should be processed on the separate socket. Signed-off-by: Mattias Nissler <mnissler@rivosinc.com>
mnissler-rivos
force-pushed
the
cmd_socket
branch
from
925b2f6
to
7eb0082
Compare
|
Rebased after #763 was merged. |
|
Now that I understand you're squashing, let me know whether I should break out any of the individual commits here into additional separate PRs - my intention was to make small self-contained commits in order to simplify reviews and keep the history clean for folks coming after us. |
|
Btw. I think it makes sense to break out 4025f6e into a separate MR so the mechanical s/sock/client/ changes don't clutter this MR up? |
yep |
|
As discussed, I have now broken out various bits and pieces into separate merge requests: #772 - Introduce client object in python tests And there's another one that we haven't discussed yet. This is preparation work for the server version capabilities to include the index of the socket file descriptor in the ancillary data array: #771 - Construct server capabilities using json-c I think it'll be best if we work through these preparatory MRs first, then I can rebase this one eventually. Or I can alternatively update this one to the final version now - let me know if you have a preference. I will also go through the comment threads here and resolve what has been addressed already. |
This pull request changes the VFIO-user protocol to optionally use a separate pair of sockets for server->client commands. Doing so simplifies command / reply processing for the case where both server and client send commands concurrently, which has proven problematic per issue #279.
Included are the functional changes, new pytests, as well as protocol specification updates.