Ns 2 http websocket

[yuroitaki]

#3.

[sinui0]

Great work! 🚀

Although, I'm not sure I'm convinced about the "upgrade" approach for the TCP case. It seems that it introduces unnecessary complexity and I'm not sure how clients are going to behave with it.

I'm now wondering if it would be better to split these services up to different ports. Having to establish a new connection isn't too big of a deal, given we now know that axum/hyper is not very good at supporting what we need. We can put the websocket endpoint on something like ws.tlsnotary.org.

What do you think about the following:

1. New route called /new_session which is used for configuration negotation and issues the prover a session ID as it does now.
2. TCP clients get to reuse the connection, but now hit /notarize to intiiate notarization sending an HTTP Close. If the session id is good, we set a flag indicating the connection info in the Notary global state. We use the without_shutdown provided by axum and when a connection closes we check if the flag is set and act accordingly.
3. Websocket clients hit a different port which we run a separate service on which only provides the websocket handler you've written.

This shouldn't be that much of a backtrack, lmk what you think.

[sinui0]

Mulling this over further, I'm not completely against the "upgrade" approach for TCP clients. However, I do think we should decouple the configuration negotiation/session id issuance from the route for starting notarization.

What do you think about combining the websocket and tcp extractors into 1?

[yuroitaki]

Mulling this over further, I'm not completely against the "upgrade" approach for TCP clients. However, I do think we should decouple the configuration negotiation/session id issuance from the route for starting notarization.

What do you think about combining the websocket and tcp extractors into 1?

Agreed on separating configuration from notarization — it was coupled together because originally I didn't know Axum can reuse the same TCP connection for consecutive HTTP requests — separating will make configuration endpoint much simpler and straightforward to understand and interact with.

For combining websocket and tcp extractor into one, if I understand correctly, it means we only have one handler (hence one endpoint with either https:// or wss://) for both websocket and tcp notarization — I can definitely see more pros than cons here where

(1) from prover developer perspective, they only have one notarization endpoint to deal with regardless of client type and the request format is very similar where it only differs in the value of Upgrade header (and the other necessary headers for websocket), and using wss:// versus using https:// basically.

(2) from notary developer perspective, the pro is that the logic of retrieving session_id and fetching configuration data from the store is centralised in the single handler now. Though one con is that we might need to modify more of Axum's websocket code to achieve this — but I think it can be done minimally by using a 'wrapper-extractor' that contains Axum's websocket extractor (hence not modifying any of its internal code) and our custom tcp extractor — need to experiment on this more to tell.

Please let me know if I understand your statement of combining the extractors correctly, and also if you have any other reasons that I did not think of when you suggested this :)

[sinui0]

Yes, that is essentially what I have in mind for composing the extractors. You would first strip the session ID off, then you would check which protocol is present in the Upgrade header and pass it to the appropriate extractor (TCP or WS)

[sinui0]

Good work!