Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set TCP_NODELAY #122

Merged
merged 1 commit into from
Sep 14, 2023
Merged

Set TCP_NODELAY #122

merged 1 commit into from
Sep 14, 2023

Conversation

shanth96
Copy link
Contributor

@shanth96 shanth96 commented Sep 13, 2023

This PR sets the TCP_NODELAY flag on sockets.

Sometimes, when clients use SSL, Nagle's could cause increased connection latency. We were seeing around 40ms additional latency.

@shanth96 shanth96 marked this pull request as ready for review September 13, 2023 15:26
@shanth96
Copy link
Contributor Author

cc @adrianna-chang-shopify

src/socket.c Outdated Show resolved Hide resolved
@brendar
Copy link
Contributor

brendar commented Sep 13, 2023

Should we enable TCP_NODELAY by default? It seems like more of a footgun than a benefit for mysql connections and there is a solid precedent for it. libmysql appears to set it by default (based on strace), and golang sets it by default for all TCP connections.

@byroot do you have any opinions on this?

@adrianna-chang-shopify
Copy link
Collaborator

Makes sense to me to set the flag by default -- although let's ensure that it can be disabled with tcp_no_delay_enabled: false so that applications can opt out if necessary.

cc @composerinteralia @jhawthorn @matthewd -- GitHub folks, any opinions here?

src/socket.c Outdated Show resolved Hide resolved
@byroot
Copy link
Contributor

byroot commented Sep 13, 2023

@byroot do you have any opinions on this?

Yes. It's extremely common to set it by default on connections that are expected to be on LAN like a DB connection.

I do set it in redis, pretty sure Dalli sets it as well, etc.

@jhawthorn
Copy link
Member

jhawthorn commented Sep 13, 2023

I don't think we should have a tcp_no_delay_enabled option (even to disable). Either the library works better with TCP_NODELAY or it doesn't (though I guess one could want it only for SSL connections). If others do feel strongly that we should keep the option (I don't feel that strongly about removing it but would prefer that) I agree it should be enabled by default.

I don't think the actual protocol would benefit from TCP_NODELAY (it does writes then reads), but it makes sense that with SSL it would. So 👍 from me.

I think our code is also well suited to TCP_NODELAY, I believe we always send large constructed buffers (though that is worth double checking that we don't have consecutive writes which could have been batched).

@shanth96
Copy link
Contributor Author

I think our code is also well suited to TCP_NODELAY, I believe we always send large constructed buffers (though that is worth double checking that we don't have consecutive writes which could have been batched).

In that case, it definitely makes sense to omit the option and set it by default - will make things a lot simpler too.

@shanth96
Copy link
Contributor Author

Updated

@shanth96 shanth96 changed the title Allow setting TCP_NODELAY Set TCP_NODELAY Sep 13, 2023
@kovyrin
Copy link

kovyrin commented Sep 13, 2023

Here is the effect from this change on our Vitess clients:

image

@@ -195,6 +195,14 @@ static int raw_connect_internal(struct trilogy_sock *sock, const struct addrinfo
return TRILOGY_SYSERR;
}

#ifdef TCP_NODELAY
if (sock->addr->ai_family != PF_UNIX) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this make sense?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! We can't and don't need to apply this to unix sockets.

@jhawthorn jhawthorn merged commit 1f86ce1 into trilogy-libraries:main Sep 14, 2023
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants