Possibility for datagrams to be dropped when not able to send

[iiztp]

Description

When adding a Token-Bucket filter and sending more data than we are allowed to, I found out that there was a lot of delay (the more we wait the more we have delay) between sending and receiving.
After some discussion, it was established that msquic is buffering datagrams when it can't send them immediately.
With this PR, it is now possible to drop them when you can't send them immediately.
I added a flag to the QUIC_SEND_FLAGS : "QUIC_SEND_FLAG_DGRAM_CANCEL_ON_BLOCKED"
When this flag is set, on sending it will check if there is queued messages to send, if there is, it drops (cancels) them.

Testing

Do any existing tests cover this change?

I don't think so.

Are new tests needed?

Maybe...

Documentation

Is there any documentation impact for this change?

Just the new flag : "QUIC_SEND_FLAG_DGRAM_CANCEL_ON_BLOCKED"

Some more infos

This is the one-way delay of some datagrams with a TBF after sending more than we are allowed to, see images.
Without flag (before patch):

With flag:

The values are like this : "{packet number}, {one-way delay in ms}"
The values with the flag are around the same values as I have with UDP (which normally drops packets when it can't send them).

[iiztp]

@microsoft-github-policy-service agree

[codecov]

Codecov Report

Attention: Patch coverage is 95.23810% with 1 line in your changes missing coverage. Please review.

Project coverage is 85.52%. Comparing base (ae542fa) to head (6f716c2).

Files with missing lines	Patch %	Lines
src/core/datagram.c	95.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4320      +/-   ##
==========================================
- Coverage   87.17%   85.52%   -1.66%     
==========================================
  Files          56       56              
  Lines       17354    17375      +21     
==========================================
- Hits        15129    14860     -269     
- Misses       2225     2515     +290     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[nibanks]

Are new tests needed?

Maybe...

Yeah, we will need to add some tests, as well as make the minor edits to the docs.

[nibanks]

You will also need to run .\scripts\generate-dotnet.ps1 to update the .NET files.

[nibanks]

Looking pretty good so far! Thanks!

[nibanks]

The changes looks good, but how do should we add a test to verify it? One minimum option is to update spinquic.cpp to ensure it uses this new flag (should just require changing one number). But ideally, it'd be nice to have a functional test case.

[iiztp]

The changes looks good, but how do should we add a test to verify it? One minimum option is to update spinquic.cpp to ensure it uses this new flag (should just require changing one number). But ideally, it'd be nice to have a functional test case.

Maybe we should just add more than we can send i.e a lot of DatagramSend stuck in a loop until we reach out the MTU which is 1500 if we are over ethernet or wifi (so maybe 2 datagrams of a thousand bytes then multiple datagrams that we can drop) or afaik ~65000 for localhost (2 datagrams of 45 000 bytes and multiple datagrams that we can drop) ?

I don't know if this would work, otherwise we could just create a new CI and run a QUIC server on it with special traffic control rules (i.e bandwidth limit) and check if some packets that we allowed to be dropped are dropped and the others are not but the outputs would be different for each run and wouldn't give us a general idea.

[nibanks]

The changes looks good, but how do should we add a test to verify it? One minimum option is to update spinquic.cpp to ensure it uses this new flag (should just require changing one number). But ideally, it'd be nice to have a functional test case.

Maybe we should just add more than we can send i.e a lot of DatagramSend stuck in a loop until we reach out the MTU which is 1500 if we are over ethernet or wifi (so maybe 2 datagrams of a thousand bytes then multiple datagrams that we can drop) or afaik ~65000 for localhost (2 datagrams of 45 000 bytes and multiple datagrams that we can drop) ?

I don't know if this would work, otherwise we could just create a new CI and run a QUIC server on it with special traffic control rules (i.e bandwidth limit) and check if some packets that we allowed to be dropped are dropped and the others are not but the outputs would be different for each run and wouldn't give us a general idea.

I think a test that queues up more than 10 1000-byte datagrams before it starts a connection, and then calls connection start might trigger the discard notification.

[iiztp]

Quick question on this PR, even though some tests have failed (because they were too long to run or because they had their connection was interrupted (?) ) do I have something more to do? Like, do I have to fix the failed tests? I saw that it was the same thing for everyone about these tests so I don't know?

[nibanks]

I'm not sure what happened, but it looks like you pushed a lot of unexpected new files.

[iiztp]

I'm not sure what happened, but it looks like you pushed a lot of unexpected new files.

Other than merging, I did nothing else :/

[nibanks]

I think the problem is the msquicdocs folder shouldn't be in the main branch. Can you please remove that?

[nibanks]

Can we actually create a new test for this stuff, instead of putting it in this existing one? Also, It's possible, though unlikely, that you don't get any cancelled datagrams if they all happen to fit in the congestion control window. So, I recommend the new test queues (sends) all these datagrams on the connection before you call Client.Start to ensure you've queued up more datagrams than can be sent in the initial congestion window.