iperf3 is not able to read "--set-mss" parameter for TCP

[MuhammadMunir12]

Context

I am running iperf3 for TCP and want to set packet size using flag "--set-mss or -M" but it's not piking up this argument.
The same issue if observed for window size parameter.

# iperf3 -s -p 5201
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
iperf3: error - unable to set TCP/SCTP MSS: Bad file descriptor
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

# iperf3 -c 192.168.10.189 --time 10 -M 64B
Connecting to host 192.168.10.189, port 5201
iperf3: error - unable to set TCP/SCTP MSS: Invalid argument

# iperf3 -c 192.168.10.189 --time 10 -w 2M
Connecting to host 192.168.10.189, port 5201
iperf3: error - socket buffer size not set correctly

• Version of iperf3:
  iperf 3.5 (cJSON 1.5.2)

• Hardware:
  Dell R740xd

• Operating system (and distribution, if any):
  OS: RHEL 8.2 VM on OpenStack

Bug Report

Unable to set parameters for iperf3.

• Expected Behavior
  The window size and MTU should be set as specified in command parameters.

• Actual Behavior

• Steps to Reproduce

# iperf3 -s -p 5201
# iperf3 -c 192.168.10.189 --time 10 -M 64B

• Possible Solution
  Don't know.

[davidBar-On]

iperf3 -c 192.168.10.189 --time 10 -M 64B

On my computer, the minimum MSS that can be set is 88 bytes, which means minimum 128 MTU. Did you try with larger values to see what is the minimum MSS that can be set on your machine?

Also note that it may be that TCP will override the value you set. See TCP_MAXSEG description in https://man7.org/linux/man-pages/man7/tcp.7.html.

iperf3 -c 192.168.10.189 --time 10 -w 2M

This is probably because of the window size system limit. In Linux maximum seems to be twice the size of rmem_max / wmem_max (to see the valuaes use sysctl net.core.rmem_max / wmem_max).

[MuhammadMunir12]

@davidBar-On I have enabled jumbo frames (MTU 9000) on my infrastructure.
Contrarily, I was able to set the options in Openshift (where MTU was set to 1500) but not able to set in Openstack (MTU 9000).

For window size, this is limit for read/write:

net.core.rmem_max = 212992
net.core.wmem_max = 212992 

Is this value in bytes or some other unit?

[davidBar-On]

@MuhammadMunir12 regarding:

I have enabled jumbo frames (MTU 9000) on my infrastructure.
Contrarily, I was able to set the options in Openshift (where MTU was set to 1500) but not able to set in Openstack (MTU 9000).

I am not sure I understand what you mean. The issue was setting MSS to 64 bytes, so I am not sure how this is related to setting MTU to 1500 or 9000 bytes.

For window size, this is limit for read/write:
net.core.rmem_max = 212992
net.core.wmem_max = 212992
Is this value in bytes or some other unit?

These values are in bytes (same values as on my computer). Probably meaning that you can set window size up to 212992*2=425986 bytes, which is less than 0.42M, and this is why setting window size to 2M didn't succeed.

[bmah888]

If you really have a need for huge socket buffers, you'll have to set some of these kernel limits first, for example:

% sudo sysctl net.core.rmem_max=1048576
% sudo sysctl net.core.wmem_max =1048576

There's a bit of an assumption you know what you're doing...it's possible to get yourself in trouble by tweaking around with sysctl parameters (usually solvable by a reboot).