Iperf does not complete on the client side

[zergeii]

NOTE: The iperf3 issue tracker is for registering bugs, enhancement
requests, or submissions of code. It is not a means for asking
questions about building or using iperf3. Those are best directed
towards the iperf3 mailing list at iperf-dev@googlegroups.com or
question sites such as Stack Overflow
(http://www.stackoverflow.com/). A list of frequently-asked questions
regarding iperf3 can be found at http://software.es.net/iperf/faq.html.

Context

• Version of iperf3: 3.1.7

• Hardware:
  (Client) CentOS Machine deployed on Vmware Desktop (4vcpu,4GB ram)
  (server) AWS EC2 Linux 2 machine (t2.micro)

• Operating system (and distribution, if any): centOS 7

Please note: iperf3 is supported on Linux, FreeBSD, and macOS.
Support may be provided on a best-effort basis to other UNIX-like
platforms. We cannot provide support for building and/or running
iperf3 on Windows, iOS, or Android.

• Other relevant information (for example, non-default compilers,
  libraries, cross-compiling, etc.): running on a internet overlay prvate network slution

Please fill out one of the "Bug Report" or "Enhancement Request"
sections, as appropriate.

Bug Report

• Expected Behavior:
  Normal Iperf3 test that will run for ten seconds

• Actual Behavior
  Iperf3 on the client side will stop up to 9th second

Iperf3 on the server side will continue to process more than 10 seconds up to a certain point in time.

CLIENT SIDE:

[nfadmin@122 ~]$ iperf3 -c 10.10.3.243
Connecting to host 10.10.3.243, port 5201
[  4] local 122.3.238.245 port 55710 connected to 10.10.3.243 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  22.5 MBytes   189 Mbits/sec  225    441 KBytes
[  4]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    0    246 KBytes
[  4]   2.00-3.00   sec  1.25 MBytes  10.5 Mbits/sec    0   87.7 KBytes
[  4]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    0    262 KBytes
[  4]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    0    110 KBytes
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0    181 KBytes
[  4]   6.00-7.00   sec  1.25 MBytes  10.5 Mbits/sec    0   76.4 KBytes
[  4]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    0    205 KBytes
[  4]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0    175 KBytes
^C[  4]  10.00-86.08  sec  0.00 Bytes  0.00 bits/sec    0    204 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-86.08  sec  25.0 MBytes  2.44 Mbits/sec  225             sender
[  4]   0.00-86.08  sec  0.00 Bytes  0.00 bits/sec                  receiver
iperf3: interrupt - the client has terminated
[nfadmin@122 ~]$ ^C
[nfadmin@122 ~]$

SERVER Side:

       __|  __|_  )
       _|  (     /   Amazon Linux 2 AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-2/
[ec2-user@ip-10-10-3-243 ~]$
[ec2-user@ip-10-10-3-243 ~]$
[ec2-user@ip-10-10-3-243 ~]$ iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.10.3.70, port 51998
[  5] local 10.10.3.243 port 5201 connected to 10.10.3.70 port 52000
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   291 KBytes  2.38 Mbits/sec
[  5]   1.00-2.00   sec   204 KBytes  1.67 Mbits/sec
[  5]   2.00-3.00   sec   423 KBytes  3.46 Mbits/sec
[  5]   3.00-4.00   sec   251 KBytes  2.05 Mbits/sec
[  5]   4.00-5.00   sec   320 KBytes  2.62 Mbits/sec
[  5]   5.00-6.00   sec   272 KBytes  2.23 Mbits/sec
[  5]   6.00-7.00   sec   263 KBytes  2.15 Mbits/sec
[  5]   7.00-8.00   sec   240 KBytes  1.97 Mbits/sec
[  5]   8.00-9.00   sec   247 KBytes  2.02 Mbits/sec
[  5]   9.00-10.00  sec   231 KBytes  1.89 Mbits/sec
[  5]  10.00-11.00  sec  16.0 KBytes   131 Kbits/sec
[  5]  11.00-12.00  sec   443 KBytes  3.63 Mbits/sec
[  5]  12.00-13.00  sec   216 KBytes  1.77 Mbits/sec
[  5]  13.00-14.00  sec  57.3 KBytes   469 Kbits/sec
[  5]  14.00-15.00  sec   363 KBytes  2.97 Mbits/sec
[  5]  15.00-16.00  sec   192 KBytes  1.57 Mbits/sec
[  5]  16.00-17.00  sec   192 KBytes  1.57 Mbits/sec
[  5]  17.00-18.00  sec   196 KBytes  1.61 Mbits/sec
[  5]  18.00-19.00  sec   172 KBytes  1.41 Mbits/sec
[  5]  19.00-20.00  sec   112 KBytes   917 Kbits/sec
[  5]  20.00-21.00  sec   247 KBytes  2.02 Mbits/sec
[  5]  21.00-22.00  sec   171 KBytes  1.40 Mbits/sec
[  5]  22.00-23.00  sec   165 KBytes  1.35 Mbits/sec
[  5]  23.00-24.00  sec   164 KBytes  1.34 Mbits/sec
[  5]  24.00-25.00  sec   133 KBytes  1.09 Mbits/sec
[  5]  25.00-26.00  sec   169 KBytes  1.39 Mbits/sec
[  5]  26.00-27.00  sec   144 KBytes  1.18 Mbits/sec
[  5]  27.00-28.00  sec   145 KBytes  1.19 Mbits/sec
[  5]  28.00-29.00  sec   142 KBytes  1.17 Mbits/sec
[  5]  29.00-30.00  sec  57.8 KBytes   473 Kbits/sec
[  5]  30.00-31.00  sec   216 KBytes  1.77 Mbits/sec
[  5]  31.00-32.00  sec   133 KBytes  1.09 Mbits/sec
[  5]  32.00-33.00  sec   134 KBytes  1.10 Mbits/sec
[  5]  33.00-34.00  sec   128 KBytes  1.05 Mbits/sec
[  5]  34.00-35.00  sec   134 KBytes  1.10 Mbits/sec
[  5]  35.00-36.00  sec   125 KBytes  1.02 Mbits/sec
[  5]  36.00-37.00  sec   128 KBytes  1.05 Mbits/sec
[  5]  37.00-38.00  sec   126 KBytes  1.03 Mbits/sec
[  5]  38.00-39.00  sec   125 KBytes  1.02 Mbits/sec
[  5]  39.00-40.00  sec   124 KBytes  1.01 Mbits/sec
[  5]  40.00-41.00  sec  79.3 KBytes   649 Kbits/sec
[  5]  41.00-42.00  sec   153 KBytes  1.26 Mbits/sec
[  5]  42.00-43.00  sec   106 KBytes   870 Kbits/sec
[  5]  43.00-44.00  sec   113 KBytes   925 Kbits/sec
[  5]  44.00-45.00  sec   112 KBytes   914 Kbits/sec
[  5]  45.00-46.00  sec   105 KBytes   859 Kbits/sec
[  5]  46.00-47.00  sec   105 KBytes   859 Kbits/sec
[  5]  47.00-48.00  sec  73.9 KBytes   605 Kbits/sec
[  5]  48.00-48.37  sec  47.7 KBytes  1.06 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-48.37  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-48.37  sec  8.30 MBytes  1.44 Mbits/sec                  receiver
iperf3: error - unable to receive results: Resource temporarily unavailable

• Steps to Reproduce

• Possible Solution

Please submit patches or code changes as a pull request.

Enhancement Request

• Current behavior

• Desired behavior

• Implementation notes

If submitting a proposed implementation of an enhancement request,
please use the pull request mechanism.

[davidBar-On]

@zergeii note that while the average network throughput is about 180Kbytes/sec (=1.44Mbits/sec per the server log), the client sent 22.5MB in the first second. It means that the system buffers, probably determined by the Window Size, can hold 22.5MB. The client stopped sending after 10 seconds as expected, but since it already sent 25MB, it would take about 140 seconds (25MB/180KB) to send the buffered data. This is probably the reason for what you see.

To very that this is the case, try to reduce the window size in the client, using the -w option, so only small amount of data will be buffered. E.g. on the client side set window size to 256KB: -w 256K.

[zergeii]

Thanks @davidBar-On. The one big hit at the start of the iperf always happens, does it mean that the client sees that it can send that much data but in reality the pipe allows lesser than that?

Also there are times that the data sent is 0.0 kbits/sec. What does it mean?

[davidBar-On]

@zergeii, did you try setting -w 256K? If yes, did it make any change?

As I explained, the problem is probably the buffering at the client side which is much larger than what is reasonable for the pipe to the server. Another way to test if this is the case is to set test time to 10 second (the default, -t 10) and let the client continue running. Per my previous calculation the test may take up to 3 minutes.

Regarding the times of 0.0 kbits/sec. This usually happens in your conditions with low throughput pipe to the server. Because the buffers at the client side are almost full, the client cannot send new packets until enough data was sent to the server.