Decreased results since November 2023

[shalak]

Hello!

Since around November 2023, my results became much slower, by a factor of 5:

The problem is, that whenever I run the test via official https://speedtest.net web interface, I'm getting much better results, just like I used to get here.

I did some tests, and the results from sivel's speedtest-cli are also far lower than the official ones, also by a factor of 5

I installed the Ookla's official CLI tool and the results seem somewhat better, but still worse than what I see on the web - by a factor of 3.

Has anyone experienced such issues?

[mrflowjoe]

I do hourly speedtests and noticed changes between my old install and a new install I made. The old ones were way faster.
I noticed, that the old ones are running every hour at 55 minutes and the new ones at 0 minutes.
I changed the new runtimes to 5 minutes past the full hour and get better results.

[shalak]

Have you tried executing a test directly from the container using the cli?

For the future - what's the command? I don't see it in the docs

Nevertheless, I just noticed - I have no idea why, but since February - it all looks good, as it used to:

I tuned the cron on 30th of January, two days later - everything was better.

(I'm seeing weird gaps in the cron - above there's a gap between 3rd and 5th of February, but that seems like a different issue)

[rursache]

same issue here, i get 2gbps down and 1 gpbs up using the speedtest net website with a fixed speed test server. using the same speed test server with speedtest-tracker in docker gets me 500mbps down and 800mbps up. same device, docker vs chrome

[alexjustesen]

@shalak from within the container you can run speedtest --accept-license --accept-gdpr to use the command line tool.

[alexjustesen]

In #1263 (comment) we think we found the issue when it comes to scheduled tests. If someone on this thread can also verify if this setting impacts their tests that be awesome.

[seh2000]

I run scheduled test each 30 minutes using the same fixed test server.
No strange behaviour, get more or less same results when running from web browser and as well from within the container.
Not sure how to go further back more than one moth.

[skyeci-draytek]

Offset cured it for me using the same server that was giving me poor upload on the hour. Offset results as below

[Runesmite]

I noticed for about a month now as well (Feb 10th, to be exact), my upload has been incredibly inconsistent, but if I check other devices (or even the host machine) from the Speedtest website, it's a much better result.

I've changed the offset, I've updated docker, updated to the latest Speedtest-Tracker version, updated BIOS, ubuntu, and I'm really not sure what else it could be causing this inconsistency. I have Symmetrical gig, and sometimes my upload will be 900 Mbps, and the next test it's 100, which I can't replicate (on the web) anywhere else. I've had AT&T out to check, and it seems they can't find anything.

I've tried the old version of Henry Whitaker's just to see what would happen, and strangely enough I'm getting the same issues there, only upload is affected, and the download is fine. I feel a little bit like I'm going crazy, but could this be related to a docker update perhaps?

I also have done double my normal amount of speedtests troubleshooting this, so I'm waiting for Ookla to block me at this point. I've reduced my cron job for a bit just in case.

[skyeci-draytek]

What sort of network carsd and os? I used to use ubuntu but had issues with strange results. Moved to debian using docker and portainer which cured general results issues aside from the poor upload. Intel nic seems to work best for me.

[seh2000]

Just a comment, I use a Pi 4 / Debian Bullseye, using Docker which give very stable DL/UL results using one Server, it is just not synchronous as not offer here - yet...

[Runesmite]

What sort of network carsd and os? I used to use ubuntu but had issues with strange results. Moved to debian using docker and portainer which cured general results issues aside from the poor upload. Intel nic seems to work best for me.

I have a dual port gig ethernet Intel NIC that I've been using, but I did try using the built in Realtek NIC when I started noticing these issues (Also tried a new Ethernet cable). The OS is ubuntu, which I had been using for over a year with no issues with Speedtest Tracker. I am running my container in docker, and I used to have it running as a simple container, and have since moved to a stack.

I've also experimented with Docker for Windows (on my personal computer, Windows 11) which just has poor network performance in general, but seems to having the same issues.

I've also tried speedtest-cli from outside the docker containers, and it experiences the same slowness (around 100 Mbps Up). However, if I go back to speedtest.net, the results are much more normal, with my upload only dipping to around 800Mbps on some tests. I am wondering whether the issue is with speedtest-cli somehow.

[alexjustesen]

Checkout some of the debugging done here: #1263

[Runesmite]

I appreciate it, I had actually looked at that discussion too, to no avail.

I do believe I have figured out what my issue is, however. AT&T Active Armor, to my knowledge this has been enabled on my service for a long time (I can't remember when it was turned on, I believe years ago when I started the service), but apparently it has started having issues with speedtest-cli.

I discovered this when I did further troubleshooting with OpenSpeedtest on my server. My internal network speeds were rock-solid, and after having some friends test to my network, it was obvious the issue was with my WAN. As I searched through my remaining options, I found Active Armor had caused issues for other people when it came to speeds, and sure enough! I've disabled it, and 16 hours later my slowest test is 880Mbps through speedtest-tracker with most being well above 900Mbps and consistent with web results.

Hopefully this helps anyone else with AT&T, and thank you all for your help with this!