nuget list -allversions does not return all versions and hangs when server does not observe $top

[stgit123]

NuGet product used: NuGet.exe

NuGet version: 4 and up

Worked before? If so, with which NuGet version: 3.5

Repro steps:

(I am not able to share the details of my package and source)

1. Running in bash
2. nuget list <package> -source <source> -allversions

Expected results: All versions of <package>
Actual results: Only 30 versions of <package> starting with the earliest, so I am not able to see the most recent versions that have been published. Pressing Enter does not show more results.

The cli command works with nuget.exe version 3.5. All versions are shown starting from the earliest to the most recent.

It does not work with any version above 3.5 that I have tried.

[joelverhagen]

Hey @stgit123, what package source are you using? This is a known issue on nuget.org:
NuGet/NuGetGallery#3274

[stgit123]

Hi @joelverhagen thanks for the reply! The package source is specific to my employer so I can't share it unfortunately. I did read the issue you have referenced but as you say it still seems to be unresolved.

Versions of nuget.exe higher than 3.5 seem to be capping results at 30 ordered them from oldest to newest which is not especially useful...

I've switched over to Nuget Package Explorer for now.

[joelverhagen]

Would it be possible to know which NuGet server implementation your company uses internally? For example, NuGet.Server, Artifactory, or NuGetGallery? If you can't share this information, no worries :-)

[stgit123]

According to the higher ups, "We use an open source implementation of the nuget server protocol called sunside/simple-nuget-server"

Thanks for looking into this :)

[joelverhagen]

I'm not an expert in Docker, but it looks like sunside/simple-nuget-server is a Docker image hosted here:
https://hub.docker.com/r/sunside/simple-nuget-server
... which points to here...
https://github.com/sunsided/docker-nuget

This GitHub repo has a submodule pointing to this other GitHub repository which seems to be the core of the NuGet server implementation:
https://github.com/Daniel15/simple-nuget-server

However, it looks like the Docker image pulls down a very old version of the Daniel15/simple-nuget-server which does not have some this bug fix:
Daniel15/simple-nuget-server#27

This issue mentions something about 30 versions so it sounds like that's the root cause: you are using an old version of your NuGet server which still has a pagination bug.

The Docker image GitHub repo has this open bug report:
sunsided/docker-nuget#9

It looks like the Docker image has not been updated in quite a while and is perhaps even abandoned (hard to tell for sure, though).

I can think of a couple workarounds:

1. Clone the Docker image repo (https://github.com/sunsided/docker-nuget) and update the submodule to the latest commit then use that update Docker image instead.

2. Switch server implementation. It looks like you need V2 so you could consider NuGet.Server.

3. (scary) You could maybe even patch the PHP scripts of your running server to have the bug fix.

... but that's just what I can think of.

One thing I am curious about is why your higher ups chose to use this server implementation? What convenience or hosting considerations made this server the most desirable to run? Is it because it was in a Docker image? Or perhaps because your hosting environment requires Linux or PHP?

[stgit123]

@joelverhagen thank you so much for all the research you have done. I'll pass on the info to the boss. I'm assuming we're using this server implementation because it's running in Docker (everything here is run in Docker Linux containers) Edit: I asked and the reply was "I didnt want to run any Winblows stuff" :)

This was a slightly confusing one because if I get version 3.5 of the nuget cli exe from https://www.nuget.org/downloads then everything works as expected.

Thanks again for your help!

[joelverhagen]

From nuget.exe 3.5.0 to 4.0.0 we overhauled our list implementation to remove a dependency on the old NuGet 2.x codebase. During this, the behavior around this edge case changed. I call this an edge case because the server is not observing the $skip and $take (pagination) parameters provided.

The first HTTP request made by nuget.exe list (both 3.5.0 and 4.0.0 do this):

/Search()?$orderby=Id&searchTerm=''&targetFramework=''&includePrerelease=false&$skip=0&$top=30

The client expects up to 30 results to come back but Daniel15/simple-nuget-server returns all versions (even if that number is more than 30). In my own repro, there was one ID with 1 version and one ID with 48 versions (49 total versions).

In nuget.exe 3.5.0, the client handles this break in protocol by making no more requests but printing out all data fetched in that oversized page (all 49 results).

In nuget.exe 4.0.0, the client prints out the first 30 results and issues a query for the next 30 ($skip=30). It sees the same exact first 30 results and ignore them. Then it skips another 30 ($skip=60). It does this forever, at least in my experience. This causes nuget.exe to never terminate but keep issuing HTTP requests with higher and higher skips.

In my opinion, it is definitely a bug in the server (as mentioned in Daniel15/simple-nuget-server#27) to not observe the pagination parameters.

However, it is also a bad behavior in nuget.exe to never terminate and continuously make unnecessary requests. As you mentioned the behavior of the 3.5.0 in this case is better.

I was able to verify things worked better by patching all of the PHP files in the /var/www/inc and /var/www/public directories in the Docker container with the latest versions of the respective files on GitHub. I haven't tested other scenarios but this at least fixed the nuget.exe list problem.

In summary, I think it's probably best for your to workaround this in the short term by updating your server. Let's keep this issue open to track the poor behavior of nuget.exe 4.0.0+ list command.

[stgit123]

@joelverhagen,

Thank you so much once again for the comprehensive explanation. I have passed this all on to the powers-that-be and they will be updating our server very soon (if they haven't done so already)

I really appreciate the time you have take to educate me/us on the issue