Libraries could not be resolved by the "cdnjs" provider

[duykhiem]

Describe the bug

We are having an issue with the "cdnjs" provider when building our project in both Hanoi, Vietnam, and Grimstad, Norway.
It is happening with both local build and Azure DevOps Pipeline build.
Failed on some random libraries like jquery@3.4.1, tabulator@4.4.1, tempusdominus-bootstrap-4@5.1.2...

Error:

Restore operation started
##[error]libman.json(0,0): Error LIB002: The "jquery@3.4.1" library could not be resolved by the "cdnjs" provider
     1>libman.json : error LIB002: The "jquery@3.4.1" library could not be resolved by the "cdnjs" provider 
         One or more libraries failed to restore
         
Build FAILED.

       (LibraryManagerRestore target) -> 
         **libman.json : error LIB002: The "jquery@3.4.1" library could not be resolved by the "cdnjs" provider** 

To Reproduce

Steps to reproduce the behavior:

1. Using this version of Visual Studio 2022
2. Build the solution
3. See error:
   

Expected behavior

Build and restore client libraries successfully

Screenshots

Additional context

• We are using cdnjs as the default provider in LibMan
• It was working perfectly before but suddenly failed today without any changes related to Libman or package references

LibMan CLI version: 2.1.161+abc97ecc7d.RR
Visual Studio 2022
Microsoft.Web.LibraryManager.Build" version="2.1.161" targetFramework="net472"

[duykhiem]

The workaround is to clear Libman cache
cdnjs/cdnjs#14141 (comment)

[stebueh]

possibly related to cdnjs API change, see cdnjs/cdnjs#14140; it's being rolled out partially, which would explain the intermittent failure

[duykhiem]

The workaround is to clear Libman cache cdnjs/cdnjs#14141 (comment)

Ah. Cleaning cache is just a lucky shot

I think the main issue is related to
cdnjs/cdnjs#14140

Libman still uses the old endpoint:

LibraryManager/src/LibraryManager/Providers/Cdnjs/CdnjsCatalog.cs

Line 23 in 8dd0ac5

public const string MetaPackageUrlFormat = "https://api.cdnjs.com/libraries/{0}"; // https://aka.ms/goycwu/{0}

Failed restore/install fall into their 10% production request
cdnjs/cdnjs#14140 (comment)

I think clean cache somehow and sometime helps our request to fall outside of above 10% :D

[duykhiem]

possibly related to cdnjs API change, see cdnjs/cdnjs#14140; it's being rolled out partially, which would explain the intermittent failure

yes I think so

[MattIPv4]

Hey folks, just wanted to note that after a quick read through the code here, it does seem like this is caused by the deprecation of the assets property returning every version on cdnjs.

The fix for libman here would be to use the versions property to determine the library versions, not assets. And then to use the api.cdnjs.com/libraries/:library/:version endpoint when it needs to actually get the assets for a specific version.

[jthomas81]

Does someone have an example of syntax to do this? This is what I have been doing:
{
"version": "1.0",
"defaultProvider": "cdnjs",
"libraries": [
{
"library": "jquery-validate@1.17.0",
"destination": "wwwroot/lib/jquery-validate/"
}
]
}

[the-doctor1994]

Does someone have an example of syntax to do this? This is what I have been doing:

it's not a change for the end user, MattIPv4 is suggesting LibMan issue an update to their code, something that could take days, to conform to the altered API behavior.... instead of just reverting the cdnjs api change that is causing this issue

[MattIPv4]

Unfortunately, LibMan is not the only consumer of the cdnjs API, and the deprecation that caused this issue is part of larger ongoing work across the API to improve stability (the cdnjs API often runs 5xx errors for assorted consumers, not just LibMan users). We're not really in a position where we can revert this API change as doing so will only increase the failure rate for the API generally.

[the-doctor1994]

I understand the change is necessary and inevitable, but time could and should be given to allow for proper deprecation updates by all the users of the API who may be negatively affected (or outright broken) by this change. By your own proposal the deprecation was fast-tracked under the incorrect assumption nobody would be affected...

[MattIPv4]

I appreciate what you're saying, and this is why we're approaching this as a slow rollout with only 10% of traffic reaching endpoints that implement this deprecation, so that folks are made aware of if they're impacted by the deprecation (turns out it's pretty impossible to determine who consumes what properties of a public API just looking at unauthenticated GET requests), but can also still easily get around the deprecation by just re-requesting (falling into the 90% of traffic that is unaffected at present).

Until we hear more here in terms of what a fix would look like on LibMan's end, we don't intend to further rollout the change, so it will remain at 10% for now. That being said, ultimately we will have to continue with this deprecation as until we complete this rollout a large number of requests to the API from all consumers, not just LibMan users, will continue to fail unexpectedly.

[jimmylewis]

Just quickly jumping in here to say: I'm working on the code change for LibMan, but the rollout will require servicing each version of Visual Studio that includes it (2017, 2019, 2022). There will be some delays to prepare the LibMan builds and VS servicing changes in each product, and then ultimately it will require customers to install the updates as well.

I don't have timelines yet, I'm focusing first on making the code change for VS2022 and our CLI tool, then I will start investigating the servicing requirements for each VS release.

[duykhiem]

I think the safest and fastest way to workaround this issue is to include the successfully generated static files by LibMan to our project (when our restore client-side libraries command falls into the 90% of traffic that is unaffected at present) and then disable LibMan. We can re-enable it when the issue is fixed.

Another workaround is try to use other CDN providers like filesystem, jsdelivr, unpkg instead of cdnjs for LibMan

Cleaning LibMan cache is not really a workaround for this issue.
Our build will still have a failure probability of 10% even though we tried to clear the LibMan cache

[gavin-theta]

I'm not sure if this will help but in our case we had a "dotnet restore" followed by a "dotnet publish" the publish was restoring the packages again. So I disable that by "dotnet publish --no-restore" and the build is now happy.

[ennerperez]

Also, Azure Pipelines are failing too.

[gavin-theta]

Also, Azure Pipelines are failing too.

Try my fix above. The azure restore seems to work as a different step. Then disable the restore in the publish. Worked for me.

[jimmylewis]

Alright, for folks using the CLI tool or MSBuild task could you please update to the latest version (2.1.174) and see if you have any issues with that? From a few projects I've tried, it's working fine. This will be the fastest way to workaround the issue (at least for unblocking CI or local builds) as the VS servicing cycle will take a longer to sort out.

Timeline on VS updates still TBD...

[calvaradocl]

On our azure pipelines we are getting this error after updating the package to the 2.1.174 version, this persists even if we have Newtonsoft.Json installed.

/root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: The "Microsoft.Web.LibraryManager.Build.RestoreTask" task failed unexpectedly. [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: System.TypeInitializationException: The type initializer for 'Microsoft.Web.LibraryManager.Cache.WebRequestHandler' threw an exception. ---> System.TypeInitializationException: The type initializer for 'Microsoft.Web.LibraryManager.Configuration.Settings' threw an exception. ---> System.IO.FileLoadException: Could not load file or assembly 'Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'. Could not find or load a specific file. (Exception from HRESULT: 0x80131621) [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Web.LibraryManager.Configuration.Settings..ctor() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Web.LibraryManager.Configuration.Settings..cctor() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: --- End of inner exception stack trace --- [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Web.LibraryManager.Configuration.ProxySettings.get_Default() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Web.LibraryManager.Cache.WebRequestHandler..cctor() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: --- End of inner exception stack trace --- [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Web.LibraryManager.Build.Contracts.Dependencies.Initialize() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Web.LibraryManager.Build.RestoreTask.Execute() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [/src/Frontend/Frontend.csproj] 2 Warning(s) 1 Error(s)

[stebueh]

updating the MSBuild task fixed the issue for me building locally, thank you @jimmylewis for the timly fix; our azure devops pipeline is using the CLI tool and was running flawlessly all the time, so I guess it is/was still on the 90% of traffic unaffected by the API change.

[ardalis]

I appreciate what you're saying, and this is why we're approaching this as a slow rollout with only 10% of traffic reaching endpoints that implement this deprecation, so that folks are made aware of if they're impacted by the deprecation (turns out it's pretty impossible to determine who consumes what properties of a public API just looking at unauthenticated GET requests), but can also still easily get around the deprecation by just re-requesting (falling into the 90% of traffic that is unaffected at present).

Running into Hyrum's Law:

With a sufficient number of users of an API,
it does not matter what you promise in the contract:
all observable behaviors of your system
will be depended on by somebody.

[farhadmammadli]

cdnjs issue is resolved in the new update of the LibraryManager package.
the updated version is 2.1.174

[thiago-bessa]

I just tried the new version, and I'm still having problems:

C:\Users\thiago.bessa>dotnet tool install -g Microsoft.Web.LibraryManager.Cli
You can invoke the tool using the following command: libman
Tool 'microsoft.web.librarymanager.cli' (version '2.1.174') was successfully installed.

libman restore
[LIB002]: The "jquery@2.2.4" library could not be resolved by the "cdnjs" provider
[LIB002]: The "jquery-validate@1.19.1" library could not be resolved by the "cdnjs" provider

[jimmylewis]

@thiago-bessa try again after running libman cache clean? We use the cache library metadata to disk for 1 day (IIRC of the top of my head) to avoid hitting the service repeatedly on queries, so if you have recently cached metadata from the older build, it might be conflicting with the changes.

I was able to restore fine with this libman.json file

{
  "version": "1.0",
  "defaultProvider": "cdnjs",
  "libraries": [
    {
      "library": "jquery@2.2.4",
      "destination": "lib\\jquery"
    },
    {
      "library": "jquery-validate@1.19.1",
      "destination": "lib\\jquery-validate"
    }
  ]
}

[farhadmammadli]

I just tried the new version, and I'm still having problems:

C:\Users\thiago.bessa>dotnet tool install -g Microsoft.Web.LibraryManager.Cli
You can invoke the tool using the following command: libman
Tool 'microsoft.web.librarymanager.cli' (version '2.1.174') was successfully installed.

libman restore
[LIB002]: The "jquery@2.2.4" library could not be resolved by the "cdnjs" provider
[LIB002]: The "jquery-validate@1.19.1" library could not be resolved by the "cdnjs" provider

have you tried to clean cache using this?
libman cache clean

[thiago-bessa]

Sorry @farhadmammadli and @jimmylewis, I didn't copy/paste everything.

I did run libman cache clean before running libman restore, and it still failed.

Now I ran it an hour later (same cmd prompt, with the same project), and it could restore everything.

Then I ran it in other project of ours (after the first one restored correctly), and it again failed to resolve:

Then I just ran restore for 3 times in a row, it finally worked. It seems kind of intermittent.

[JosephAmalfitanoSSA]

any ETA?

[MattIPv4]

@JosephAmalfitanoSSA The releases with the fix were made available mid July: #685 (comment)

[JosephAmalfitanoSSA]

@MattIPv4 I noticed that, but i see the ticket is still marked open. I assumed because it wasn't working again. I'm still running into.

I've tried:

• updating VS to latest stable
• updating VS to latest preview build (17.4.0 preview 1.0)
• manually installing: dotnet tool install -g Microsoft.Web.LibraryManager.Cli and running libman cache clean & libman install
• maybe something else i can't remember

All results are the same X library could not be resolved by the "cdnjs" provider.

[WyrmUK]

Update VS to latest stable AND update the Library Manager (Microsoft.Web.LibraryManager.Build) to the latest in the project.

[JosephAmalfitanoSSA]

Update VS to latest stable AND update the Library Manager (Microsoft.Web.LibraryManager.Build) to the latest in the project.

Ahh -- that was the "maybe something else i can't remember"...tried as well. If it's working for the general population, then maybe it's a proxy issue on my end then.

[jimmylewis]

If you can capture network traces of the requests sent by VS or by the libman tool, we can try to figure out what is going wrong (either the message is getting caught by a proxy, an error being returned from the API, or unexpected response content).

[mmcdonald47]

The type initializer for 'Microsoft.Web.LibraryManager.Cache.WebRequestHandler' threw an exception. ---> System.TypeInitializationException: The type initializer for 'Microsoft.Web.LibraryManager.Configuration.Settings' threw an exception. ---> System.IO.FileLoadException: Could not load file or assembly 'Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'. Could not find or load a specific file. (Exception from HRESULT: 0x80131621)

I'm also running into this Newtonsoft.Json issue in my CI pipeline, using Microsoft.Web.LibraryManager.Build 2.1.175

[JosephAmalfitanoSSA]

Downloading file https://cdnjs....

My issue was a proxy issue. It looks like if you have a HTTP_PROXY and/or HTTPS_PROXY value set as an environment variable, then libman traffic ends up routing traffic through it.

Steps for me to remedy

1. Renamed those system/environment variables to xHTTP_PROXY and xHTTPS_PROXY
2. Installed / restored libraries
3. Renamed those system/environment variables back to HTTP_PROXY and HTTPS_PROXY
   🕺

[jimmylewis]

@mmcdonald47 can you capture a Fusion log while running the build? The NuGet package includes a copy of that assembly, so I'm wondering if that log can help clarify why it's not being located. (See how to capture Fusion logs at https://docs.microsoft.com/en-us/dotnet/framework/tools/fuslogvw-exe-assembly-binding-log-viewer)

[mmcdonald47]

@jimmylewis how can I install this in my Azure CI pipeline? I've tried adding a Command Line Script task, but when running the build it failed because it couldn't recognize fuslogvw.

(Apologies for my ignorance. I've not had much experience configuring CI pipelines.)

[jimmylewis]

@mmcdonald47 I assume this only occurs on your CI and not locally? In that case, we can try doing it with via the registry, as in https://stackoverflow.com/questions/255669/how-to-enable-assembly-bind-failure-logging-fusion-in-net

The EnableLog key will enable the log, and the text will be included in the thrown exception's message. This might or might not show up in the message you showed above (since it's wrapped, I'm not sure if it will show the full message).
The LogFailures key will enable logging failures to a file, and LogPath is (obviously) the log path. Make sure you remove these keys afterwards, or else it will affect any other assembly loads on the machine (and can eventually consume the remaining disk space).
For testing purposes, you could set these then add the LogPath as a build artifact, and see if there's a logged file for the Newtonsoft.Json binding failure.

[mmcdonald47]

@jimmylewis per that SO post I was able to locate the Fusion Log Viewer and get some more information after running my application locally. In checking the logs I see this message

The operation failed.
Bind result: hr = 0x80004005. Unspecified error

Assembly manager loaded from:  C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Running under executable  C:\Program Files\IIS Express\iisexpresstray.exe
A detailed error log follows. 

Pre-bind state information
LOG: DisplayName = System.Numerics, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
 (Fully-specified)
LOG: Appbase = file:///C:/Program Files/IIS Express/
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = NULL
LOG: AppName = iisexpresstray.exe
Calling assembly : Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed.

LOG: Start validating all the dependencies.
LOG: [Level 1]Start validating native image dependency mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089.
Rejecting native image because native image dependency C:\WINDOWS\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll had a different identity than expected
LOG: Start validating all the dependencies.
LOG: [Level 1]Start validating native image dependency mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089.
Native image has correct version information.
LOG: Validation of dependencies succeeded.
LOG: Bind to native image succeeded.

[duykhiem]

The changes to Visual Studio are now available in the 16.9.23, 16.11.17, 17.0.12, 17.2.6, and 17.3 Preview 3 releases. Please let me know if you are still having issues after updating to one of these versions. In my testing with each of them, I was able to install the latest and non-latest versions of multiple packages and did not encounter any issues.

Updating to Microsoft.Web.LibraryManager.Build.2.1.175 and Visual Studio 2022 version 17.3.1 fixed this issue for me.
Thanks @jimmylewis :)

[mentormate-ivandzhunov]

I also have the issue with Newtonsoft.Json, Version=13.0.0.0 during deployments. Did anyone found a solution for it?

[WyrmUK]

If you are using Newtonsoft directly then upgrade it to the latest v13. If you are including another package that uses Netwonsoft then upgrade that. Or add Newtonsoft v13 and set it to be used for all versions. Newtonsoft less than v13 has a vulnerability so should no longer be used. This is not on Topic though and there is enough help for mismatched package versions on stack overflow etc.

[mmcdonald47]

@mentormate-ivandzhunov you can try the solution in this StackOverflow post: https://stackoverflow.com/questions/46111749/adding-a-bindingredirect-to-a-net-standard-library/46120907#46120907

My solution was to uninstall LibraryManager, though that doesn't seem like a real solution to me.

[paillave]

At my side, in my csproj, I made the following change, and it finally works gracefully!!!

from

<PackageReference Include="Microsoft.Web.LibraryManager.Build" Version="2.1.161" />

to

<PackageReference Include="Microsoft.Web.LibraryManager.Build" Version="2.1.175" />

[paillave]

The changes to Visual Studio are now available in the 16.9.23, 16.11.17, 17.0.12, 17.2.6, and 17.3 Preview 3 releases. Please let me know if you are still having issues after updating to one of these versions. In my testing with each of them, I was able to install the latest and non-latest versions of multiple packages and did not encounter any issues.

Updating to Microsoft.Web.LibraryManager.Build.2.1.175 and Visual Studio 2022 version 17.3.1 fixed this issue for me. Thanks @jimmylewis :)

I confirm

[duykhiem]

This issue is fixed.
I'm going to close it now.

Please create another bugs for other issue!

Thank you very much! 😄

[GeorgeMaharis123]

Is there a workaround for VS2017? The issue is still present there with Microsoft.Web.LibraryManager.Build.2.1.175

[duykhiem]

@GeorgeMaharis123 Did you update VS2017 to the latest version?

[jimmylewis]

@GeorgeMaharis123 unfortunately we hadn't maintained the build infrastructure for VS2017 over the last several years, and our CI configurations aren't currently able to build the rel/vs2017 branch without making some significant changes. However, as a workaround, you should be able to build and install from the tip of that branch with the fix. A locally built VSIX should be able to install on top of the one that comes as part of VS.

This is the first request we've received about this issue on VS2017; our telemetry showed very little remaining usage in that version of VS, so we originally postponed the effort to get the build pipeline running again. I'll see if I can request some resources (i.e. time) to get the CI build running again to put the fix into a servicing patch for VS2017.

[GeorgeMaharis123]

@duykhiem Yes, i am using version 15.9.50

[LLIAMAH]

Visual Studio 2022 Latest updates

Problem not fixed. Cache cleaning does not help. Any ideas? Help?

Microsoft.Web.LibraryManager.Build.2.1.175 - installed.

[jimmylewis]

@LLIAMAH can you get a network trace of the requests and responses?

[LLIAMAH]

@LLIAMAH can you get a network trace of the requests and responses?

I took fiddler stack - if you need some special info please guide me which props i've to switch on to help you to investigate problem.

https://github.com/LLIAMAH/CDNPoblemVS2022/blob/master/FiddlerStack.txt

[jtbrower]

Edit : As I spoke about all of the versions of Visual Studio that I tried to use to overcome this issue below, I had one remaining Visual Studio component listed in the installer as having an available update that should have had absolutely nothing to do with my reported problem. Visual Studio Build Tools 2017 had an available update and after updating 2019, 2022 and 2022 Preview, I went ahead and updated the 2017 Build Tools to 15.9.50 and imediately afterwards, I tried to build my project in vs2019 and it worked. I then did a clean and tried vs2022 and it worked there too.

I don't know how this could have affected this issue or if it is a Red Herring, but I no longer have the problem, for now.

Here is the Original Problem
Hello Everyone, I believe that my companies stable Azure WebApp is now hitting this issue based on what I am seeing. I will try to provide as much information as possible, but I was actually working hot on another project and accidentally discovered a dead link on our company website and thought this would be a quick fix, but I have been on this for hours without luck.

The Error
Note : I replaced the source path with the word redacted to hide company details.

libman.json : error LIB002: The "jquery@3.6.0" library could not be resolved by the "cdnjs" provider [redacted.csproj]
libman.json : error LIB002: The "choices.js@9.0.1" library could not be resolved by the "cdnjs" provider [redacted.App.csproj]

** Most of What I have Tried and I got the Same Error**

• Ran dotnet tool install -g Microsoft.Web.LibraryManager.Cli, then libman cache clean & libman restore
• Tried building through powershell using cli then switched back to troubleshooting with Visual Studio
• I was using an older vs2022 preview build so I updated it to version 17.4.0 Preview 3
• I was using an older vs2022 stable build, then updated to 17.3.6
• I tried using vs2019 v16.11.11 then updated that to v16.11.20
• Opened the the csproj file for the app and updated Microsoft.Web.LibraryManager.Build from 2.1.113 to 2.1.175
• Intellicode suggested that I reorder my properties so I let it place the defaultDestination ahead of defaultProvider
• Deleted bin and obj directories and a handful of other voodoo "standing on one foot and patting my head" nonsense

Beyond That
... I can't promise I will be able to help debug this much further. As I noted I should be working on something else and didn't expect that changing a deadlink on a stable WebApp would be such a major problem. If this is truly closed and fixed and I am doing something odd, forgive me. Otherwise, I don't see how this issue is resolved?