RFC: Shared Version Specifications

[boginw]

Provide a mechanism for users to specify reusable version specifications for multiple dependencies. For example, many dependencies are published with the same version. This RFC introduces a syntax for specifying the version in a single place and then allowing reusing it across dependencies.

[ljharb]

Publishing dependent versions like this is something that many consider a bad practice, because it forces releases when none would otherwise be needed, confuses dependency graphs, and causes massive costs in terms of churn and bandwidth to the ecosystem. There’s a reason that projects like Babel moved away from this approach. I don’t think it’d be a good thing for npm to tacitly endorse this approach.

[boginw]

Maybe there is something I am missing here, so feel free to correct me, but I cannot see how this would force releases. These "shared versions" are to be used when defining dependencies, not in publishing packages. I would understand if frameworks felt incentivized to publish new versions of packages that did not need updates, but they would never be forced to. Furthermore, the dependency graphs should be strictly equal after implementing this proposal. Wrt. to Babel, I could not find any reference to what you are referring to. Maybe you could point me in the right direction?

It reads to me like there is some misunderstanding in what this proposal is. This might be me, who hasn't given an apt description of what it is. In short, this proposal should be considered a layer of syntactic sugar, which converts the following:

  ...
  "versions": {
      "framework": "~1.2.3"
  },
  "dependencies": {
      "@framework/core": "versions:framework",
      "@framework/common": "versions:framework",
      ...
  }
  ...

To the following:

  ...
  "dependencies": {
      "@framework/core": "~1.2.3",
      "@framework/common": "~1.2.3",
      ...
  }
  ...

[ljharb]

ok - so you’re just suggesting version numbers be substitutable from a map of alias names elsewhere in package.json?

This only seems like it’d provide value when packages are using dependent versioning. In other words, most of the time you shouldnt have more than one package whose version is pegged like this (even DefinitelyTyped packages won’t likely track the patch number).

[boginw]

Yes, exactly that :)

Most of the time you would not use this feature, as most packages are not using version numbers that follow each other. In my experience with Gradle, such shared versions are usually only used when I depend on multiple packages from a framework, such as Spring Boot packages, most of which use the same version as the "framework".

But this is also useful in NPM. Take for instance @angular where around 50 packages use the same version (our projects usually use around 15 of them). It saves countless CI minutes and PRs for us, if, instead of updating these packages separately, they would update together all at once.

The versions referred to should be as exact as the referred version says they should be. By that I mean, if the latest version of "@framework/core" is "1.2.3" and the latest version of "@framework/common" is "1.2.4" and if the version alias is "~1.2.3" then the version "1.2.3" and "1.2.4" should be fetched respectively.

Version aliases might also be a better name than shared versions :)

[ljharb]

right, but the better thing to do is for angular not to do that :-)

[thescientist13]

I'm pretty sure the main reason for why Angular (and other tools like Babel) do this, is because they are monorepos that publish the same version for all packages at publish time, as opposed to independent package versioning. Thus when upgrading, they often recommend to keep all versions the same in package.json.

edit: That said, for projects like this, I usually just leverage the package manager, like in Yarn, this would / should update them for me in one go

$ yarn upgrade --scope @angular --latest

[ljharb]

Right - but that's a problematic approach, because it forces a publish of packages even when nothing has changed - and worse, it could force a minor version when nothing was added, or a major version when nothing was broken.

[thescientist13]

Fair, but having managed both ways myself, in very closely knit monorepos, it can be very useful when communicating to consumers just upgrade to x.y.z as opposed to having potentially complicated combinations of inter-mixed packages and having to track / document all the possible permutations of which packages / versions can work with other combinations of packages / versions.

Anyway, with the package manager it is usually a non-issue so I am personally ambivalent to this RFC but just adding an anecdote to the conversation.

[boginw]

edit: That said, for projects like this, I usually just leverage the package manager, like in Yarn, this would / should update them for me in one go

Doing a scoped upgrade might not always be desired. Organizations might have several frameworks under the same scope, or there might be one package that does not follow the version of the rest, where one might not want to upgrade. Furthermore, not all of these frameworks are scoped, take for instance react.

Right - but that's a problematic approach, because it forces a publish of packages even when nothing has changed - and worse, it could force a minor version when nothing was added, or a major version when nothing was broken.

I cannot see how this would force anyone to publish packages. They won't be any more forced than they already are. Sure, the authors might be incentivized to do so, as maybe some of their users will complain, but this wouldn't force anything. And regardless these authors already publish packages under the same version.

[ljharb]

@thescientist13 that it’s slightly more work to manage things properly doesn’t mean it’s a good idea to add burden for users in exchange for avoiding the proper work :-)

[ljharb]

@boginw because versions can only be kept in sync if a version is published for every package - but not every package will have changes.

[boginw]

Well, I don't have insights into how big an issue that would be for the registry. I have my doubts that much would change since as far as I know, most of those who publish packages with dependent versions are already publishing new versions of packages where nothing has changed. You could argue, that these publishers should not do that, but they do.

I would like to add, that this feature would lead to fewer PRs, fewer CI minutes used along with fewer MBs of artifacts and logs, and less work, across all repositories, using this feature in conjunction with tools like Dependabot.

[thescientist13]

that it’s slightly more work to manage things properly doesn’t mean it’s a good idea to add burden for users in exchange for avoiding the proper work

Where is the burden on users in this scenario though? I think it's a reasonable argument to make that the lowest burden for users and maintainers with an ecosystem like say angular or babel is to keep all versions in lockstep and following semver.

"@angular/cli": "^12.0.1",
"@angular/core": "^12.0.1",
"@angular/http": "^12.0.1",
"@angular/etc": "^12.0.1",

vs

"@angular/cli": "^12.0.1",
"@angular/core": "^7.2.4",
"@angular/http": "^15.0.0",
"@angular/etc": "^1.0.1",

With the second one, there now becomes a not direct relationship between ecosystem packages and what they support at the sibling level and that the (relative) effort of tracking and documenting that ever expanding graph of interconnectedness would be just as burdensome to both parties, as opposed to just saying: "everyone upgrade to x.y.z".

---

That aside, your comment seems to imply that maintainer's should not have agency for what they feel is the best way to manage their ecosystem for themselves and / or users? I get that you personally have a preference, but is said preference a standard documented somewhere that these projects are willfully neglecting? I am sincere here as I am not aware there is actually a one size fits all solution so any education on the topic here would be useful in the context of this RFC I think. I get the point you make about "phantom" releases but does reducing burden on the maintainer not account for anything either? What is so horrible about that versus having to make user's do all the leg work themselves by having to look up and cross reference packages across a bunch of independent semver iterations? And that's assuming the docs are not out of date. 😬

Again, I'm not necessarily advocating for this RFC, but maybe I'm just too naive in thinking that (especially mature) projects can figure out how to best manage the distribution and versioning of packages for themselves and their community based on their own goals, needs, and preferences? 🤷‍♂️

[ljharb]

@thescientist13 im not saying maintainers should be prevented from doing this; I’m saying npm shouldn’t encourage it by adding a feature like this.

[darcyclarke]

I'm in favor of fixing overrides (that's right, I think it's a bug this isn't already supported). Supporting direct dependency overrides + references gives you what you want (ref).

We can probably backlog this work without having to accept this separate RFC (direct dependency overrides were/are in scope as outlined in the original RFC)

[boginw]

Due to the issues with this RFC relating to encouragement, that might be the best compromise. I would be very happy with that outcome

[darcyclarke]

Removing Agenda label as we've discussed this & will fix the current overrides implementation to help support this usecase as described in earlier comments.