Auto-installing peer dependencies in npm@7.

[jansiegel]

I have a question about auto-installing peer dependencies.

In this PR @isaacs mentions that:

we should never place a dep in a target where the target has a peerDep on that dependency

while the Install Peer Dependencies RFC states that:

Ensure that a validly matching peer dependency is found at or above the peer-dependant's location in the node_modules tree.

I'm not sure if there's a contradiction, or am I missing something? Currently, it causes some problems, which I'm not sure how to get around properly:

I created a simple repo: https://github.com/jansiegel/npm7-peerDeps-test

It has workspaces enabled and consists of 2 packages:

• lib1 with a devDependency of "ts-node": "~8.0.2"
• lib2 with a devDependency of "jest": "^26.6.3"

When calling npm i on the repo, it throws an ERESOLVE unable to resolve dependency tree error:

npm ERR! Found: ts-node@8.0.3
npm ERR! node_modules/ts-node
npm ERR!   dev ts-node@"~8.0.2" from lib1@1.0.0
npm ERR!   packages/lib1
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peerOptional ts-node@">=9.0.0" from jest-config@26.6.3
npm ERR! node_modules/jest-config
npm ERR!   jest-config@"^26.6.3" from @jest/core@26.6.3
npm ERR!   node_modules/@jest/core
npm ERR!     @jest/core@"^26.6.3" from jest@26.6.3
npm ERR!     node_modules/jest
npm ERR!       dev jest@"^26.6.3" from lib2@1.0.0
npm ERR!       packages/lib2
npm ERR!     1 more (jest-cli)
npm ERR!   jest-config@"^26.6.3" from jest-cli@26.6.3
npm ERR!   node_modules/jest-cli
npm ERR!     jest-cli@"^26.6.3" from jest@26.6.3
npm ERR!     node_modules/jest
npm ERR!       dev jest@"^26.6.3" from lib2@1.0.0
npm ERR!       packages/lib2
npm ERR!   2 more (jest-runner, jest-runtime)

From what I understand, correct me if I'm wrong:

1. lib1 installs ts-node@8 in ./node_modules/
2. lib2 cannot install ts-node@9 in its own ./packages/lib2/node_modules, installs jest in ./node_modules/ and finds ts-node@8, which doesn't correspond to the optional peer dependency of jest -> @jest/core -> jest-config , so it throws an error.

An even simpler example would be:

./packages/lib1 // peerDependency on `some-package@1`
./packages/lib2 // peerDependency on `some-package@2`

Is there any way to get around it? I can easily imagine a monorepo with n idependent packages with some of their dependencies (or dependencies of their dependencies) relying on different versions of the same peer dependency.

[MylesBorins]

Hey @jansiegel,

I've transferred this discussion to the RFC repo as it is about a specific RFC as opposed to a specific feature request. Also pinged the CLI team to take a look.

[isaacs]

I have a question about auto-installing peer dependencies.

In this PR @isaacs mentions that:

we should never place a dep in a target where the target has a peerDep on that dependency

while the Install Peer Dependencies RFC states that:

Ensure that a validly matching peer dependency is found at or above the peer-dependant's location in the node_modules tree.

Subtle distinction: "in" vs "at".

If foo has a peerDep on bar, and foo is installed at .../node_modules/foo then we may not install bar at .../node_modules/foo/node_modules/bar. That would be "in" foo as the "target" in the terminology used within Arborist. It must be placed at foo's level or higher. Ie, at .../node_modules/bar or more shallow. So if you have node_modules/x/node_modules/foo, then you could have node_modules/x/node_modules/bar (at foo's level) or node_modules/bar (above foo's level).

peerOptional deps are deps that are listed in peerDependencies and also have an entry in peerDependenciesMeta containing {"optional": true}. These sorts of dependencies are not installed by default, but if installed, must be resolved to a version that satisfies the optional peer dependency relationship.

Thus, if foo had a peerOptional dependency on bar@1, then you could not install both foo and bar@2 at the top level.

Is there any way to get around it? I can easily imagine a monorepo with n idependent packages with some of their dependencies (or dependencies of their dependencies) relying on different versions of the same peer dependency.

Perhaps "top" packages (ie, those outside of a node_modules hierarchy) should be able to have their peerDependencies placed within their node_modules folder? Currently we only allow that for the root package (typically the cwd where npm is run, but not necessarily), but you're right, it would perhaps be better to allow them to nest to that degree, since workspaces are in a way their own separate projects.

The caveat would be that if you have a bunch of workspaces which you want to all use the same singleton peerDep, you could easily end up with some of them on different versions. This could even happen as a result of meta-dependencies. For example, say you have 5 projects peer-depending on react@^16, and resolving to 16.12.0. Then one of them adds a dependency on something with a peerDependency on react@16.8.0, while another has a dependency with a peerDep on react@>=16.13.0. You'd end up with this weird situation:

project
+-- packages/  (all have peerDependencies:{react:"^16"}
|   +-- foo
|   |   +-- node_modules
|   |       +-- oldthing
|   |       +-- react@16.8.0
|   +-- bar
|   |   +-- node_modules
|   |       +-- newthing
|   |       +-- react@16.13.0
|   +-- others...
|       +-- node_modules (no react here, but maybe other stuff)
+-- node_modules
    +-- foo@ -> ../packages/foo
    +-- bar@ -> ../packages/bar
    +-- others... -> ../packages/others...
    +-- react@16.12.0

So, even though your workspace loads just fine, your set of packages do not all work together properly, because they have conflicting peer deps that are being obscured by the fact that we treat them all like independent universes.

There would be no way to know that installing oldthing in foo and newthing in bar is what caused the conflict to occur downstream. By forcing workspaces (and any nested link targets) to resolve their peerDeps in their filesystem parent (ie, the root project), we surface these conflicts much earlier in the process.

While workspaces are independent projects, they're assumed to be related projects. For example, we do our best to ensure that if foo depends on bar, it gets the linked version, rather than installing a new copy in its own node_modules folder. That's sort of the point of using them. If they appear to work together in the workspaces project, but can't actually all be installed together, then that might be very surprising.

[jansiegel]

Ah, that makes sense, I think I get the idea now.

The thing is, it really depends on the nature of the packages - in my case they're closely related, but "independent" at the same time, as they're angular, react and vue components based on a library that's kept at the root (which I also declared as a package/workspace). I can try to keep them sort of in sync, but I can't be sure if some of the deeper framework-based dependencies won't cause any conflicts at some point in the future.