[RRFC] New updatePolicy metadata section

[castarco]

Introduction

I acknowledge that this text might be not complete or formal enough. I posted it here so I can gather early feedback before putting in all the extra effort needed to have a super-serious RFC document. The blog post where I originally wrote about the idea is slightly more complete, but its structure is really not good for an RFC document.

I acknowledge that the proposed change is quite complex, but (in my opinion) it does not really introduce new complexity for users (although most certainly does for NPM maintainers 🤷🏻 ), but just surfaces it to a place where users can manage it.

Motivation ("The Why")

(Copied from the original article where I posted the idea in the first place)

It’s crucial to recognise that not all developers adhere to semantic versioning, leading to a variety of challenges:

1. Assumed adherence: Some package authors do not clearly state if their package does not follow semantic versioning. As a result, users may assume adherence due to its widespread use, only to encounter unexpected breaking changes.
2. Missed communication: Even when package authors explicitly state their non-adherence to semantic versioning, users may overlook this information if they haven’t read the documentation thoroughly. It’s easy to blame users for not reading carefully, but that doesn’t fix the issue. Moreover, the time and effort to manually check every dependency can be substantial.
3. Accidental non-compliance: Sometimes, package authors who generally follow semantic versioning may unintentionally release a breaking change in a patch or minor version.
4. Issues with transitive dependencies: Even if we meticulously check all our direct dependencies, we cannot control how careful the authors of those dependencies are. They might inadvertently introduce breaking changes via transitive dependencies.

Example

What follows is not a "complete example", but the "shape" it would have a new optional section called updatePolicy:

{
  "updatePolicy": {
    /**
     * Defines the versioning strategy followed by this package. Some
     * possible values are:
     * - "none":   The package does not offer any guarantees regarding
     *             breaking changes.
     * - "patch":  The package only offers non-breaking guarantees for patch
     *             versions.
     * - "semver": The package follows semantic versioning.
     */
    "versioningStrategy": "semver",
    "dependencies": {
      /**
        * Defines the assumed versioning strategy for the dependencies that
        * do not specify their own versioning strategy (whether they are
        * direct or transitive dependencies). It can take the same values
        * as the "versioningStrategy" property.
        * It only applies to dependencies with a version range defined with
        * a caret (^), although it can be passed down to transitive deps of
        * dependencies with other version ranges (~, *, or pinned).
        */
      "defaultVersioningStrategy": "patch",
      /**
        * This property can only tighten the default versioning strategy of
        * transitive dependencies when set to true, by setting the
        * strictest between the default at the current level, and the
        * default at deeper levels.
        * It only applies to dependencies with a version range defined with
        * a caret (^), although it can be passed down to transitive deps of
        * dependencies with other version ranges (~, *, or pinned).
        */
      "overrideTransitiveDefaults": true,
      /**
       * Allows us to "unpin" transitive dependencies that are labeled as
       * "patch" or "semver" for their "versioningStrategy" field. This
       * can help to update transitive dependencies for unmaintained
       * packages that are too conservative and pin exact versions of
       * their dependencies instead of using flexible version ranges (with
       * ~ or ^).
       * Allowed values are:
       * - false:    Do not unpin any transitive dependencies.
       * - "patch":  Unpin transitive dependencies that have
       *             "versioningStrategy" set to "patch" or "semver", and
       *             only allow patch updates.
       * - "semver": Unpin transitive dependencies that have
       *             "versioningStrategy" set to "patch" or "semver", and
       *             only allow updates that follow the constrains
       *             specified by their versioning strategy.
       *             This option can also affect not-pinned dependencies
       *             using the "~" opeator for their version range.
       */
      "unpinTrustedSemver": false,
      /**
       * Allows us to set "versioningStrategy" values for specific transitive
       * dependencies. Although we can use ~ for direct dependencies when we
       * want updates only at the patch level, this is not possible for
       * transitive dependencies (unless our dependencies apply a similar
       * version range to their own transitive dependencies, but we don't have
       * control over that).
       * The applied versioning strategy will be the strictest between the one
       * specified by the package itself, and all the overrides defined along
       * that specific "path" of the dependency tree.
       * Note that, when no other overrides are specified and no versioning
       * strategy is defined by the package itself, the override could be less
       * strict than what's defined by the "defaultVersioningStrategy" field.
       */
      "overrides": {
        "typescript": "patch",
      }
    },
  }
}

updatePolicy.dependencies.unpinTrustedSemver

This option can help us to update transitive dependencies for unmaintained packages, reducing the risk of having unpatched vulnerabilities in our projects.

Here you can see an example of how this field would work on transitive dependencies (here versioningStrategy is the computed value after applying all overrides):

versionRange	versioningStrategy	unpin	nextVersions	updatedTo
1.0.0	none	false	1.0.1, 1.1.0, 2.0.0	1.0.0
1.0.0	none	true	1.0.1, 1.1.0, 2.0.0	1.0.0
1.0.0	patch	false	1.0.1, 1.1.0, 2.0.0	1.0.0
1.0.0	patch	true	1.0.1, 1.1.0, 2.0.0	1.0.1
1.0.0	semver	false	1.0.1, 1.1.0, 2.0.0	1.0.0
1.0.0	semver	true	1.0.1, 1.1.0, 2.0.0	1.1.0
~1.0.0	semver	false	1.0.1, 1.1.0, 2.0.0	1.0.1
~1.0.0	semver	true	1.0.1, 1.1.0, 2.0.0	1.1.0

How

Current Behaviour

It is difficult to characterise a "current behaviour", as this proposal focuses on the complex interaction between many small factors, but I'll list some (but not all) of the problems it tries to mitigate.

• Many packages do not follow semantic versioning, and even worse, among them, many do not explicitly document that fact.
• We depend on the maintainers of our dependencies to honour semantic versioning, and to verify their own dependencies doing the same (this does not always happens, in fact, it happens much less than desirable).
• Unmaintained dependencies that pin their dependencies (or our transitive dependencies) can force us to be stuck with old versions of libraries containing unpatched vulnerabilities.
• When new breaking changes are introduced by accident (not respecting semantic versioning), it might take a long time before the problem is detected upstream/

Desired Behaviour

• Package managers should be able to take advantage of metadata (when available) to know whether a package follows semantic versioning or not, to decide what's the best update strategy.
• Project maintainers should be able to have better control over the update policy of their dependencies (direct and transitive), to ensure better stability and reliability (and decrease their dependency on the goodwill of upstream dependency maintainers).

References

• Beyond Semantic Versioning: updatePolicy: The first article where I outline the idea.
• [RRFC] Ability to set minimum maturity (in days) of versions to upgrade #646 : Another RFC that could be integrated with this one (although it has a different purpose, the nature of the rules it describes could be encoded inside the updatePolicy section.