v2.0.0
Release tag: v2.0.0
Upgrade Notes
No steps are required to upgrade to this release as a user of ClearlyDefined. There are no changes to the API.
The change of most interest is the addition of support for scancode LicenseRefs and the update to scancode v32.1.0.
All major changes are related to changes in newly created definitions based on changes in the crawler data output by license tool updates and license extraction process.
Note: Requests for definitions will result in a recomputation of the definition to include the changes described in this release. Definition requests do not initiate a harvest request when a definition already exists. In that case, the caller must make a harvest request through the service API in order to update raw tool results from which the definition will be constructed. Note as well that harvesting takes significant time. There will be a delay from the time the harvest request is made before the results will be reflected in a definition request.
What’s changed
Major Changes
- Update definition schema to 1.7.0 by @qtomlinson in #1144
Forces definitions with older schema to be recalculated the next time they are requested. This is required for the data changes including the addition of support for scancode LicenseRefs.
Support scancode v32.1.0 and non-SPDX licenses using LicenseRef
ScanCode major versions 31 and 32 introduced pretty drastic changes to its output format which required significant changes to our summarizing logic. Multiple PRs brought in the support for LicenseRefs identified by ScanCode.
What this means for you?
When a license is identified as NOASSERTION
or OTHER
, it is possible that ScanCode can identify the license as something other than one of the SPDX licenses. Several possibilities each with a different solution...
- ScanCode has already identified a non-SPDX license - In this case, simply requesting the definition will initiate a recompute of the definition which will replace the current license with the ScanCode LicenseRef
- ScanCode has not identified a non-SPDX license with the previous version os ScanCode - In this case, a
/harvest
request is required to get ScanCode to run again. Ultimately, once harvesting is completed, the definition will be re-generated. If a new LicenseRef was identified, it will be part of the re-computed definition. - ScanCode cannot identify a license - You won't know this in advance meaning the step to take is to send a
/harvest
request. The result of the re-computed definition will be the license is unchanged. Sending additional/harvest
and/definitions
requests will not change the results or the definition.
PRs for LicenseRef support
- Add new summarizer for recent ScanCode versions (e.g. v32.1.0) by @lumaxis in #1056
- Update to SPDX v0.1.9 to support LicenseRef mapping in scanner and parser by @qtomlinson in #1205
- Update license normalization process to support LicenseRef by @lumaxis in #1148
- maintain precedence when joining Scancode license expressions by @lumaxis in #1087
When joining license expressions with AND: 'MIT OR Apache-2.0', 'GPL', precedence should be preserved in the result. The joined expression was incorrectly constructed as GPL AND MIT OR Apache-2.0
. It is now correctly constructed with precedence as GPL AND (MIT OR Apache-2.0)
.
Additional data related changes
- Update to SPDX v0.1.8 to avoid adding unnecessary brackets in stringify by @qtomlinson in #1203
This update brings in SPDX PR clearlydefined/spdx#30
The expressions "LGPL-2.1-only OR MIT OR BSD-3-Clause" and "LGPL-2.1-only OR BSD-3-Clause AND MIT" are valid and simplified forms of SPDX expressions. Refer to the SPDX specification for more information (https://spdx.github.io/spdx-spec/v2-draft/SPDX-license-expressions/#d4-composite-license-expressions)
Minor Changes
Bug Fixes and Patches
Development related
- Add source location in definitions for sourcearchive packages by @qtomlinson in #1108
- Fixed origins api for pypi components throwing 500 error when invalid group id is provided by @yashkohli88 in #1172
- Fixed origins api for maven components throwing 500 error when invalid group id is provided by @yashkohli88 in #1176)
DevOps
- Add separate CI job for testing Docker build by @lumaxis in #1126
- docs: add SECURITY.md by @nickvidal in https://github.com/clearlydefined/service/pull//1143
- update deploy scripts to v2.0.0 by @ in https://github.com/clearlydefined/service/pull//1154
- Update Deployment Pipeline to v3.0.0, App Version and Build Sha available on GET / and logs by @elrayle and @ljones140 in #1199
- Reduce logging which is our biggest Azure cost by @ljones140 in #1201
- Update .npmrc so that @clearlydefined packages come from GitHub package registry by @ljones140 in #1204
- auto-start test workflow when PR into prod by @elrayle in #1208
- Update deployment pipelines 3.1.0 by @ljones140 in #1209
- Update deployment pipelines 3.1.1 by @ljones140 in #1210
Dependencies
- Bump express from 4.17.3 to 4.19.2 by @dependabot in #1080
- Bump clearlydefined/operations from 1.0.0 to 1.1.0 by @dependabot in #1123
- Bump github/codeql-action from 3.24.6 to 3.25.8 by @dependabot #1131
- Bump braces and patch-package by by @dependabot in #1136
- Updated deprecated dependency request-promise-native by @yashkohli88 in #1163
Full Changelog: v1.3.1...v2.0.0