Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge v1.19 into master #1580

Merged
merged 2 commits into from
Jun 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,6 @@ coverage*

# drivers-evergreen-tools secrets handling
secrets-export.sh

# temporary purls file
/purls.txt
115 changes: 61 additions & 54 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,63 +182,72 @@ a 1.23.1 tag also existed at the time. The bump to libmongoc 1.23.1 was left to
another PHPC ticket in the 1.15.0 milestone, which actually depended on the
libmongoc changes therein.

### Updating libmongoc and libbson
### Updating bundled libraries

#### Update libmongoc submodule
The following steps are the same for libmongoc and libmongocrypt. When updating
libmongocrypt, follow the same steps but replace `libmongoc` with
`libmongocrypt`, retaining the same capitalization. The following examples
always refer to libmongoc.

```
$ cd src/libmongoc
$ git fetch
$ git checkout 1.20.0
#### Update submodule

```shell
cd src/libmongoc
git fetch
git checkout 1.20.0
```

During development, it may be necessary to temporarily point the libmongoc
submodule to a commit on the developer's fork of libmongoc. For instance, the
developer may be working on a PHP driver feature that depends on an unmerged
pull request to libmongoc. In this case, `git remote add` can be used to add
the fork before fetching and checking out the target commit. Additionally, the
submodule path in
[`.gitmodules`](https://github.com/mongodb/mongo-php-driver/blob/master/.gitmodules)
must also be updated to refer to the fork.
During development, it may be necessary to temporarily point the submodule to a
commit on the developer's fork. For instance, the developer may be working on a
PHP driver feature that depends on unmerged or unreleased changes. In this case,
the submodule path can be updated using the `git submodules set-url` command can
be used to change the URL, and `git submodules set-branch` can be used to point
the submodule to a development branch:

#### Ensure libmongoc version information is correct
```shell
git submodules set-url src/libmongoc https://github.com/<owner>/<repo>.git
git submodules set-branch -b <branch> src/libmongoc
```

The build process for Autotools and Windows rely on
`src/LIBMONGOC_VERSION_CURRENT` to infer version information for libmongoc and
libbson. This file can be regenerated using the following Makefile target:
#### Ensure version information is correct

```
$ make libmongoc-version-current
Various build processes and tools rely on the version files to infer version
information. This file can be regenerated using Makefile targets:

```shell
make libmongoc-version-current
```

Alternatively, the `build/calc_release_version.py` script in libmongoc can be
executed directly.
Alternatively, the `build/calc_release_version.py` script in the submodule can
be executed directly.

Note: If the libmongoc submodule points to a non-release, non-master branch, the
script may fail to correctly detect the version. This issue is being tracked in
[CDRIVER-3315](https://jira.mongodb.org/browse/CDRIVER-3315) and can be safely
ignored since this should only happen during development (any PHP driver release
should point to a tagged libmongoc release).
Note: If the submodule points to a non-release, non-master branch, the script
may fail to correctly detect the version. This issue is being tracked in
[CDRIVER-3315](https://jira.mongodb.org/browse/CDRIVER-3315) and can be safely ignored since this should only happen
during development (any PHP driver release should point to a tagged submodule
version).

#### Update sources in build configurations

The Autotools and Windows build configurations (`config.m4` and `config.w32`,
respectively) define several variables (e.g. `PHP_MONGODB_MONGOC_SOURCES`) that
collectively enumerate all of the the sources within the libmongoc submodule to
include in a bundled build.
collectively enumerate all of the sources within the submodules to include in a
bundled build.

These variables should each have a shell command in a preceding comment, which
should be run to regenerate that particular list of source files. Each command
may be run manually or `scripts/update-submodule-sources.php` may be used to
update all variables. In the event that either libmongoc or libbson introduce a
new source directory, that will need to be manually added (follow prior art).
update all variables. In the event that a new source directory is introduced,
this directory will need to be manually added following prior art.

#### Update package dependencies

The Autotools configuration additionally includes some `pkg-config` commands for
using libmongoc and libbson as system libraries (in lieu of a bundled build).
When bumping the libmongoc version, be sure to update the version check _and_
error message in the `pkg-config` blocks for both libmongoc and libbson.
using libmongoc, libbson, and libmongocrypt as system libraries (in lieu of a
bundled build). When bumping the bundled version, be sure to update the version
check _and_ error message in the `pkg-config` blocks for the submodule being
updated. When updating libmongoc, be sure to update both version checks for
libmongoc and libbson.

For example, the following lines might be updated for libmongoc:

Expand All @@ -250,7 +259,7 @@ if $PKG_CONFIG libmongoc-1.0 --atleast-version 1.20.0; then
AC_MSG_ERROR(system libmongoc must be upgraded to version >= 1.20.0)
```

#### Update tested versions in Evergreen configuration
#### Update tested versions in Evergreen configuration (libmongoc only)

Evergreen tests against multiple versions of libmongoc. When updating to a newer
libmongoc version, make sure to update the libmongoc build tasks in `.evergreen/config/templates/build/build-libmongoc.yml`
Expand All @@ -264,32 +273,30 @@ against two additional versions of libmongoc:

#### Update sources in PECL package generation script

If either libmongoc or libbson introduce a new source directory, that may also
If a new version of a submodule introduces a new source directory, that may also
require updating the glob patterns in the `bin/prep-release.php` script to
ensure new source files will be included in any generated PECL package.

#### Test and commit your changes
#### Update SBOM file

Verify that the upgrade was successful by ensuring that the driver can compile
using both the bundled sources and system libraries for libmongoc and libbson,
and by ensuring that the test suite passes. Once done, commit the changes to all
of the above files/paths. For example:
After updating dependencies, the SBOM file needs to be updated. There is a
script to automate this process:

```
$ git commit -m "Bump libmongoc to 1.20.0" config.m4 config.w32 src/libmongoc src/LIBMONGOC_VERSION_CURRENT
```shell
./scripts/update-sbom.sh
```

### Updating libmongocrypt
This script will generate a temporary purl file with our dependencies, then run
the internal silkbomb tool to update the SBOM. Note that you need to have docker
installed in order to run this.

To update libmongocrypt, the steps are similar to the above:
#### Test and commit your changes

```
$ cd src/libmongocrypt
$ git fetch
$ git checkout 1.3.0
$ make libmongocrypt-version-current
```
Verify that the upgrade was successful by ensuring that the driver can compile
using both the bundled sources and system libraries, and by ensuring that the
test suite passes. Once done, commit the changes to all of the above
files/paths. For example:

Package dependencies in `config.m4` must also be updated (either manually or
with `scripts/update-submodule-sources.php`), as do the sources in the PECL
generation script.
```shell
git commit -m "Bump libmongoc to 1.20.0" config.m4 config.w32 src/libmongoc src/LIBMONGOC_VERSION_CURRENT sbom.json
```
97 changes: 97 additions & 0 deletions sbom.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
{
"components": [
{
"bom-ref": "pkg:github/mongodb/libmongocrypt@1.10.0",
"externalReferences": [
{
"type": "distribution",
"url": "https://github.com/mongodb/libmongocrypt/archive/refs/tags/1.10.0.tar.gz"
},
{
"type": "website",
"url": "https://github.com/mongodb/libmongocrypt/tree/1.10.0"
}
],
"group": "mongodb",
"name": "libmongocrypt",
"purl": "pkg:github/mongodb/libmongocrypt@1.10.0",
"type": "library",
"version": "1.10.0"
},
{
"bom-ref": "pkg:github/mongodb/mongo-c-driver@1.27.2",
"externalReferences": [
{
"type": "distribution",
"url": "https://github.com/mongodb/mongo-c-driver/archive/refs/tags/1.27.2.tar.gz"
},
{
"type": "website",
"url": "https://github.com/mongodb/mongo-c-driver/tree/1.27.2"
}
],
"group": "mongodb",
"name": "mongo-c-driver",
"purl": "pkg:github/mongodb/mongo-c-driver@1.27.2",
"type": "library",
"version": "1.27.2"
}
],
"dependencies": [
{
"ref": "pkg:github/mongodb/libmongocrypt@1.10.0"
},
{
"ref": "pkg:github/mongodb/mongo-c-driver@1.27.2"
}
],
"metadata": {
"timestamp": "2024-06-06T07:13:52.679415+00:00",
"tools": [
{
"externalReferences": [
{
"type": "build-system",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/actions"
},
{
"type": "distribution",
"url": "https://pypi.org/project/cyclonedx-python-lib/"
},
{
"type": "documentation",
"url": "https://cyclonedx-python-library.readthedocs.io/"
},
{
"type": "issue-tracker",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/issues"
},
{
"type": "license",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/blob/main/LICENSE"
},
{
"type": "release-notes",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/blob/main/CHANGELOG.md"
},
{
"type": "vcs",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib"
},
{
"type": "website",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/#readme"
}
],
"name": "cyclonedx-python-lib",
"vendor": "CycloneDX",
"version": "6.4.4"
}
]
},
"serialNumber": "urn:uuid:acb30d08-ee47-4ff0-b301-d66ef1f54082",
"version": 1,
"$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json",
"bomFormat": "CycloneDX",
"specVersion": "1.5"
}
19 changes: 19 additions & 0 deletions scripts/update-sbom.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/usr/bin/env bash

SCRIPT_DIR=$(dirname ${BASH_SOURCE[0]})
ROOT_DIR=$(realpath "${SCRIPT_DIR}/../")
PURLS_FILE="${ROOT_DIR}/purls.txt"

LIBMONGOC_VERSION=$(cat ${ROOT_DIR}/src/LIBMONGOC_VERSION_CURRENT | tr -d '[:space:]')
LIBMONGOCRYPT_VERSION=$(cat ${ROOT_DIR}/src/LIBMONGOCRYPT_VERSION_CURRENT | tr -d '[:space:]')

# Generate purls file from stored versions
echo "pkg:github/mongodb/mongo-c-driver@${LIBMONGOC_VERSION}" > $PURLS_FILE
echo "pkg:github/mongodb/libmongocrypt@${LIBMONGOCRYPT_VERSION}" >> $PURLS_FILE

# Use silkbomb to update the sbom.json file
docker run --platform="linux/amd64" -it --rm -v ${ROOT_DIR}:/pwd \
artifactory.corp.mongodb.com/release-tools-container-registry-public-local/silkbomb:1.0 \
update --sbom-in /pwd/sbom.json --purls /pwd/purls.txt --sbom-out /pwd/sbom.json

rm $PURLS_FILE
Loading