Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update object_store release documentation #6565

Merged
merged 8 commits into from
Oct 16, 2024
Merged

Update object_store release documentation #6565

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
fecc682
Update object_store release documentation
alamb Oct 15, 2024
3d18a60
Update object store
alamb Oct 15, 2024
293a9af
prettier
alamb Oct 15, 2024
e5c167d
tweak
alamb Oct 15, 2024
28abde2
Update object_store/dev/release/README.md
alamb Oct 15, 2024
cd0c60e
update instructions
alamb Oct 15, 2024
f95ca17
fix
alamb Oct 15, 2024
7fe2bf7
Update object_store/dev/release/README.md
alamb Oct 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 13 additions & 44 deletions dev/release/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,29 +21,24 @@

## Overview

This file documents the release process for:

1. The "Rust Arrow Crates": `arrow`, `arrow-flight`, `parquet`, and `parquet-derive`.
2. The `object_store` crate.
This file documents the release process for the "Rust Arrow Crates": `arrow`, `arrow-flight`, `parquet`, and `parquet-derive`.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed all references to object_store from this file


### The Rust Arrow Crates

The Rust Arrow Crates are interconnected (e.g. `parquet` has an optional dependency on `arrow`) so we increment and release all of them together. We try to release a new version of "Rust Arrow Crates" every two weeks. This cadence balances getting new features into the community without overwhelming downstream projects with too frequent changes or overly burdening maintainers.

If any code has been merged to master that has a breaking API change, as defined in [Rust RFC 1105](https://github.com/rust-lang/rfcs/blob/master/text/1105-api-evolution.md), the major version number incremented changed (e.g. `9.0.2` to `9.0.2`). Otherwise the new minor version incremented (e.g. `9.0.2` to `7.1.0`).

### `object_store` crate
The Rust Arrow Crates are interconnected (e.g. `parquet` has an optional dependency on `arrow`) so we increment and release all of them together.

At the time of writing, we release a new version of `object_store` on demand rather than on a regular schedule.
If any code has been merged to master that has a breaking API change, as defined
in [Rust RFC 1105] he major version number is incremented (e.g. `9.0.2` to `10.0.2`).
Otherwise the new minor version incremented (e.g. `9.0.2` to `9.1.0`).

As we are still in an early phase, we use the 0.x version scheme. If any code has been merged to master that has a breaking API change, as defined in [Rust RFC 1105](https://github.com/rust-lang/rfcs/blob/master/text/1105-api-evolution.md), the minor version number incremented changed (e.g. `0.3.0` to `0.4.0`). Otherwise the patch version is incremented (e.g. `0.3.0` to `0.3.1`).
[rust rfc 1105]: https://github.com/rust-lang/rfcs/blob/master/text/1105-api-evolution.md

# Release Mechanics

## Process Overview

As part of the Apache governance model, official releases consist of
signed source tarballs approved by the PMC.
signed source tarballs approved by the Arrow PMC.

We then use the code in the approved source tarball to release to
crates.io, the Rust ecosystem's package manager.
Expand All @@ -60,9 +55,7 @@ labels associated with them.

Now prepare a PR to update `CHANGELOG.md` and versions on `master` to reflect the planned release.

For the Rust Arrow crates, do this in the root of this repository. For example [#2323](https://github.com/apache/arrow-rs/pull/2323)

For `object_store` the same process is done in the `object_store` directory. Examples TBD
Do this in the root of this repository. For example [#2323](https://github.com/apache/arrow-rs/pull/2323)

```bash
git checkout master
Expand Down Expand Up @@ -90,17 +83,13 @@ python dev/release/label_issues.py
# review change log / edit issues and labels if needed, rerun
git commit -a -m 'Create changelog'


# Manually edit ./dev/release/update_change_log.sh to reflect the release version
# Create the changelog
CHANGELOG_GITHUB_TOKEN=<TOKEN> ./dev/release/update_change_log.sh
# Review change log / edit issues and labels if needed, rerun
git commit -a -m 'Create changelog'

git push

# File the release PR
export BRANCH=<RELEASE_BRANCH> && export GITHUB_USERNAME=<USERNAME> && export GITHUB_TOKEN=<TOKEN> && ./file_release_pr.sh
```

Note that when reviewing the change log, rather than editing the
Expand All @@ -120,9 +109,7 @@ distribution servers.

While the official release artifact is a signed tarball, we also tag the commit it was created for convenience and code archaeology.

For a Rust Arrow Crates release, use a string such as `4.0.1` as the `<version>`.

For `object_store` releases, use a string such as `object_store_0.4.0` as the `<version>`.
Use a string such as `43.0.0` as the `<version>`.

Create and push the tag thusly:

Expand All @@ -147,12 +134,6 @@ Rust Arrow Crates:
./dev/release/create-tarball.sh 4.1.0 2
```

`object_store`:

```shell
./object_store/dev/release/create-tarball.sh 4.1.0 2
```

The `create-tarball.sh` script

1. creates and uploads a release candidate tarball to the [arrow
Expand Down Expand Up @@ -198,7 +179,7 @@ For the release to become "official" it needs at least three Apache Arrow PMC me

## Verifying release candidates

The `dev/release/verify-release-candidate.sh` or `object_store/dev/release/verify-release-candidate.sh` are scripts in this repository that can assist in the verification process. Run it like:
The `dev/release/verify-release-candidate.sh` script in this repository can assist in the verification process. Run it like:

```
./dev/release/verify-release-candidate.sh 4.1.0 2
Expand All @@ -218,19 +199,13 @@ Rust Arrow Crates:
./dev/release/release-tarball.sh 4.1.0 2
```

`object_store`

```shell
./object_store/dev/release/release-tarball.sh 4.1.0 2
```

Congratulations! The release is now official!

### Publish on Crates.io

Only approved releases of the tarball should be published to
crates.io, in order to conform to Apache Software Foundation
governance standards.
It is important that only approved releases of the tarball should be published
to crates.io, in order to conform to Apache Software Foundation governance
standards.

An Arrow committer can publish this crate after an official project release has
been made to crates.io using the following instructions.
Expand Down Expand Up @@ -269,9 +244,3 @@ Rust Arrow Crates:
(cd parquet_derive && cargo publish)
(cd arrow-integration-test && cargo publish)
```

`object_store`

```shell
cargo publish
```
40 changes: 0 additions & 40 deletions dev/release/file_release_pr.sh
View file
Open in desktop

This file was deleted.

185 changes: 184 additions & 1 deletion object_store/dev/release/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,187 @@
under the License.
-->

See instructions in [`/dev/release/README.md`](../../../dev/release/README.md)

# Release Process
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved the relevant content for object store here -- it is somewhat redundant with arrow-rs but I think that is ok as it is much clearer (and will be required if/when we move to a new repo)


## Overview

This file documents the release process for the `object_store` crate.

At the time of writing, we release a new version of `object_store` on demand rather than on a regular schedule.

As we are still in an early phase, we use the 0.x version scheme. If any code has
been merged to master that has a breaking API change, as defined in [Rust RFC 1105]
the minor version number is incremented changed (e.g. `0.3.0` to `0.4.0`).
Otherwise the patch version is incremented (e.g. `0.3.0` to `0.3.1`).

[Rust RFC 1105]: https://github.com/rust-lang/rfcs/blob/master/text/1105-api-evolution.md
# Release Mechanics

## Process Overview

As part of the Apache governance model, official releases consist of
signed source tarballs approved by the PMC.

We then use the code in the approved source tarball to release to
crates.io, the Rust ecosystem's package manager.

We create a `CHANGELOG.md` so our users know what has been changed between releases.

The CHANGELOG is created automatically using
[update_change_log.sh](https://github.com/apache/arrow-rs/blob/master/object_store/dev/release/update_change_log.sh)

This script creates a changelog using github issues and the
labels associated with them.

## Prepare CHANGELOG and version:

Now prepare a PR to update `CHANGELOG.md` and versions on `master` to reflect the planned release.

For `object_store` this process is done in the `object_store` directory. See [#6227] for an example

[#6227]: https://github.com/apache/arrow-rs/pull/6227

```bash
git checkout master
git pull
git checkout -b <RELEASE_BRANCH>

# Update versions. Make sure to run it before the next step since we do not want CHANGELOG-old.md affected.
sed -i '' -e 's/14.0.0/39.0.0/g' `find . -name 'Cargo.toml' -or -name '*.md' | grep -v CHANGELOG.md`
git commit -a -m 'Update version'

# ensure your github token is available
export ARROW_GITHUB_API_TOKEN=<TOKEN>

# manually edit ./dev/release/update_change_log.sh to reflect the release version
alamb marked this conversation as resolved.
Show resolved Hide resolved
# create the changelog
./dev/release/update_change_log.sh

# run automated script to copy labels to issues based on referenced PRs
# (NOTE 1: this must be done by a committer / other who has
# write access to the repository)
#
# NOTE 2: this must be done after creating the initial CHANGELOG file
python dev/release/label_issues.py

# review change log / edit issues and labels if needed, rerun
git commit -a -m 'Create changelog'


# Manually edit ./dev/release/update_change_log.sh to reflect the release version
# Create the changelog
CHANGELOG_GITHUB_TOKEN=<TOKEN> ./dev/release/update_change_log.sh
# Review change log / edit issues and labels if needed, rerun
git commit -a -m 'Create changelog'

git push
```

Note that when reviewing the change log, rather than editing the
`CHANGELOG.md`, it is preferred to update the issues and their labels
(e.g. add `invalid` label to exclude them from release notes)

Merge this PR to `master` prior to the next step.

## Prepare release candidate tarball

After you have merged the updates to the `CHANGELOG` and version,
create a release candidate using the following steps. Note you need to
be a committer to run these scripts as they upload to the apache `svn`
distribution servers.

### Create git tag for the release:

While the official release artifact is a signed tarball, we also tag the commit it was created for convenience and code archaeology.

For `object_store` releases, use a string such as `object_store_0.4.0` as the `<version>`.

Create and push the tag thusly:

```shell
git fetch apache
git tag <version> apache/master
# push tag to apache
git push apache <version>
```

### Pick an Release Candidate (RC) number

Pick numbers in sequential order, with `1` for `rc1`, `2` for `rc2`, etc.

### Create, sign, and upload tarball

Run `create-tarball.sh` with the `<version>` tag and `<rc>` and you found in previous steps.

```shell
./object_store/dev/release/create-tarball.sh 0.11.1 1
```

The `create-tarball.sh` script

1. creates and uploads a release candidate tarball to the [arrow
dev](https://dist.apache.org/repos/dist/dev/arrow) location on the
apache distribution svn server

2. provide you an email template to
send to dev@arrow.apache.org for release voting.

### Vote on Release Candidate tarball

Send an email, based on the output from the script to dev@arrow.apache.org. The email should look like

```

alamb marked this conversation as resolved.
Show resolved Hide resolved
```

For the release to become "official" it needs at least three Apache Arrow PMC members to vote +1 on it.

## Verifying release candidates

The `object_store/dev/release/verify-release-candidate.sh` script can assist in the verification process. Run it like:

```
./object_store/dev/release/verify-release-candidate.sh 0.11.0 1
```

#### If the release is not approved

If the release is not approved, fix whatever the problem is and try again with the next RC number

### If the release is approved,

Move tarball to the release location in SVN, e.g. https://dist.apache.org/repos/dist/release/arrow/arrow-4.1.0/, using the `release-tarball.sh` script:


```shell
./object_store/dev/release/release-tarball.sh 4.1.0 2
```

Congratulations! The release is now official!

### Publish on Crates.io

Only approved releases of the tarball should be published to
crates.io, in order to conform to Apache Software Foundation
governance standards.

An Arrow committer can publish this crate after an official project release has
been made to crates.io using the following instructions.

Follow [these
instructions](https://doc.rust-lang.org/cargo/reference/publishing.html) to
create an account and login to crates.io before asking to be added as an owner
of the [arrow crate](https://crates.io/crates/arrow).

Download and unpack the official release tarball

Verify that the Cargo.toml in the tarball contains the correct version
(e.g. `version = "0.11.0"`) and then publish the crate with the
following commands


```shell
cargo publish
```

Loading