object_store: Migrate from snafu to thiserror

[Turbo87]

Which issue does this PR close?

#6166, though only partially, since the PR is only targeting the object_store crate.

Rationale for this change

The rationale is explained in the linked issue :)

What changes are included in this PR?

The first commit in the PR adds a dependency on thiserror, the following commits incrementally port all error enums from snafu to thiserror, and then the final commit removes the now obsolete snafu dependency.

Note that this PR also includes #6265 to avoid merge conflicts from the bugfix.

Are there any user-facing changes?

As the linked issue mentions, some of the snafu type may leak to the outside and this PR will remove them so this should probably be treated as a breaking change, even though these types have been considered private API before.

[Turbo87]

/cc @tustvold :)

[tustvold]

HI @Turbo87, thank you for this, it looks pretty spot on. Unfortunately this ended up missing the 0.11 release, the RC was cut on the 12th August, and the next breaking release will not be for another 3 months...

I'm not really sure what the best way to proceed with this is, as we can't merge it to master for another 2 months. One option might be to keep this PR as a draft until then, but I appreciate that's likely a longer time commitment than you might have reasonably intuited when you picked this up 😅

Also tagging @Xuanwo and @alamb who may have other thoughts

[Turbo87]

the next breaking release will not be for another 3 months

no worries :)

as we can't merge it to master for another 2 months

any reason for not being able to merge before then? I would guess potential bug fixes could be branched off before the merge of this PR, but I'm not familiar with the release process here :D

[tustvold]

any reason for not being able to merge before then

Non-backport releases, including minor releases, are made from master. There have been some discussions in the past about alternative approaches, but they're not without their own trade-offs. #5907

[alamb]

I am depressed about the large review backlog in this crate. We are looking for more help from the community reviewing PRs -- see #6418 for more

[findepi]

I am depressed about the large review backlog in this crate. We are looking for more help from the community reviewing PRs -- see #6418 for more

Smaller PRs are faster to merge, because the review time is more than linear.

@Turbo87 your PR has awesome commit-by-commit structure. Would you consider moving eg first ~4 commits to a separate one?

[tustvold]

This PR is probably fine as is, it's largely mechanical, it is actually just waiting on us to start preparation for the next breaking release

[Turbo87]

Would you consider moving eg first ~4 commits to a separate one?

we can do that, but I'm not sure how valuable that would actually be since we'd then end up in a situation where we have both dependencies in the Cargo.toml file. it's probably easiest to review this commit-by-commit. since the changes are indeed pretty mechanical this should get quicker after reviewing the first few commits.

[Turbo87]

FYI I've just rebased the branch to fix the merge conflict and integrate #6266 (comment)

[Turbo87]

rebased again to fix merge conflicts caused by #6453

[jondot]

This is great! I've been looking for this to optimize builds of Loco (https://github.com/loco-rs/loco)

[Turbo87]

rebased once more and updated to thiserror v2

[crepererum]

I think enough time has been passed to cut another major release. Before we send @Turbo87 off to fix the conflicts AGAIN, @alamb do you agree that we could merge this? Since the PR is a purely mechanical translation, the size is totally fine.

[tustvold]

I think enough time has been passed to cut another major release

I think we we would want to coordinate this with the arrow major release, as otherwise everything will end up lagging. Given the arrow release has already started, this will probably need to wait until the next arrow breaking release next year.

I'm also not sure we really have enough functionality to warrant making a breaking release - see discussion on #6596

[alamb]

I think enough time has been passed to cut another major release. Before we send @Turbo87 off to fix the conflicts AGAIN, @alamb do you agree that we could merge this? Since the PR is a purely mechanical translation, the size is totally fine.

I think the as yet unresolved question is what the next object_store release will be -- breaking or non breaking:

• Discuss: Release object store 0.12.0 (breaking) or 0.11.2 (non breaking API) around Dec 15, 2024 #6596

If it will be breaking we can merge this PR, if not, we should wait

Let's figure out what the next release will be on #6596

[alamb]

tustvold beat me to it!

[Turbo87]

I have to admit that it's getting a bit frustrating that this PR has been open for 4 months and now it's looking like it gets to stay open for at least another 2 months...

Same for #6636. It sounds like the argument is that there are not enough breaking changes gathered, but I'm wondering what the threshold is since I kept maintaining these two breaking change PRs for a while now 😅

[tustvold]

Same for #6636. It sounds like the argument is that there are not enough breaking changes gathered,

I apologise that we've ended up in this position, and thank you for your patience. We've had a fair amount of push back from the community about the rate of breaking changes and so are trying hard to keep them down. Just under half of the install base still hasn't made it to 0.11 yet... - https://crates.io/crates/object_store

I've proposed that the next release after the one next week be a breaking release, and so we'd be able to get this merged to main in a week or so - #6596 (comment)