feat: Über Migration (and Boxo rename)

[Jorropo]

feat: Über Migration (and Boxo rename)

Include rename from:
github.com/ipfs/go-libipfs => github.com/ipfs/boxo

This migration was reverted:
./blocks => github.com/ipfs/go-block-format

Migrated repos:

• github.com/ipfs/interface-go-ipfs-core => ./coreiface
• github.com/ipfs/go-pinning-service-http-client => ./pinning/remote/client
• github.com/ipfs/go-path => ./path
• github.com/ipfs/go-namesys => ./namesys
• github.com/ipfs/go-mfs => ./mfs
• github.com/ipfs/go-ipfs-provider => ./provider
• github.com/ipfs/go-ipfs-pinner => ./pinning/pinner
• github.com/ipfs/go-ipfs-keystore => ./keystore
• github.com/ipfs/go-filestore => ./filestore
• github.com/ipfs/go-ipns => ./ipns
• github.com/ipfs/go-blockservice => ./blockservice
• github.com/ipfs/go-ipfs-chunker => ./chunker
• github.com/ipfs/go-fetcher => ./fetcher
• github.com/ipfs/go-ipfs-blockstore => ./blockstore
• github.com/ipfs/go-ipfs-posinfo => ./filestore/posinfo
• github.com/ipfs/go-ipfs-util => ./util
• github.com/ipfs/go-ipfs-ds-help => ./datastore/dshelp
• github.com/ipfs/go-verifcid => ./verifcid
• github.com/ipfs/go-ipfs-exchange-offline => ./exchange/offline
• github.com/ipfs/go-ipfs-routing => ./routing
• github.com/ipfs/go-ipfs-exchange-interface => ./exchange
• github.com/ipfs/go-unixfs => ./ipld/unixfs
• github.com/ipfs/go-merkledag => ./ipld/merkledag
• github.com/ipld/go-car => ./ipld/car

Fixes #215
Updates #202

[codecov]

Codecov Report

Merging #220 (038bdd2) into main (b3c7bf1) will increase coverage by 18.08%.
The diff coverage is 28.96%.

@@             Coverage Diff             @@
##             main     #220       +/-   ##
===========================================
+ Coverage   30.09%   48.18%   +18.08%     
===========================================
  Files         101      267      +166     
  Lines       11319    32583    +21264     
===========================================
+ Hits         3407    15699    +12292     
- Misses       7545    15224     +7679     
- Partials      367     1660     +1293     

Impacted Files	Coverage Δ
bitswap/bitswap.go	37.20% <ø> (+37.20%)	⬆️
bitswap/client/client.go	82.21% <ø> (+82.21%)	⬆️
bitswap/client/internal/getter/getter.go	64.10% <ø> (+64.10%)	⬆️
...tswap/client/internal/messagequeue/messagequeue.go	65.74% <ø> (+65.74%)	⬆️
...wap/client/internal/notifications/notifications.go	72.15% <ø> (+72.15%)	⬆️
bitswap/client/internal/session/session.go	82.18% <ø> (+82.18%)	⬆️
...tswap/client/internal/session/sessionwantsender.go	80.94% <ø> (+80.94%)	⬆️
...l/sessioninterestmanager/sessioninterestmanager.go	82.01% <ø> (+82.01%)	⬆️
...p/client/internal/sessionmanager/sessionmanager.go	90.62% <ø> (+90.62%)	⬆️
bitswap/client/wantlist/wantlist.go	69.86% <ø> (+69.86%)	⬆️
... and 124 more

... and 89 files with indirect coverage changes

[guseggert]

This needs to be tested on Kubo and Lotus before merging right?

[Jorropo]

@guseggert this is being tested in Kubo.
I've looked at lotus it should be fine after the fact given we can build go-ipld-prime already without even trying (it will require some manual inteligence to put the cut point).

[BigLep]

Per verbal maintainer conversation on 2023-03-23, this will be vetted on Lotus as well before merge.

[guseggert]

Let me know when this is ready for another review. We need to see what it looks like for Kubo and Lotus to upgrade, so bubbling latest changes up to the Kubo PR, and opening a PR w/ Lotus (or in a forked repo) seems like a prereq.

[Jorropo]

This works in Lotus and Kubo (CI failures are flaky tests and docs gen), builds and non broken tests:

• chore: boxo: migrate from go-libipfs to boxo filecoin-project/lotus#10562
• chore: migrate go-libipfs to boxo kubo#9746

Imo this is RFM @guseggert

[guseggert]

Removed cmd/car because it was forcing a newer version of urfave/cli/v2 which was causing Lotus tests to fail (just some diff tests on CLI output, because that newer version included --help in each command output...not a big deal but nice to avoid if we can). We can add it back later if we want and either downgrade the lib in boxo or make it a separate module so that the lib is not included in the main boxo module.

[guseggert]

I spot-checked w/ ipfs-lite (required for ipfs-cluster) and tests fail with a runtime error in go-libp2p-kad-dht due to go-libp2p-kad-dht not being updated yet, the error is from this line: https://github.com/libp2p/go-libp2p-kad-dht/blob/3f53669165720238fdd7a5bb71d60966c0785ef7/internal/config/config.go#L158

So we should probably merge libp2p/go-libp2p-kad-dht#832 first and release it. I think it's already covered by the migration tool because the issue occurs when go-ipns is in your dependency graph which is already covered by the check-dependencies command of the migration tool:

$ ~/git/boxo/cmd/migrate/migrate check-dependencies
You still have dependencies on repos which have migrated to go-libipfs.
You should consider not having these dependencies to avoid multiple versions of the same code.
You can use 'go mod why' or 'go mod graph' to find the reason for these dependencies.

Dependent module versions:

github.com/ipfs/go-unixfs v0.4.5
github.com/ipfs/go-ipns v0.3.0
github.com/ipfs/go-ipfs-chunker v0.0.5
github.com/ipfs/go-ipfs-blockstore v1.3.0
github.com/ipfs/go-ipfs-util v0.0.2
github.com/ipfs/go-ipfs-ds-help v1.1.0

go mod why answers this question directly:

$ go mod why github.com/ipfs/go-ipns
# github.com/ipfs/go-ipns
github.com/hsanjuan/ipfs-lite
github.com/libp2p/go-libp2p-kad-dht
github.com/libp2p/go-libp2p-kad-dht/internal/config
github.com/ipfs/go-ipns

[guseggert]

Okay I worked through ipfs-cluster and it builds and CI passes for it too, we're going to need to update github.com/libp2p/go-libp2p-kad-dht and github.com/ipfs/go-ds-crdt in order to unblock ipfs-cluster, so we'll do that after the boxo release.

[hsanjuan]

Thank you @guseggert , I really appreciate that you worried about ipfs-cluster in advance ❤️ Send me the PRs, or just let me know when we can upgrade.

[Kubuxu]

@Jorropo Kudos to you for preserving the history of migrated repos! Thank you!

[Jorropo]

@Kubuxu ❤️ you can thx @guseggert too.
My first plan was to do a dumb-ish git merge --allow-unrelated-histories and gus made a git-filter-branch based python tool which add links and do the tree filtering.