Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Über Migration (and Boxo rename) #220

Merged
merged 4,206 commits into from
Mar 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4206 commits
Select commit Hold shift + click to select a range
0e59488
add a fetcher constructor for the case where we already have a session
whyrusleeping Oct 21, 2021
a22b7e0
Merge pull request ipfs/go-fetcher#26 from ipfs/feat/fetcher-from-ses…
whyrusleeping Oct 21, 2021
b7dfe90
feat: plumb through context changes
guseggert Oct 21, 2021
793bb03
Revert "feat: plumb through context changes"
guseggert Oct 21, 2021
27eb714
creation of car from file / directory (#246)
willscott Oct 21, 2021
66b393c
feat: fix get-dag and add version=1 option
rvagg Oct 21, 2021
6ea6de4
fix!: use -version=n instead of -v1 for index command
rvagg Oct 22, 2021
b066559
feat: add context to interface (#18)
guseggert Oct 28, 2021
d041c54
feat: plumb through context changes (#28)
guseggert Oct 29, 2021
815dd33
Add a barebones readme to the car CLI (#262)
willscott Nov 1, 2021
a8d200c
support extraction of unixfs content stored in car files (#263)
willscott Nov 9, 2021
d1cf54c
feat: plumb through contexts (#42)
guseggert Nov 10, 2021
378fb21
feat: add context to interfaces (#86)
guseggert Nov 10, 2021
1e8a6de
feat: plumb through contexts (#78)
guseggert Nov 10, 2021
ae9364a
feat: plumb through context changes (#28)
guseggert Nov 10, 2021
3a39d7e
feat: plumb through context changes (#47)
guseggert Nov 10, 2021
8604dc0
feat: plumb through context changes (#18)
guseggert Nov 11, 2021
01d2a5e
feat: plumb through datastore contexts (#39)
guseggert Nov 11, 2021
8232413
Size-based unsharding (#94)
schomatis Nov 12, 2021
2614f3e
feat: add context to interfaces (#90)
guseggert Nov 12, 2021
bfa7e92
feat: plumb through datastore context changes
guseggert Oct 21, 2021
91853fb
support threshold based automatic sharding and unsharding of director…
schomatis Nov 16, 2021
8c04627
chore: add keys to composite literal fields
galargh Nov 17, 2021
f47878a
chore: use Warnf instead of Warningf
galargh Nov 17, 2021
404d259
Merge pull request ipfs/go-ipfs-provider#40 from galargh/ci/govet-and…
marten-seemann Nov 17, 2021
dcf2244
Seek to start before index generation in `ReadOnly` blockstore
masih Nov 16, 2021
c57ae79
fix: remove context from HashOnRead
guseggert Nov 19, 2021
e81cb4c
feat: plumb through context changes
guseggert Nov 19, 2021
c66b60c
feat: plumb through datastore contexts
aschmahmann Nov 22, 2021
ab62a53
Traversal-based car creation (#269)
willscott Nov 30, 2021
261de8a
feat: per-cid locking
Stebalien Sep 23, 2021
641311a
update context datastore
whyrusleeping Dec 10, 2021
61142c5
Merge pull request ipld/go-car#273 from ipld/feat/context-datastore
whyrusleeping Dec 10, 2021
6226033
Update v2 to context datastores (#275)
arajasek Dec 11, 2021
a4712fa
make specification of root cid in get-dag command optional (#281)
willscott Jan 6, 2022
18d51e4
add `car root` command (#283)
willscott Jan 13, 2022
5c3ecfa
add `car detach-index list` to list detached index contents (#287)
willscott Jan 20, 2022
b262c2b
feat: add option to create blockstore that writes a plain CARv1 (#288)
rvagg Jan 21, 2022
0c07b31
feat: --version selector for `car create` & update deps
rvagg Jan 20, 2022
bb04114
cmd/car: use a better install command in the README
Jorropo Feb 9, 2022
de93c86
chore: improve error message for invalid ipfs paths
shushen Feb 10, 2022
a3031a7
chore: update doc to match implementation for CID only paths
shushen Feb 10, 2022
1b9d107
Merge pull request ipfs/go-path#51 from shushen/35-path-parse-error-msg
aschmahmann Feb 10, 2022
2877898
Allow extracton of a raw unixfs file (#284)
willscott Feb 15, 2022
3ea7266
Resolver: convert to interface.
hsanjuan Feb 25, 2022
c0633c1
fix(test): rootless fixture should have no roots, not null roots
rvagg Mar 3, 2022
4547f9e
Use ipld.ErrNotFound for NotFound errors
hsanjuan Apr 7, 2020
8dceb79
s/log/logger
hsanjuan Apr 7, 2020
8001a94
Use ipld.ErrNotFound instead of ErrNotFound
hsanjuan Apr 7, 2020
00a68a6
s/log/logger
hsanjuan Mar 2, 2022
4ec405c
Improve NotFound error description
hsanjuan Apr 7, 2020
a5ff8e4
Improve ErrNotFound
hsanjuan Apr 7, 2020
049d13b
Use ipld.ErrNotFound
hsanjuan Apr 7, 2020
3f1d60e
refactor: follow the happy left practice in Filestore.DeleteBlock
Jorropo Mar 17, 2022
2263eca
Update tests to use ipld.IsNotFound to check for notfound errors
hsanjuan Mar 2, 2022
44ec330
sync: update CI config files (#15)
web3-bot Mar 18, 2022
e9c49a2
Resolver: unexport BasicResover
hsanjuan Mar 23, 2022
47f4c30
Merge pull request ipfs/go-path#53 from ipfs/resolver-interface
hsanjuan Mar 24, 2022
98f012c
Car command supports for `largebytes` nodes (#296)
willscott Mar 24, 2022
f9d0ad9
fix: document error (#74)
godcong Mar 25, 2022
af325c0
fix(publisher): fix garbled code output (#28)
godcong Mar 25, 2022
548e3d4
fix: use IPLD.ErrNotFound instead of string comparison in tests
Jorropo Mar 31, 2022
f249ce1
fix: this value of c is never used
galargh Apr 6, 2022
a501ffa
Merge pull request ipfs/go-merkledag#84 from ipfs/web3-bot/sync
galargh Apr 6, 2022
948dcb3
feat: add tracing (#30)
guseggert Apr 8, 2022
aca3a18
refactor(block): CIDv1 and BlockPutSettings CidPrefix (#80)
schomatis Apr 21, 2022
373d180
bump to newer blockstore err not found (#301)
willscott Apr 25, 2022
a2af87e
fix: error handling while enumerating pins
lidel Apr 15, 2022
043eed1
fix: CIDv1 error with go-libp2p 0.19 (#32)
lidel Apr 27, 2022
435dbbc
feat: add basic tracing
iand May 4, 2022
80f06e2
feat: add basic tracing
iand May 4, 2022
0bf8c2a
feat: fast-path for PutMany, falling back to Put for single block cal…
MichaelMure May 6, 2022
9b58e52
Remove unused context
iand May 12, 2022
2ad1dc9
Merge pull request ipfs/go-blockservice#91 from iand/feat-tracing
iand May 12, 2022
695f791
Merge pull request ipfs/go-path#59 from iand/feat-tracing
iand May 12, 2022
5414376
PrototypeChooser support (#305)
willscott May 31, 2022
9194b97
Make switchToSharding more efficient
hsanjuan Jun 15, 2022
33899cd
Add API to regenerate index from CARv1 or CARv2
masih Jun 29, 2022
3c6b45e
fix: don't OOM if the header size is too big
Jorropo Jun 16, 2022
6ed9f16
fix: do bound check while checking for CIDv0
Jorropo Jun 16, 2022
eed3b0b
test: add fuzzing of NewCarReader
Jorropo Jun 16, 2022
18a60b0
fix: v2 don't OOM if the header size is too big
Jorropo Jun 16, 2022
879fc55
test: v2 add fuzzing to BlockReader
Jorropo Jun 16, 2022
0583a2c
fix: v2 don't accept overflowing offsets while reading v2 headers
Jorropo Jun 16, 2022
a1f8ca2
test: v2 add fuzzing to Reader
Jorropo Jun 16, 2022
5cc8a74
fix: v2 don't allocate indexes too big
Jorropo Jun 16, 2022
f4e13fd
fix: v2 don't divide by zero in width indexes
Jorropo Jun 17, 2022
99417ba
test: v2 add fuzzing of the index
Jorropo Jun 17, 2022
9ec5f98
ci: add fuzzing on CI
Jorropo Jun 17, 2022
821a3d4
feat: Refactor indexes to put storage considerations on consumers
Jorropo Jun 17, 2022
6431951
fix index comparisons
rvagg Jun 21, 2022
58e8f1a
fix: revert to internalio.NewOffsetReadSeeker in Reader#IndexReader
rvagg Jun 21, 2022
48d19ae
fix: staticcheck catches
rvagg Jun 21, 2022
6481ccf
fix: don't use multiple-go-modules for fuzzing
rvagg Jun 21, 2022
674eb25
fix: use CidFromReader() which has overread and OOM protection
rvagg Jun 21, 2022
2eead95
feat: MaxAllowedSectionSize default to 32M
rvagg Jun 21, 2022
aaed1d1
feat: MaxAllowed{Header,Section}Size option
rvagg Jun 21, 2022
e69b001
fix: explicitly disable serialization of insertionindex
rvagg Jun 22, 2022
108f8e1
fix: tighter constraint of singleWidthIndex width, add index recommen…
rvagg Jun 23, 2022
1cca428
Fix testutil assertion logic and update index generation tests
masih Jun 30, 2022
e6180e7
Use a fix code as the multihash code for `CarIndexSorted`
masih Jun 30, 2022
0b4355f
Remove support for `ForEach` enumeration from car-index-sorted
masih Jun 30, 2022
c4e4438
feat: add Reader#Inspect() function to check basic validity of a CAR …
rvagg Jun 30, 2022
2df9663
feat: add block hash validation to Inspect()
rvagg Jul 1, 2022
94eb990
test: add fuzzing for reader#Inspect
Jorropo Jul 1, 2022
39fdfc0
Use streaming APIs to verify the hash of blocks in CAR `Inspect`
masih Jul 1, 2022
c8d5150
Use consistent CID mismatch error in `Inspect` and `BlockReader.Next`
masih Jul 2, 2022
8685c69
Benchmark `Reader.Inspect` with and without hash validation
masih Jul 2, 2022
86c35fc
Drop repeated package name from `CarStats`
masih Jul 3, 2022
9da4299
Return error when section length is invalid `varint`
masih Jul 4, 2022
8006b6f
Fix fuzz CI job
masih Jul 4, 2022
51fed4d
Revert changes to `index.Index` while keeping most of security fixes
masih Jul 4, 2022
da2c8b7
Revert changes to `insertionindex`
masih Jul 6, 2022
7503800
ci: remove the reverted FuzzIndex fuzzer
Jorropo Jul 6, 2022
bbcd79c
Empty identity CID should be indexed when options are set
masih Jul 6, 2022
5494ea8
Upgrade to the latest `go-car/v2`
masih Jul 7, 2022
9abb693
Rename HasBlock to NotifyNewBlocks, and make it accept multiple blocks
MichaelMure Jul 13, 2022
1b9178f
remove IsOnline() as it's not used anywhere
MichaelMure Jul 13, 2022
8af2916
Only read index codec during inspection
masih Jul 12, 2022
f439298
Separate `index.ReadFrom` tests
masih Jul 14, 2022
179030b
feat: add `car inspect` command to cmd pkg (#320)
rvagg Jul 15, 2022
2fac27e
write blocks retrieved from the exchange to the blockstore
MichaelMure Jul 8, 2022
931b053
Exchange don't add blocks on their own anymore
MichaelMure Jul 14, 2022
b739af1
test: remove TestHasBlockReturnsNil
Jorropo Jul 27, 2022
8ee16bb
blockservice should notify the exchange when caching blocks in GetBlo…
MichaelMure Jul 27, 2022
620a34f
fix: correctly fallback to non session exchanges with sessions.
Jorropo Jul 27, 2022
41de1a5
Merge pull request ipfs/go-blockservice#1 from Jorropo/fix-sessions
MichaelMure Jul 27, 2022
00087bb
refactor: rewrite getBlocks batch loop to be clearer
Jorropo Jul 28, 2022
8c46cf0
blockstore: allow to pass a file to write in (#323)
MichaelMure Aug 1, 2022
51ec22e
OpenReadWriteFile: add test
MichaelMure Aug 1, 2022
89bd5ed
docs: Add proper documenation to the interface.
MichaelMure Aug 5, 2022
912fc33
feat: add blake3 as a good hash
laudiacay Jun 30, 2022
0f23770
use peer.IDFromBytes instead of peer.IDFromString (#38)
marten-seemann Aug 19, 2022
1b8377b
sync: update CI config files (#105)
web3-bot Aug 23, 2022
3cb06eb
sync: update CI config files (#34)
web3-bot Aug 23, 2022
5cfe8e6
sync: update CI config files (#60)
web3-bot Aug 23, 2022
b180aa1
sync: update CI config files (#39)
web3-bot Aug 23, 2022
e5532fe
fix!: keep deserialised state stable until explicit mutation
rvagg Aug 16, 2022
16fda56
run gofmt -s
web3-bot Aug 24, 2022
c71dcd8
fix: remove use of ioutil
rvagg Aug 24, 2022
ae1afa3
bump go.mod to Go 1.18 and run go fix
web3-bot Aug 24, 2022
0301bfb
bump go.mod to Go 1.18 and run go fix
web3-bot Aug 24, 2022
354d1c8
run gofmt -s
web3-bot Aug 24, 2022
444973f
fix: remove use of ioutil
rvagg Aug 24, 2022
ea6ac8c
sync: update CI config files (#87)
web3-bot Aug 25, 2022
c5d4aa0
sync: update CI config files (#125)
web3-bot Aug 26, 2022
fe8211a
update go-libp2p to v0.22.0, release v0.2.0 (#39)
marten-seemann Aug 27, 2022
e23fe44
stop using the deprecated io/ioutil package
web3-bot Aug 29, 2022
5c8841a
stop using the deprecated io/ioutil package
web3-bot Aug 29, 2022
eea2118
stop using the deprecated io/ioutil package
web3-bot Aug 29, 2022
504dcf9
Merge pull request ipfs/go-mfs#102 from ipfs/web3-bot/sync
galargh Aug 31, 2022
c2c98d0
Merge pull request ipfs/go-filestore#60 from ipfs/web3-bot/sync
galargh Aug 31, 2022
c7bfc09
fix: import io as _io
galargh Aug 31, 2022
39cc238
Merge pull request ipfs/go-pinning-service-http-client#21 from ipfs/w…
galargh Aug 31, 2022
3bf94cd
stop using the deprecated io/ioutil package
web3-bot Sep 1, 2022
78dca4b
Merge pull request ipfs/go-ipfs-keystore#10 from ipfs/web3-bot/sync
galargh Sep 1, 2022
dbe72d0
refactor: cleanup Sprintf for Bearer token
Jorropo Aug 18, 2022
62bb938
fix: send up to nanosecond precision
Jorropo Aug 18, 2022
0599510
feat: Has() and Get() will respect StoreIdentityCIDs option
rvagg Sep 2, 2022
f02f43c
chore: require V2 signatures
lidel Sep 12, 2022
c7a4817
chore: update go-libp2p to v0.22.0
nisainan Sep 13, 2022
8169e7a
Merge pull request ipfs/go-ipfs-routing#35 from nisainan/master
willscott Sep 19, 2022
fc4155a
refactor: avoid nested code
lidel Sep 20, 2022
f748bd4
fix: MaxRecordSize of 10 KiB
lidel Sep 20, 2022
2c0a4ab
Merge pull request ipfs/go-ipns#41 from ipfs/chore/require-v2-signatures
lidel Sep 20, 2022
6866c15
feat: check that the CidBuilder hasher is usable
rvagg Sep 28, 2022
a9af67f
feat: check links on setting and sanitise on encoding
rvagg Sep 29, 2022
e072e42
fix: simplify Cid generation cache & usage
rvagg Sep 29, 2022
2121361
doc: document potential panics and how to avoid them
rvagg Sep 30, 2022
105eb5f
Fix: panic when childer is nil (#127)
ajnavarro Oct 12, 2022
8399426
add a `SkipNext` method on block reader (#338)
willscott Oct 20, 2022
528afb5
docs: Update commands list
ajnavarro Oct 21, 2022
890bb3b
Merge pull request ipld/go-car#340 from ipld/ajnavarro-patch-1
ajnavarro Oct 21, 2022
3f68091
Add a debugging form for car files. (#341)
willscott Nov 18, 2022
1b9a642
feat: improve broken cid.Builder testing for CidBuilder
rvagg Oct 31, 2022
499e8f0
feat: remove panic() from non-error methods
rvagg Nov 22, 2022
3605c16
stop using the deprecated io/ioutil package
web3-bot Dec 6, 2022
65758c3
chore: update go-lib2p, avoid depending on go-libp2p-core, bump go.mo…
marten-seemann Nov 20, 2022
9bf7907
Merge commit "fix: queue: switch from using a time based counter to a…
Jorropo Dec 9, 2022
07c0099
Merge branch 'master'
Jorropo Dec 9, 2022
02df822
fix: multihash keying in the tests
Jorropo Dec 9, 2022
5341e61
Merge pull request ipfs/go-ipfs-provider#36 from ipfs/web3-bot/sync
Jorropo Dec 9, 2022
a8baf45
chore: update go-libp2p
marten-seemann Nov 20, 2022
90ca296
chore: update go-libp2p to v0.23.4
marten-seemann Nov 20, 2022
7f24945
fix: make queue 64bits on 32bits platforms too
Jorropo Dec 9, 2022
8d41689
stop using the deprecated io/ioutil package
web3-bot Dec 6, 2022
156fc84
chore: update go-libp2p to v0.23.4, update go.mod version to 1.18
marten-seemann Nov 20, 2022
ff2eb2b
feat: add UseCumulativeSize UnixfsLs option (#95)
hacdias Dec 12, 2022
01de18f
chore: migrate files (#97)
hacdias Jan 12, 2023
bcf0498
chore: migrate files (#134)
hacdias Jan 12, 2023
e3a1b78
feat(cmd): add index create subcommand to create an external carv2 index
aschmahmann Jan 16, 2023
d9db510
Update install instructions in README.md
gammazero Jan 18, 2023
06ff64e
cleanup readme a bit to make the cli more discoverable (#353)
willscott Jan 23, 2023
bcb9190
feat: add namesys publish options (#94)
hacdias Jan 24, 2023
00661e3
feat: use PublishOptions for publishing IPNS records (#35)
hacdias Jan 24, 2023
b3ab888
feat: add RoutingAPI to CoreAPI
hacdias Nov 14, 2022
bafac68
feat: upgrade from go-block-format to go-libipfs/blocks
hacdias Jan 27, 2023
b4cd7d4
fix: update go-block-format to the version that includes the stubs
Jorropo Jan 27, 2023
8016547
Allow using WalkOption in WriteCar function
leszko Jan 27, 2023
b9894f3
Merge pull request ipld/go-car#357 from leszko/allow-using-walk-func
willscott Jan 28, 2023
f17129c
fix: use goreleaser instead
rvagg Jan 27, 2023
3706bb9
test(cmd): add index creation test
aschmahmann Jan 30, 2023
b813f4f
Merge pull request ipld/go-car#350 from ipld/feat/cmd-create-external…
willscott Feb 1, 2023
599eae3
feat(blockstore): implement a streaming read only storage
hannahhoward Jan 14, 2023
0e32290
feat: StorageCar as a Readable storage, separate from blockstore
rvagg Feb 2, 2023
d851b58
feat: add Writable functionality to StorageCar
rvagg Feb 3, 2023
a6bbce9
feat: ReadableWritable; dedupe shared code
rvagg Feb 6, 2023
cd86ec8
feat: docs for StorageCar interfaces
rvagg Feb 6, 2023
bdfb88d
fix: minor lint & windows fd test problems
rvagg Feb 6, 2023
2fdb14e
chore: add experimental note
rvagg Feb 6, 2023
1ad58ea
chore: move insertionindex into store pkg
rvagg Feb 6, 2023
38af659
fix: return errors for unsupported operations
rvagg Feb 8, 2023
050974f
fix(doc): fix storage package doc formatting
rvagg Feb 8, 2023
6199506
sync: update CI config files (#45)
web3-bot Feb 8, 2023
790ce94
sync: update CI config files (#18)
web3-bot Feb 8, 2023
5902867
stop using the deprecated io/ioutil package
web3-bot Feb 8, 2023
18b8f6f
fix: switch to crypto/rand.Read
rvagg Feb 9, 2023
48f8c69
test: basic routing interface test
hacdias Feb 8, 2023
4e262c8
fix: correctly handle degenerate hamts while reading data
Jorropo Jan 10, 2023
6c761b3
test: fix tests after hamt issues fixes
Jorropo Feb 9, 2023
54d20f0
test: use two nodes in publish
Jorropo Feb 10, 2023
6e23bc2
fix: correctly handle errors in balancedbuilder's Layout
cpucorecore Dec 14, 2022
9f11814
fix: correctly handle return errors
Jorropo Feb 10, 2023
21784f0
stop using the deprecated io/ioutil package
web3-bot Feb 8, 2023
2506711
fix: switch to crypto/rand.Read
rvagg Feb 9, 2023
e8e0049
blockstore: fast path for AllKeysChan using the index
MichaelMure Feb 14, 2023
e19b61d
feat!: add and connect missing context, remove RemovePinWithMode (#23)
MichaelMure Feb 22, 2023
cd0d3b1
feat: expose ErrInvalidPath and implement .Is function (#66)
hacdias Feb 23, 2023
fbb6fad
docs(pinning): eliminate copy-n-paste typo (#28)
smoyer64 Feb 27, 2023
41a4383
feat: add WithTrustedCar() reader option (#381)
hsanjuan Mar 6, 2023
f4e1df1
ReadWrite: add an alternative FinalizeReadOnly+Close flow
MichaelMure Feb 16, 2023
7349b74
blockstore: try to close during Finalize(), even in case of previous …
MichaelMure Mar 7, 2023
7ecc10d
fix: if we don't read the full block data, don't error on !EOF
rvagg Mar 6, 2023
0904e49
feat: extract specific path, accept stdin as streaming input
rvagg Mar 6, 2023
30034da
feat: extract accepts '-' as an output path for stdout
rvagg Mar 6, 2023
e1368b1
chore: add test cases for extract modes
rvagg Mar 7, 2023
663fd91
fix: update cmd/car/README with latest description
rvagg Mar 7, 2023
0771bee
fix: make -f optional, read from stdin if omitted
rvagg Mar 7, 2023
9127d28
fix: error when no files extracted
rvagg Mar 7, 2023
dc90d16
Allow using a NewWriteThrough() blockstore.
hsanjuan Feb 27, 2023
2cd3f88
Accept options for blockstore: start with WriteThrough and NoPrefix
hsanjuan Mar 8, 2023
eeceee5
feat: stub and deprecate NewBlockstoreNoPrefix
Jorropo Mar 8, 2023
fdfe12c
feat: human-readable cache keys for IPFS_NS_MAP (#38)
laurentsenta Mar 8, 2023
a8700eb
docs: improved DNSLink lookup error
lidel Mar 8, 2023
8a12444
fix: let `extract` skip missing unixfs shard links
rvagg Mar 8, 2023
0405449
ReadWrite: faster Has() by using the in-memory index instead of readi…
MichaelMure Mar 14, 2023
ca32fef
fix: handle (and test) WholeCID vs not; fast Has() path for storage
rvagg Mar 17, 2023
4cb3791
blockstore: only close the file on error in OpenReadWrite, not OpenR…
MichaelMure Mar 16, 2023
c270c83
blockstore: give a direct access to the index for read operations
MichaelMure Mar 10, 2023
038bdd2
feat: Über Migration (and Boxo rename)
Jorropo Mar 23, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
11 changes: 6 additions & 5 deletions .github/workflows/gateway-sharness.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,24 +19,25 @@ jobs:
uses: actions/setup-go@v3
with:
go-version: 1.19.1
- name: Checkout go-libipfs
- name: Checkout boxo
uses: actions/checkout@v3
with:
path: go-libipfs
path: boxo
- name: Checkout Kubo
uses: actions/checkout@v3
with:
repository: ipfs/kubo
path: kubo
ref: 503edee648e29c62888f05fa146ab13d9c65077d
- name: Install Missing Tools
run: sudo apt install -y socat net-tools fish libxml2-utils
- name: Restore Go Cache
uses: protocol/cache-go-action@v1
with:
name: ${{ github.job }}
- name: Replace go-libipfs in Kubo go.mod
- name: Replace boxo in Kubo go.mod
run: |
go mod edit -replace=github.com/ipfs/go-libipfs=../go-libipfs
go mod edit -replace=github.com/ipfs/boxo=../boxo
go mod tidy
cat go.mod
working-directory: kubo
Expand All @@ -47,5 +48,5 @@ jobs:
run: find . -maxdepth 1 -name "*gateway*.sh" -print0 | xargs -0 -I {} bash -c "echo {}; {}"
working-directory: kubo/test/sharness
- name: Run Kubo CLI Tests
run: go test -v -run=Gateway .
run: go test -run=Gateway .
working-directory: kubo/test/cli
6 changes: 3 additions & 3 deletions .github/workflows/go-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
# Use -coverpkg=./..., so that we include cross-package coverage.
# If package ./A imports ./B, and ./A's tests also cover ./B,
# this means ./B's coverage will be significantly higher than 0%.
run: go test -v -shuffle=on -coverprofile=module-coverage.txt -coverpkg=./... ./...
run: go test -shuffle=on -coverprofile=module-coverage.txt -coverpkg=./... ./...
- name: Run tests (32 bit)
# can't run 32 bit tests on OSX.
if: matrix.os != 'macos' &&
Expand All @@ -58,14 +58,14 @@ jobs:
with:
run: |
export "PATH=$PATH_386:$PATH"
go test -v -shuffle=on ./...
go test -shuffle=on ./...
- name: Run tests with race detector
# speed things up. Windows and OSX VMs are slow
if: matrix.os == 'ubuntu' &&
contains(fromJSON(steps.config.outputs.json).skipOSes, matrix.os) == false
uses: protocol/multiple-go-modules@v1.2
with:
run: go test -v -race ./...
run: go test -race ./...
- name: Collect coverage files
shell: bash
run: echo "COVERAGES=$(find . -type f -name 'module-coverage.txt' | tr -s '\n' ',' | sed 's/,$//')" >> $GITHUB_ENV
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/test-examples.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
- name: Run tests
uses: protocol/multiple-go-modules@v1.2
with:
run: go test -v -shuffle=on ./...
run: go test -shuffle=on ./...
- name: Run tests (32 bit)
if: ${{ matrix.os != 'macos' }} # can't run 32 bit tests on OSX.
uses: protocol/multiple-go-modules@v1.2
Expand All @@ -46,9 +46,9 @@ jobs:
with:
run: |
export "PATH=${{ env.PATH_386 }}:$PATH"
go test -v -shuffle=on ./...
go test -shuffle=on ./...
- name: Run tests with race detector
if: ${{ matrix.os == 'ubuntu' }} # speed things up. Windows and OSX VMs are slow
uses: protocol/multiple-go-modules@v1.2
with:
run: go test -v -race ./...
run: go test -race ./...
54 changes: 27 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
<h1 align="center">
go-libipfs 🍌
Boxo 🍌
<br>
<img src="https://raw.githubusercontent.com/ipfs/go-libipfs/main/logo.svg" alt="go-libipfs logo" title="go-libipfs logo" width="200">
<img src="https://raw.githubusercontent.com/ipfs/boxo/main/logo.svg" alt="Boxo logo" title="Boxo logo" width="200">
<br>
</h1>
<p align="center" style="font-size: 1.2rem;">A library for building IPFS applications and implementations.</p>

<hr />

[![Go Test](https://github.com/ipfs/go-libipfs/actions/workflows/go-test.yml/badge.svg)](https://github.com/ipfs/go-libipfs/actions/workflows/go-test.yml)
[![Go Docs](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/ipfs/go-libipfs)
[![codecov](https://codecov.io/gh/ipfs/go-libipfs/branch/main/graph/badge.svg?token=9eG7d8fbCB)](https://codecov.io/gh/ipfs/go-libipfs)
[![Go Test](https://github.com/ipfs/boxo/actions/workflows/go-test.yml/badge.svg)](https://github.com/ipfs/boxo/actions/workflows/go-test.yml)
[![Go Docs](https://img.shields.io/badge/godoc-reference-blue.svg)](https://pkg.go.dev/github.com/ipfs/boxo)
[![codecov](https://codecov.io/gh/ipfs/boxo/branch/main/graph/badge.svg?token=9eG7d8fbCB)](https://codecov.io/gh/ipfs/boxo)

<!-- TOC -->

- [About](#about)
- [Motivation](#motivation)
- [What kind of components does go-libipfs have?](#what-kind-of-components-does-go-libipfs-have)
- [Does go-libipfs == IPFS?](#does-go-libipfs--ipfs)
- [What kind of components does Boxo have?](#what-kind-of-components-does-boxo-have)
- [Does Boxo == IPFS?](#does-boxo--ipfs)
- [Is everything related to IPFS in the Go ecosystem in this repo?](#is-everything-related-to-ipfs-in-the-go-ecosystem-in-this-repo)
- [Getting started](#getting-started)
- [Should I add my IPFS component to go-libipfs?](#should-i-add-my-ipfs-component-to-go-libipfs)
- [Should I add my IPFS component to Boxo?](#should-i-add-my-ipfs-component-to-boxo)
- [Help](#help)
- [Governance and Access](#governance-and-access)
- [Release Process](#release-process)
Expand All @@ -31,75 +31,75 @@ go-libipfs 🍌

## About

go-libipfs is a component library for building IPFS applications and implementations in Go.
Boxo is a component library for building IPFS applications and implementations in Go.

Some scenarios in which you may find go-libipfs helpful:
Some scenarios in which you may find Boxo helpful:

* You are building an application that interacts with the IPFS network
* You are building an IPFS implementation
* You want to reuse some components of IPFS such as its Kademlia DHT, Bitswap, data encoding, etc.
* You want to experiment with IPFS

go-libipfs powers [Kubo](https://github.com/ipfs/kubo), which is [the most popular IPFS implementation](https://github.com/protocol/network-measurements/tree/master/reports),
Boxo powers [Kubo](https://github.com/ipfs/kubo), which is [the most popular IPFS implementation](https://github.com/protocol/network-measurements/tree/master/reports),
so its code has been battle-tested on the IPFS network for years, and is well-understood by the community.

### Motivation
**TL;DR** The goal of this repo is to help people build things. Previously users struggled to find existing useful code or to figure out how to use what they did find. We observed many running Kubo and using its HTTP RPC API. This repo aims to do better. We're taking the libraries that many were already effectively relying on in production and making them more easily discoverable and usable.

The maintainers primarily aim to help people trying to build with IPFS in Go that were previously either giving up or relying on the [Kubo HTTP RPC API](https://docs.ipfs.tech/reference/kubo/rpc/). Some of these people will end up being better served by IPFS tooling in other languages (e.g., Javascript, Rust, Java, Python), but for those who are either looking to write in Go or to leverage the set of IPFS tooling we already have in Go we’d like to make their lives easier.
The maintainers primarily aim to help people trying to build with IPFS in Go that were previously either giving up or relying on the [Kubo HTTP RPC API](https://docs.ipfs.tech/reference/kubo/rpc/). Some of these people will end up being better served by IPFS tooling in other languages (e.g., Javascript, Rust, Java, Python), but for those who are either looking to write in Go or to leverage the set of IPFS tooling we already have in Go we’d like to make their lives easier.

We’d also like to make life easier on ourselves as the maintainers by reducing the maintenance burden that comes from being the owners on [many repos](https://github.com/ipfs/kubo/issues/8543) and then use that time to contribute more to the community in the form of easier to use libraries, better implementations, improved protocols, new protocols, etc.
We’d also like to make life easier on ourselves as the maintainers by reducing the maintenance burden that comes from being the owners on [many repos](https://github.com/ipfs/kubo/issues/8543) and then use that time to contribute more to the community in the form of easier to use libraries, better implementations, improved protocols, new protocols, etc.

Go-libipfs is not exhaustive nor comprehensive--there are plenty of useful IPFS protocols, specs, libraries, etc. that are not in go-libipfs. The goal of go-libipfs is to provide cohesive and well-maintained components for common IPFS use cases.
Boxo is not exhaustive nor comprehensive--there are plenty of useful IPFS protocols, specs, libraries, etc. that are not in Boxo. The goal of Boxo is to provide cohesive and well-maintained components for common IPFS use cases.

## What kind of components does go-libipfs have?
## What kind of components does Boxo have?

Go-libipfs includes high-quality components useful for interacting with IPFS protocols, public and private IPFS networks, and content-addressed data, such as:
Boxo includes high-quality components useful for interacting with IPFS protocols, public and private IPFS networks, and content-addressed data, such as:

- Content routing (DHT, delegated content routing, providing)
- Data transfer (gateways, Bitswap, incremental verification)
- Naming and mutability (name resolution, IPNS)
- Interacting with public and private IPFS networks
- Working with content-addressed data

Go-libipfs aims to provide a cohesive interface into these components. Note that not all of the underlying components necessarily reside in this respository.
Boxo aims to provide a cohesive interface into these components. Note that not all of the underlying components necessarily reside in this respository.

### Does go-libipfs == IPFS?
### Does Boxo == IPFS?
No. This repo houses some IPFS functionality written in Go that has been useful in practice, and is maintained by a group that has long term commitments to the IPFS project

### Is everything related to IPFS in the Go ecosystem in this repo?

No. Not everything related to IPFS is intended to be in go-libipfs. View it as a starter toolbox (potentially among multiple). If you’d like to build an IPFS implementation with Go, here are some tools you might want that are maintained by a group that has long term commitments to the IPFS project. There are certainly repos that others maintainer that aren't included here (e.g., ipfs/go-car) which are still useful to IPFS implementations. It's expected and fine for new IPFS functionality to be developed that won't be part of go-libipfs.
No. Not everything related to IPFS is intended to be in Boxo. View it as a starter toolbox (potentially among multiple). If you’d like to build an IPFS implementation with Go, here are some tools you might want that are maintained by a group that has long term commitments to the IPFS project. There are certainly repos that others maintainer that aren't included here (e.g., ipfs/go-car) which are still useful to IPFS implementations. It's expected and fine for new IPFS functionality to be developed that won't be part of Boxo.

## Getting started
See [examples](./examples/README.md).

## Should I add my IPFS component to go-libipfs?
We happily accept external contributions! However, go-libipfs maintains a high quality bar, so code accepted into go-libipfs must meet some minimum maintenance criteria:
## Should I add my IPFS component to Boxo?
We happily accept external contributions! However, Boxo maintains a high quality bar, so code accepted into Boxo must meet some minimum maintenance criteria:

* Actively maintained
* Must be actively used by, or will be included in software that is actively used by, a significant number of users or production systems. Code that is not actively used cannot be properly maintained.
* Must have multiple engineers who are willing and able to maintain the relevant code in go-libipfs for a long period of time.
* If either of these changes, go-libipfs maintainers will consider removing the component from go-libipfs.
* Must have multiple engineers who are willing and able to maintain the relevant code in Boxo for a long period of time.
* If either of these changes, Boxo maintainers will consider removing the component from Boxo.
* Adequately tested
* At least with unit tests
* Ideally also including integration tests with other components
* Adequately documented
* Godocs at minimum
* Complex components should have their own doc.go or README.md describing the component, its use cases, tradeoffs, design rationale, etc.
* If the maintainers are not go-libipfs maintainers, then the component must include a CODEOWNERS file with at least two code owners who can commit to reviewing PRs
* If the maintainers are not Boxo maintainers, then the component must include a CODEOWNERS file with at least two code owners who can commit to reviewing PRs

If you have some experimental component that you think would benefit the IPFS community, we suggest you build the component in your own repository until it's clear that there's community demand for it, and then open an issue/PR in this repository to discuss including it in go-libipfs.
If you have some experimental component that you think would benefit the IPFS community, we suggest you build the component in your own repository until it's clear that there's community demand for it, and then open an issue/PR in this repository to discuss including it in Boxo.

## Help

If you have questions, feel free to open an issue. You can also find the go-libipfs maintainers in [Filecoin Slack](https://filecoin.io/slack/) at #go-libipfs-maintainers. (If you would like to engage via IPFS Discord or ipfs.io Matrix, please drop into the #ipfs-implementers channel/room or file an issue, and we'll get bridging from #go-libipfs-maintainers to these other chat platforms.)
If you have questions, feel free to open an issue. You can also find the Boxo maintainers in [Filecoin Slack](https://filecoin.io/slack/) at #Boxo-maintainers. (If you would like to engage via IPFS Discord or ipfs.io Matrix, please drop into the #ipfs-implementers channel/room or file an issue, and we'll get bridging from #Boxo-maintainers to these other chat platforms.)

## Governance and Access
See [CODEOWNERS](./docs/CODEOWNERS) for the current maintainers list. Governance for graduating additional maintainers hasn't been established. Repo permissions are all managed through [ipfs/github-mgmt](https://github.com/ipfs/github-mgmt).

## Release Process
To be documented: https://github.com/ipfs/go-libipfs/issues/170
To be documented: https://github.com/ipfs/boxo/issues/170

## Related Items
* [Initial proposal for "Consolidate IPFS Repositories" that spawned this project](https://github.com/ipfs/kubo/issues/8543)
Expand Down
4 changes: 2 additions & 2 deletions bitswap/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ wants those blocks.
```golang
import (
"context"
bitswap "github.com/ipfs/go-libipfs/bitswap"
bsnet "github.com/ipfs/go-libipfs/bitswap/network"
bitswap "github.com/ipfs/boxo/bitswap"
bsnet "github.com/ipfs/boxo/bitswap/network"
blockstore "github.com/ipfs/go-ipfs-blockstore"
"github.com/libp2p/go-libp2p-core/routing"
"github.com/libp2p/go-libp2p-core/host"
Expand Down
14 changes: 7 additions & 7 deletions bitswap/benchmarks_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,17 @@ import (
"testing"
"time"

"github.com/ipfs/go-libipfs/bitswap/internal/testutil"
blocks "github.com/ipfs/go-libipfs/blocks"
"github.com/ipfs/boxo/bitswap/internal/testutil"
blocks "github.com/ipfs/go-block-format"
protocol "github.com/libp2p/go-libp2p/core/protocol"

"github.com/ipfs/boxo/bitswap"
bsnet "github.com/ipfs/boxo/bitswap/network"
testinstance "github.com/ipfs/boxo/bitswap/testinstance"
tn "github.com/ipfs/boxo/bitswap/testnet"
mockrouting "github.com/ipfs/boxo/routing/mock"
cid "github.com/ipfs/go-cid"
delay "github.com/ipfs/go-ipfs-delay"
mockrouting "github.com/ipfs/go-ipfs-routing/mock"
"github.com/ipfs/go-libipfs/bitswap"
bsnet "github.com/ipfs/go-libipfs/bitswap/network"
testinstance "github.com/ipfs/go-libipfs/bitswap/testinstance"
tn "github.com/ipfs/go-libipfs/bitswap/testnet"
)

type fetchFunc func(b *testing.B, bs *bitswap.Bitswap, ks []cid.Cid)
Expand Down
18 changes: 9 additions & 9 deletions bitswap/bitswap.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,18 @@ import (
"context"
"fmt"

"github.com/ipfs/go-libipfs/bitswap/client"
"github.com/ipfs/go-libipfs/bitswap/internal/defaults"
"github.com/ipfs/go-libipfs/bitswap/message"
"github.com/ipfs/go-libipfs/bitswap/network"
"github.com/ipfs/go-libipfs/bitswap/server"
"github.com/ipfs/go-libipfs/bitswap/tracer"
"github.com/ipfs/boxo/bitswap/client"
"github.com/ipfs/boxo/bitswap/internal/defaults"
"github.com/ipfs/boxo/bitswap/message"
"github.com/ipfs/boxo/bitswap/network"
"github.com/ipfs/boxo/bitswap/server"
"github.com/ipfs/boxo/bitswap/tracer"
"github.com/ipfs/go-metrics-interface"

blockstore "github.com/ipfs/boxo/blockstore"
exchange "github.com/ipfs/boxo/exchange"
blocks "github.com/ipfs/go-block-format"
"github.com/ipfs/go-cid"
blockstore "github.com/ipfs/go-ipfs-blockstore"
exchange "github.com/ipfs/go-ipfs-exchange-interface"
blocks "github.com/ipfs/go-libipfs/blocks"
logging "github.com/ipfs/go-log"
"github.com/libp2p/go-libp2p/core/peer"

Expand Down
16 changes: 8 additions & 8 deletions bitswap/bitswap_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,19 @@ import (
"testing"
"time"

"github.com/ipfs/boxo/bitswap"
bsmsg "github.com/ipfs/boxo/bitswap/message"
"github.com/ipfs/boxo/bitswap/server"
testinstance "github.com/ipfs/boxo/bitswap/testinstance"
tn "github.com/ipfs/boxo/bitswap/testnet"
"github.com/ipfs/boxo/internal/test"
mockrouting "github.com/ipfs/boxo/routing/mock"
blocks "github.com/ipfs/go-block-format"
cid "github.com/ipfs/go-cid"
detectrace "github.com/ipfs/go-detect-race"
blocksutil "github.com/ipfs/go-ipfs-blocksutil"
delay "github.com/ipfs/go-ipfs-delay"
mockrouting "github.com/ipfs/go-ipfs-routing/mock"
ipld "github.com/ipfs/go-ipld-format"
"github.com/ipfs/go-libipfs/bitswap"
bsmsg "github.com/ipfs/go-libipfs/bitswap/message"
"github.com/ipfs/go-libipfs/bitswap/server"
testinstance "github.com/ipfs/go-libipfs/bitswap/testinstance"
tn "github.com/ipfs/go-libipfs/bitswap/testnet"
blocks "github.com/ipfs/go-libipfs/blocks"
"github.com/ipfs/go-libipfs/internal/test"
tu "github.com/libp2p/go-libp2p-testing/etc"
p2ptestutil "github.com/libp2p/go-libp2p-testing/netutil"
peer "github.com/libp2p/go-libp2p/core/peer"
Expand Down
14 changes: 7 additions & 7 deletions bitswap/client/bitswap_with_sessions_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,16 @@ import (
"testing"
"time"

"github.com/ipfs/boxo/bitswap"
"github.com/ipfs/boxo/bitswap/client/internal/session"
testinstance "github.com/ipfs/boxo/bitswap/testinstance"
tn "github.com/ipfs/boxo/bitswap/testnet"
"github.com/ipfs/boxo/internal/test"
mockrouting "github.com/ipfs/boxo/routing/mock"
blocks "github.com/ipfs/go-block-format"
cid "github.com/ipfs/go-cid"
blocksutil "github.com/ipfs/go-ipfs-blocksutil"
delay "github.com/ipfs/go-ipfs-delay"
mockrouting "github.com/ipfs/go-ipfs-routing/mock"
"github.com/ipfs/go-libipfs/bitswap"
"github.com/ipfs/go-libipfs/bitswap/client/internal/session"
testinstance "github.com/ipfs/go-libipfs/bitswap/testinstance"
tn "github.com/ipfs/go-libipfs/bitswap/testnet"
blocks "github.com/ipfs/go-libipfs/blocks"
"github.com/ipfs/go-libipfs/internal/test"
tu "github.com/libp2p/go-libp2p-testing/etc"
)

Expand Down
Loading