Proposal: migrate from sharness

[guseggert]

Here's a PoC of writing Go tests for CLI testing instead of sharness, see https://github.com/ipfs/go-ipfs/tree/guseggert/test-harness/test/cli

I wanted to answer the question to myself "What would it look like to test the CLI with Go tests instead of sharness?" Other folks seems interested in this, so I am bringing this up for discussion.

Benefits of sharness are that the tests are generally easier to read, since Bash is pretty terse, and is closer to how most people use the go-ipfs CLI. Also we'd avoid having to rewrite our entire test suite.

Downsides of sharness are that Bash is a pretty limited language so sophisticated tests are awkward or require additional binaries that we have to build and other deps. Many of these are Go programs like go-sleep, go-timeout, graphsync-get, ma-pipe-unidir, etc. that add to build times but could trivially be part of Go tests, and also other dependencies like jq, perl, socat, etc.

If folks are interested in this, I'd suggest we try out writing a few non-sharness tests and see if we like it, and if we don't we can scrap it and move on. If we like it, we migrate things over. We can chip away at this over time or do it all at once, I'm not sure which is better.

[Stebalien]

Also note: we can probably move most of our tests into https://github.com/ipfs/interface-go-ipfs-core/tree/master/tests (we'll probably want to do some repo consolidation first?). These tests can "orchestrate" themselves with https://github.com/ipfs/go-ipfs/blob/a651045c502435b5dfea0d928dc243a7c02b6b04/core/coreapi/test/api_test.go#L2.

We'd probably want to make a version of that that uses iptb so we test the actual binaries instead of having all in-process tests.

[BigLep]

The repo consolidation effort is #8543

[guseggert]

I went ahead and stuck w/ exercising the CLI for now, I basically just ported the relevant parts of iptb over to a test harness that's used in Go tests. These Go tests are just invoking the kubo binary, and the test harness is creating clusters of individual nodes like iptb:

#9489

[guseggert]

We've merged #9489 so I think this issue can be closed now.