New "west rebase" command? Decompose 'west update -r' into 'update' + '-r'

[marc-hb]

Sincere thanks to @mbolivar and @tejlmand for helping brainstorm and sanity check this on Slack. Note helping is not endorsing :-)

As the the west equivalent of git pull --rebase, west update -r is a very convenient command that does a lot at once. However it has a few of gaps compared to git pull --rebase. This request is not about eliminating but rather mitigating one of these gaps. More specifically, it's about adding a new command or option performing something somewhat similar to what -r already adds to west update, so users can "divide and conquer" west update -r into two consecutive commands.

The name west rebase may be used as a placeholder for this new command or option. Feel free to suggest better names but please don't bikeshed them until the detailed behavior of the feature has been fully defined and approved (if ever).

A specific use case where this new command would help is:

• work in progress to rebase across multiple repos
• relatively large conflicts.

For instance: refactoring some cross-repo API like a CMake API.

Another use case is: west developers working on west update -r itself.

Neither should admittedly be a very common use case and I can't think of other ones so please prioritize accordingly.

Relevant git rebase background

(west-free section)

As it performs multiple things at once git pull --rebase is very convenient... when it succeeds without conflict. When it fails, there are basically two approaches depending on the magnitude of conflicts:

1. --continue. When the number of conflicting lines is perceived as small, the user will typically persist and try to resolve them. Perform some testing and then use git rebase --continue.

2. When the conflicts are large, a feeling of drowning/getting lost in a vast amount of new code is typical and normal. Fortunately, git rebase --abort lets the user cancel (almost) the entire operation and reconsider.

rebase --abort is a safety net that lets users take risks with the strong confidence that they can take a big bet on what happened upstream and easily fall back on slower approaches when they lose the bet. From experience, these slower approaches all involve some form of "divide and conquer". Rebasing or merging is combining multiple things together and there's a finite number of things the human brain can focus on simultaneously. Divide and conquer examples:

• Break down the work in progress into more than one feature/bugfix branch and rebase and retest each smaller feature in progress one by one
• Find stable "base camps" in the upstream history and incrementally "climb up" that broken down history, rebasing and retesting on those base camps one by one instead of going straight for the summit.

Back to west 0.6.3

1. As a multi-repo tool with a manifest, does even more than git pull --rebase. This increases the complexity and the chances of the user getting lost in new upstream code. This can also make harder to find global, consistent states that can be tested.
2. Yet west 0.6.3 forces users to take a bet on the upstream history and do everything at once anyway.
3. It does not provide any super simple --abort safety net either.

This west rebase proposal is focused on addressing 2 and nothing else. Some sort(s) of west update --abort feature(s) would be very interesting. It has already been discussed in issue #81 and other places. However this particular github issue is NOT about --abort. It's about something simpler and hopefully much easier to achieve: the ability to divide and conquer west rebase -r into two steps: A. west update B. something similar to -r. It assumes the user either anticipated the conflicts and didn't take the bet, or that she found some other way to --abort.

PS: even without -r, west update can already conflict.

cc:

• west update: generalize -k / -r to include commits, not just branches #381
• west compare (and others) don't sync manifest-rev with west.yml: need "west fetch" #747

[marc-hb]

As discussed on Slack, a reasonable requirement for this new west rebase would probably be: same branch name across all repos. This is a very common requirement in grepo.

[mbolivar]

@tejlmand has separately suggested a west fetch [PROJECT ...] command, which would update manifest-rev in each project without changing the working tree. I agree, for separate reasons not relevant to this issue, that this is a useful command we should have.

We can debate that command in a separate issue if you prefer your solution.

However, west fetch followed by west forall -c 'git rebase manifest-rev' [PROJECT ...] would factor west update -r into two steps in a way that does not "[force] users to take a bet on the upstream history and do everything at once" as you've accurately described it :).

[marc-hb]

OK, so the answer to this west rebase ... request is west forall -c 'git rebase ... and in #337 the answer to my west grep ... request was (surprise) west forall -c 'git grep ....

Anyone seeing a pattern? :-) But wait, there's a related west diff... discussion in #337 too.

Rephrasing my 20 November suggestion in #337: can we have a generic

west forallgit {rebase,grep,diff,whatever} git args no second level quoting required

UPDATE: @mbolivar resumed that discussion in #337 at the exact same time I posted this.

[marc-hb]

@tejlmand has separately suggested a west fetch [PROJECT ...] command,

Depending on its semantics I agree some kind of west fetch would very likely help with the situations I described.

Did @tejlmand suggest this on github? I couldn't find it.
On the other hand, I found traces of a git fetch command that disappeared?! See #31 , #76 , #78 and #92

[marc-hb]

As it performs multiple things at once git pull --rebase is very convenient... when it succeeds without conflict.

BTW jj solves this by "committing" conflicts:
https://martinvonz.github.io/jj/latest/conflicts/#first-class-conflicts