Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 16.0 GitHub #70

Closed
wants to merge 512 commits into from
Closed

Release 16.0 GitHub #70

wants to merge 512 commits into from

Conversation

alexisjcarr
Copy link

Description

Related Issue(s)

Checklist

  • "Backport to:" labels have been added if this change should be back-ported
  • Tests were added or are not required
  • Documentation was added or is not required

Deployment Notes

harshit-gangal and others added 30 commits January 13, 2023 09:38
…ssio#11987)

Signed-off-by: Harshit Gangal <harshit@planetscale.com>

Signed-off-by: Harshit Gangal <harshit@planetscale.com>
GitHub did not seem to pick up the merge conflict between
these two commits (older to newer):
  - vitessio@673573a
  - vitessio@836b3c1

The first commit changed the function signature for
testOnlineDDLStatement(), while the later commit
used the old signature.

Signed-off-by: Matt Lord <mattalord@gmail.com>

Signed-off-by: Matt Lord <mattalord@gmail.com>
* Remove dead Workflow mgmt code

This was pre-Legacy sharding, which was in turn pre-VReplication.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Remove related proto

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Get rid of related flag usage

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Add release notes line and run go mod tidy

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Add missing command to removed list in release notes

Signed-off-by: Matt Lord <mattalord@gmail.com>

Signed-off-by: Matt Lord <mattalord@gmail.com>
* ComBinlogDumpGTID: receive logFile, logPos

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* fix error code

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* kontinue by default true

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* - Added `parseComBinlogDump`
- Added `BinlogEvent.IsHeartbeat()`
- Fixed `NewRotateEvent()` bug
- Added `NewFakeRotateEvent()`
- Added `NewHeartbeatEvent()`
- `mysql/conn.go` handles `ComBinlogDump`
- `mysql/conn.go` handles `ComRegisterReplica`
- Added `Conn.WriteBinlogEvent()`
- `SendBinlogDumpCommand()` accepts a binlog file name, per https://dev.mysql.com/doc/internals/en/com-binlog-dump-gtid.html

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* adding a couple unit tests

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* updates to go/mysql/binlog_event_make.go

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* updates to go/mysql/conn.go

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* more tests

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* endtoend/cluster: create a cluster without assuming availability of binaries

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* endtoend/cluster: more MysqlctlProcess functionality

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* indentation

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* QueryMatchesTemplates: CanonicalString

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* fix comments

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* Propagate binlog stream errors

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Add new unit test

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Ensure err channel I/O does not block on ctx cancellation

For example, during PRS.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Remove unnecessary WaitGroup usage in unit test

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Log error message when context is closed.

This maintains the previous behavior for the log messages
and ensures that is consistent when the context is closed
or not.

Signed-off-by: Matt Lord <mattalord@gmail.com>

Signed-off-by: Matt Lord <mattalord@gmail.com>
* feat: add failing parsing test and fix parser

Signed-off-by: Manan Gupta <manan@planetscale.com>

* test: add e2e tests for interval with math functions

Signed-off-by: Manan Gupta <manan@planetscale.com>

* test: explictly set the time-zone to prevent failures in CI

Signed-off-by: Manan Gupta <manan@planetscale.com>

Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add failing test and fix the query of insertion

Signed-off-by: Manan Gupta <manan@planetscale.com>

* empty-commit

Signed-off-by: Manan Gupta <manan@planetscale.com>

Signed-off-by: Manan Gupta <manan@planetscale.com>
…itessio#12098)

* message Rule: convert_int_to_enum

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* OnlineDDL/vreplication: handle conversion of integral type to enum type

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* admin web proto

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* minor naming and comment changes

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* store transaction isolation level in upper case

Signed-off-by: Harshit Gangal <harshit@planetscale.com>

* store system variables in the case as provided in StorageCase type

Signed-off-by: Harshit Gangal <harshit@planetscale.com>

Signed-off-by: Harshit Gangal <harshit@planetscale.com>
…12105)

The canonical string representation needs to keep the correct case for
the charset. Those are never upcased in MySQL so we should not enforce
that here either.

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
…upported (vitessio#12111)

Signed-off-by: Manan Gupta <manan@planetscale.com>

Signed-off-by: Manan Gupta <manan@planetscale.com>
…o#12107)

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
* test: add a failing test for the case where the port changes for a tablet

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix the issue by adding alias as a unique field

Signed-off-by: Manan Gupta <manan@planetscale.com>

* empty-commit

Signed-off-by: Manan Gupta <manan@planetscale.com>

Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: notfelineit <notfelineit@gmail.com>
Signed-off-by: Frances Thai <31225471+notfelineit@users.noreply.github.com>
Signed-off-by: notfelineit <notfelineit@gmail.com>
Signed-off-by: notfelineit <notfelineit@gmail.com>
Signed-off-by: notfelineit <notfelineit@gmail.com>
* Use Gen4 as the default planner version for VTExplain

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>

* change e2e test output

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>

* use querypb.ExecuteOptions_DEFAULT_PLANNER as the default vtexplain planner version

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* notify view change to vtgate

Signed-off-by: Harshit Gangal <harshit@planetscale.com>

* test: added unit test

Signed-off-by: Harshit Gangal <harshit@planetscale.com>

* fix: fix e2e test query

Signed-off-by: Harshit Gangal <harshit@planetscale.com>

Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: notfelineit <notfelineit@gmail.com>
Signed-off-by: notfelineit <notfelineit@gmail.com>
* Update SafeRewrite to allow for replacing a node that we don't want to visit the children of

Signed-off-by: Andres Taylor <andres@planetscale.com>

* Clean up ast_rewriting before refactoring it

Signed-off-by: Andres Taylor <andres@planetscale.com>

* feat: fix usages of Rewrite in ast_rewriting

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix usages of Rewrite in normalizer

Signed-off-by: Manan Gupta <manan@planetscale.com>

* safe rewrite having clause during horizon planning

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>

* move rewriteHavingAndOrderBy to safeRewrite

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>

* refactor: clean up code

Signed-off-by: Andres Taylor <andres@planetscale.com>

Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Co-authored-by: Manan Gupta <manan@planetscale.com>
Co-authored-by: Florent Poinsard <florent.poinsard@outlook.fr>
* feat: remove deprecated fields in logstats

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: remove enable_semi_sync

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: remove QueryRowCounts stat

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: remove deprecated and unused field

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: remove unused code

Signed-off-by: Manan Gupta <manan@planetscale.com>

* release-notes: fix review comments in release-notes

Signed-off-by: Manan Gupta <manan@planetscale.com>

* test: remove enable_semi_sync flag from the tests

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: remove tests for QueryRowCounts

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: deprecate QueriesProcessed and QueriesRouted

Signed-off-by: Manan Gupta <manan@planetscale.com>

* docs: make versions consistant in the release summary

Signed-off-by: Manan Gupta <manan@planetscale.com>

Signed-off-by: Manan Gupta <manan@planetscale.com>
* Strip secondary keys during schema copy

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Save ALTER to run when copy finishes

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Move work to vcopier

Signed-off-by: Matt Lord <mattalord@gmail.com>

* go mod tidy

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Re-apply stashed secondary keys

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Minor fixes and cleanup

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Add Reshard support

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Support multiple SQL actions

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Rename action->>"$.action" to action->>"$.task"

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Fix unit tests

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Add new unit test

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Improve the unit test

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Add vtctl --defer-secondary-keys client flag

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Fix binlog unit tests

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Fix wrangler unit tests

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Move workflow type check to a function

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Update error in unit test and add OnlineDDL

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Minor unit test improvements

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Improve name of table used

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Improvements after self review

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Ensure ALTERs are killed when engine shuts down

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Fixup & deflake the unit test

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Minor improvements after self review

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Auto handle more edge cases

Signed-off-by: Matt Lord <mattalord@gmail.com>

* More unit test improvements

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Address review comments

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Correct new error handling in killAction

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Improve getTableSecondaryKeys function

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Nitty comment improvements

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Address review comments

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Minor improvements to log message

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Fix unit tests

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Add defer_secondary_keys to new unit tests added via main merge

Signed-off-by: Matt Lord <mattalord@gmail.com>

* More unit test fixes

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Minor improvements to the log messages

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Minor changes and a fix after self review

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Ignore --defer-secondary-keys for Reshard merges

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Nitty improvement to comment

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Add full support for shard merges

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Move newClientConnection() to vreplicator

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Add unit test for new behavior

Other fixes and minor improvements

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Use the engine context rather than vreplicator context

Otherwise the ALTER gets KILLed when the copy phase ends

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Improve testing and other minor tweaks

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Add post copy action cleanup to workflow cancel

Signed-off-by: Matt Lord <mattalord@gmail.com>

* More unit test fixes for post copy action cleanup

Signed-off-by: Matt Lord <mattalord@gmail.com>

Signed-off-by: Matt Lord <mattalord@gmail.com>
* add debug tooling

Signed-off-by: Andres Taylor <andres@planetscale.com>

* move dbg to test

Signed-off-by: Andres Taylor <andres@planetscale.com>

Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: notfelineit <notfelineit@gmail.com>
* more testing for getViewDependentTableNames()

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* getForeignKeyParentTableNames

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* schema normalization: sort tables by fk reference order

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* minor test addition

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* ForeignKeyDependencyUnresolvedError

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* InvalidReferencedColumnInForeignKeyConstraintError, MismatchingForeignKeyColumnCountError, MismatchingForeignKeyColumnTypeError

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* rename error: ForeignKeyColumnCountMismatchError

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* rename MismatchingForeignKeyColumnTypeError to ForeignKeyColumnTypeMismatchError

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* changes per review

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
…tld2

[vtctld2] Remove vtctld2 UI and vtctld server components that serve the app UI
GuptaManan100 and others added 21 commits July 28, 2023 17:19
…sue (vitessio#13736)

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
…vitessio#13759)

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
…ob Service (vitessio#13770) (vitessio#13774)

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
…own within CI (vitessio#13793) (vitessio#13797)

Signed-off-by: Matt Lord <mattalord@gmail.com>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
…from schema tracker (vitessio#13673) (vitessio#13795)

Signed-off-by: Matt Lord <mattalord@gmail.com>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
Co-authored-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
…essio#13805)

Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Co-authored-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
vitessio#13813) (vitessio#13819)

Signed-off-by: Andrew Mason <andrew@planetscale.com>
Co-authored-by: Andrew Mason <andrew@planetscale.com>
…ation e2e test (vitessio#13803) (vitessio#13821)

Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
…Diff2 Progress command is in the future. (vitessio#13804) (vitessio#13816)

Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
…uld not be created when a movetables workflow is created (vitessio#13858)

Signed-off-by: Rohit Nayak <rohit@planetscale.com>
…ules should not be created when a movetables workflow is created" (vitessio#13899)

Signed-off-by: Rohit Nayak <rohit@planetscale.com>
… migrations (vitessio#13928) (vitessio#13936)

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
…essio#13921)

Co-authored-by: Florent Poinsard <35779988+frouioui@users.noreply.github.com>
Co-authored-by: Florent Poinsard <florent.poinsard@outlook.fr>
Co-authored-by: frouioui <frouioui@users.noreply.github.com>
Co-authored-by: Florent Poinsard <florent.poinsard@outlook.fr>
@alexisjcarr alexisjcarr self-assigned this Sep 12, 2023
@alexisjcarr alexisjcarr changed the base branch from upstream-release-16.0 to master September 12, 2023 22:21
@alexisjcarr alexisjcarr changed the base branch from master to upstream-main September 12, 2023 22:21
@alexisjcarr alexisjcarr changed the base branch from upstream-main to upstream-release-16.0 September 12, 2023 22:21
@alexisjcarr alexisjcarr changed the base branch from upstream-release-16.0 to master September 12, 2023 22:34
Signed-off-by: Patrick Carnahan <patrickcarnahan@github.com>
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
@arthurschreiber
Copy link
Member

Closing this in favor of #78, which contains any additional backports we require.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.