-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Materialize workflow support for reference tables #16787
base: main
Are you sure you want to change the base?
Conversation
Review ChecklistHello reviewers! 👋 Please follow this checklist when reviewing this Pull Request. General
Tests
Documentation
New flags
If a workflow is added or modified:
Backward compatibility
|
4ea1434
to
cdfa6b3
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #16787 +/- ##
==========================================
- Coverage 69.53% 69.51% -0.02%
==========================================
Files 1567 1567
Lines 202388 202421 +33
==========================================
- Hits 140723 140711 -12
- Misses 61665 61710 +45 ☔ View full report in Codecov by Sentry. |
55d06cd
to
f915cc0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
❤️
go/cmd/vtctldclient/command/vreplication/materialize/materialize.go
Outdated
Show resolved
Hide resolved
create.Flags().BoolVar(&common.CreateOptions.StopAfterCopy, "stop-after-copy", false, "Stop the workflow after it's finished copying the existing rows and before it starts replicating changes.") | ||
create.Flags().StringVar(&common.CreateOptions.MySQLServerVersion, "mysql_server_version", fmt.Sprintf("%s-Vitess", config.DefaultMySQLVersion), "Configure the MySQL version to use for example for the parser.") | ||
create.Flags().IntVar(&common.CreateOptions.TruncateUILen, "sql-max-length-ui", 512, "truncate queries in debug UIs to the given length (default 512)") | ||
create.Flags().IntVar(&common.CreateOptions.TruncateErrLen, "sql-max-length-errors", 0, "truncate queries in error logs to the given length (default unlimited)") | ||
create.Flags().BoolVar(&common.CreateOptions.IsReference, "reference", false, "For specifying materialize of reference tables") | ||
create.Flags().StringSliceVarP(&common.CreateOptions.Tables, "tables", "t", nil, "Used with --reference to specify the tables to materialize") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
create.Flags().StringSliceVarP(&common.CreateOptions.Tables, "tables", "t", nil, "Used with --reference to specify the tables to materialize") | |
create.Flags().StringSliceVarP(&common.CreateOptions.Tables, "tables", "t", nil, "Used with --reference to specify the reference tables to materialize on every target shard.") |
a7c84d7
to
64802a5
Compare
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
…shards in a target keyspace Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
…tables. Self-review. Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
64802a5
to
f0101c5
Compare
Description
This PR adds support for materializing reference tables by adding options to the
Materialize
workflow. Currently the user needs to specify the target table info along with the materialize query using the--table_settings
flag which takes a json spec.Two new flags
--reference
(bool) and--tables
(csv list) have been added. Reference table materialization cannot be combined with regularMaterialize
workflows. It is one or the other. Reference table materializing just requires setting up a copy of the tables into all target shards.Example:
Materialize --target-keyspace ks2 --workflow wf1 create --source-keyspace ks1 --reference --tables ref1,ref2
Doc PR: vitessio/website#1849
Related Issue(s)
Checklist