Releases: vitessio/vitess
Vitess v15.0.2
Release of Vitess v15.0.2
Major Changes
Upgrade to go1.18.9
Vitess v15.0.2 now runs on go1.18.9.
The patch release of Go, go1.18.9, was one of the main reasons for this release as it includes an important security fixe to net/http package, which is use extensively by Vitess.
Below is a summary of this patch release. You can learn more here.
go1.18.9 (released 2022-12-06) includes security fixes to the net/http and os packages, as well as bug fixes to cgo, the compiler, the runtime, and the crypto/x509 and os/exec packages.
The entire changelog for this release can be found here.
The release includes 14 commits (excluding merges)
Thanks to all our contributors: @GuptaManan100, @dbussink, @frouioui, @mattlord, @rsajwani, @shlomi-noach, @vitess-bot[bot]
Contributors
Assets 5
Vitess v15.0.1
Release of Vitess v15.0.1
Major Changes
Corrupted results for non-full-group-by queries with JOINs
An issue in versions <= v14.0.3 and <= v15.0.0 that generated corrupted results for non-full-group-by queries with a JOIN
is now fixed. The full issue can be found here, and its fix here.
VtAdmin web folder is missing while installing Vitess with local method
When we try to install Vitess locally (https://vitess.io/docs/15.0/get-started/local/#install-vitess) on v15.0, we are getting the following error
npm ERR! enoent ENOENT: no such file or directory, open '/home/web/vtadmin/package.json'
This issue is fixed in 15.0.1. The full issue can be found here, and its fix here.
The entire changelog for this release can be found here.
The release includes 25 commits (excluding merges)
Thanks to all our contributors: @GuptaManan100, @frouioui, @harshit-gangal, @rsajwani, @vitess-bot[bot]
Contributors
Assets 5
Vitess v14.0.4
Release of Vitess v14.0.4
Major Changes
Upgrade to go1.18.7
Vitess v14.0.4 now runs on go1.18.7.
The patch release of Go, go1.18.7, was one of the main reasons for this release as it includes important security fixes to packages used by Vitess.
Below is a summary of this patch release. You can learn more here.
go1.18.7 (released 2022-10-04) includes security fixes to the archive/tar, net/http/httputil, and regexp packages, as well as bug fixes to the compiler, the linker, and the go/types package.
Corrupted results for non-full-group-by queries with JOINs
An issue in versions <= v14.0.3 and <= v15.0.0 that generated corrupted results for non-full-group-by queries with a JOIN
is now fixed. The full issue can be found here, and its fix here.
The entire changelog for this release can be found here.
The release includes 24 commits (excluding merges)
Thanks to all our contributors: @GuptaManan100, @dbussink, @frouioui, @harshit-gangal, @systay, @vitess-bot[bot]
Contributors
Assets 5
Vitess v15.0.0
Release of Vitess v15.0.0
Summary
- Known Issues
- Breaking Changes
- Deprecations
- Command-Line Syntax Deprecations
- VTGate
- VDiff2
- New command line flags and behavior
- Online DDL Changes
- Tablet Throttler
- Mysql Compatibility
- Durability Policy
- New EXPLAIN Format
- VTOrc
- Flags Restructure
Known Issues
- Corrupted results for non-full-group-by queries with JOINs. This can be resolved by using full-group-by queries.
Major Changes
Breaking Changes
Flags
- The deprecated
--cpu_profileflag has been removed. Please use the--pprofflag instead. - The deprecated
--mem-profile-rateflag has been removed. Please use--pprof=meminstead. - The deprecated
--mutex-profile-fractionflag has been removed. Please use--pprof=mutexinstead. - The deprecated vtgate/vtexplain/vtcombo flag
--planner_versionhas been removed. Please use--planner-versioninstead. - The deprecated flag
--master_connect_retryhas been removed. Please use--replication_connect_retryinstead. vtctlcommands that take shard names and ranges as positional arguments (e.g.vtctl Reshard ks.workflow -80 -40,40-80) need to have their positional arguments separated from their flag arguments by a double-dash separator to avoid the new parsing library from mistaking them as flags (e.g.vtctl Reshard ks.workflow -- -80 -40,40-80).- The
--cellflag in thevtgatebinary no longer has a default value. It is a required argument that has to be specified for the binary to run. Please explicitly specify the flag, if dependent on the flag's default value. - The
--db-config-*-*VTTablet flags were deprecated inv3.0.0. They have now been deleted as part of this release. You must use--db_dba_*now.
vttablet Flag Deletions
The following VTTablet flags were deprecated in 7.0. They have now been deleted
--queryserver-config-message-conn-pool-size--queryserver-config-message-conn-pool-prefill-parallelism--client-found-rows-pool-sizeA different existing flag--queryserver-config-transaction-capwill be used instead--transaction_shutdown_grace_periodUse--shutdown_grace_periodinstead--queryserver-config-max-dml-rows--queryserver-config-allowunsafe-dmls--pool-name-prefix--enable-autocommitAutocommit is always allowed
Vindex Interface
All the vindex interface methods are changed by adding context.Context as an input parameter.
E.g:
Map(vcursor VCursor, .... ) ....
To
Map(ctx context.Context, vcursor VCursor, .... ) ....This only affects users who have added their own custom vindex implementation.
They are required to change their implementation with these new interface method expectations.
Deprecations
LogStats Table and Keyspace deprecated
Information about which tables are used was being reported through the Keyspace and Table fields on LogStats.
For multi-table queries, this output can be confusing, so we have added TablesUsed, that is a string array, listing all tables and which keyspace they are on.
Keyspace and Table fields are deprecated and will be removed in the v16 release of Vitess.
Orchestrator Integration Deprecation
Orchestrator integration in vttablet has been deprecated. It will continue to work in this release but is liable to be removed in future releases.
Consider using VTOrc instead of Orchestrator as VTOrc goes GA in this release.
Connection Pool Prefill
The connection pool with prefilled connections have been removed. The pool now does lazy connection creation.
InitShardPrimary Deprecation
The vtcltd command InitShardPrimary has been deprecated. Please use PlannedReparentShard instead.
Command-line syntax deprecations
vttablet startup flag deletions
The following VTTablet flags were deprecated in 7.0. They have now been deleted
- --queryserver-config-message-conn-pool-size
- --queryserver-config-message-conn-pool-prefill-parallelism
- --client-found-rows-pool-size --queryserver-config-transaction-cap will be used instead
- --transaction_shutdown_grace_period Use --shutdown_grace_period instead
- --queryserver-config-max-dml-rows
- --queryserver-config-allowunsafe-dmls
- --pool-name-prefix
- --enable-autocommit Autocommit is always allowed
vttablet startup flag deprecations
--enable-query-plan-field-cachinghas been deprecated. It will be removed in v16.--enable_semi_synchas been deprecated. It will be removed in v16. Instead, set the correct durability policy usingSetKeyspaceDurabilityPolicy--queryserver-config-pool-prefill-parallelism,--queryserver-config-stream-pool-prefill-parallelismand--queryserver-config-transaction-prefill-parallelismhave all been deprecated. They will be removed in v16.--backup_storage_hookhas been deprecated, consider using one of the builtin compression algorithms or--external-compressorand--external-decompressorinstead.
vtbackup flag deprecations
--backup_storage_hookhas been deprecated, consider using one of the builtin compression algorithms or--external-compressorand--external-decompressorinstead.
VTGate
vtgate --mysql-server-pool-conn-read-buffers
--mysql-server-pool-conn-read-buffers enables pooling of buffers used to read from incoming
connections, similar to the way pooling happens for write buffers. Defaults to off.
VDiff v2
Resume Workflow
We introduced the ability to resume a VDiff2 workflow:
$ vtctlclient --server=localhost:15999 VDiff --v2 customer.commerce2customer resume 4c664dc2-eba9-11ec-9ef7-920702940ee0
VDiff 4c664dc2-eba9-11ec-9ef7-920702940ee0 resumed on target shards, use show to view progress
$ vtctlclient --server=localhost:15999 VDiff --v2 customer.commerce2customer show last
VDiff Summary for customer.commerce2customer (4c664dc2-eba9-11ec-9ef7-920702940ee0)
State: completed
RowsCompared: 196
HasMismatch: false
StartedAt: 2022-06-26 22:44:29
CompletedAt: 2022-06-26 22:44:31
Use "--format=json" for more detailed output.
$ vtctlclient --server=localhost:15999 VDiff --v2 --format=json customer.commerce2customer show last
{
"Workflow": "commerce2customer",
"Keyspace": "customer",
"State": "completed",
"UUID": "4c664dc2-eba9-11ec-9ef7-920702940ee0",
"RowsCompared": 196,
"HasMismatch": false,
"Shards": "0",
"StartedAt": "2022-06-26 22:44:29",
"CompletedAt": "2022-06-26 22:44:31"
}
We leverage this resume capability to automatically restart a VDiff2 workflow that encountered a retryable error.
We also made a number of other enhancements like progress reporting and features needed to make it a full replacement for VDiff v1. You can see more details in the tracking ticket for the VDiff2 feature complete target: #10494
Now that VDiff v2 is feature complete in 15.0, we hope to make it GA in 16.0.
Please see the VDiff2 documentation ...
Contributors
Assets 5
Vitess v15.0.0-rc1
Release of Vitess v15.0.0-rc1
Summary
- Vindex Interface
- LogStats Table and Keyspace deprecated
- Command-line syntax deprecations
- New command line flags and behavior
- Online DDL changes
- Tablet throttler
- VDiff2
- Mysql Compatibility
- Durability Policy
- New EXPLAIN format
- VTOrc
Known Issues
Major Changes
Breaking Changes
Flags
- The deprecated
--cpu_profileflag has been removed. Please use the--pprofflag instead. - The deprecated
--mem-profile-rateflag has been removed. Please use--pprof=meminstead. - The deprecated
--mutex-profile-fractionflag has been removed. Please use--pprof=mutexinstead. - The deprecated vtgate/vtexplain/vtcombo flag
--planner_versionhas been removed. Please use--planner-versioninstead. - The deprecated flag
--master_connect_retryhas been removed. Please use--replication_connect_retryinstead. vtctlcommands that take shard names and ranges as positional arguments (e.g.vtctl Reshard ks.workflow -80 -40,40-80) need to have their positional arguments separated from their flag arguments by a double-dash separator to avoid the new parsing library from mistaking them as flags (e.g.vtctl Reshard ks.workflow -- -80 -40,40-80).
Vindex Interface
All the vindex interface methods are changed by adding context.Context as an input parameter.
E.g:
Map(vcursor VCursor, .... ) ....
To
Map(ctx context.Context, vcursor VCursor, .... ) ....This only affects users who have added their own custom vindex implementation.
They are required to change their implementation with these new interface method expectations.
LogStats Table and Keyspace deprecated
Information about which tables are used was being reported through the Keyspace and Table fields on LogStats.
For multi-table queries, this output can be confusing, so we have added TablesUsed, that is a string array, listing all tables and which keyspace they are on.
Keyspace and Table fields are deprecated and will be removed in the v16 release of Vitess.
Orchestrator Integration Deprecation
Orchestrator integration in vttablet has been deprecated. It will continue to work in this release but is liable to be removed in future releases.
Consider using VTOrc instead of Orchestrator as VTOrc goes GA in this release.
Connection Pool Prefill
The connection pool with prefilled connections have been removed. The pool now does lazy connection creation.
Following flags are deprecated: queryserver-config-pool-prefill-parallelism, queryserver-config-stream-pool-prefill-parallelism, queryserver-config-transaction-prefill-parallelism
and will be removed in future version.
Command-line syntax deprecations
vttablet startup flag deletions
The following VTTablet flags were deprecated in 7.0. They have now been deleted
- --queryserver-config-message-conn-pool-size
- --queryserver-config-message-conn-pool-prefill-parallelism
- --client-found-rows-pool-size --queryserver-config-transaction-cap will be used instead
- --transaction_shutdown_grace_period Use --shutdown_grace_period instead
- --queryserver-config-max-dml-rows
- --queryserver-config-allowunsafe-dmls
- --pool-name-prefix
- --enable-autocommit Autocommit is always allowed
vttablet startup flag deprecations
- --enable-query-plan-field-caching is now deprecated. It will be removed in v16.
- --enable_semi_sync is now deprecated. It will be removed in v16. Instead, set the correct durability policy using
SetKeyspaceDurabilityPolicy - --queryserver-config-pool-prefill-parallelism, --queryserver-config-stream-pool-prefill-parallelism and --queryserver-config-transaction-prefill-parallelism have all been deprecated. They will be removed in v16.
New command line flags and behavior
vtgate --mysql-server-pool-conn-read-buffers
--mysql-server-pool-conn-read-buffers enables pooling of buffers used to read from incoming
connections, similar to the way pooling happens for write buffers. Defaults to off.
VDiff2
We introduced the ability to resume a VDiff2 workflow:
$ vtctlclient --server=localhost:15999 VDiff --v2 customer.commerce2customer resume 4c664dc2-eba9-11ec-9ef7-920702940ee0
VDiff 4c664dc2-eba9-11ec-9ef7-920702940ee0 resumed on target shards, use show to view progress
$ vtctlclient --server=localhost:15999 VDiff --v2 customer.commerce2customer show last
VDiff Summary for customer.commerce2customer (4c664dc2-eba9-11ec-9ef7-920702940ee0)
State: completed
RowsCompared: 196
HasMismatch: false
StartedAt: 2022-06-26 22:44:29
CompletedAt: 2022-06-26 22:44:31
Use "--format=json" for more detailed output.
$ vtctlclient --server=localhost:15999 VDiff --v2 --format=json customer.commerce2customer show last
{
"Workflow": "commerce2customer",
"Keyspace": "customer",
"State": "completed",
"UUID": "4c664dc2-eba9-11ec-9ef7-920702940ee0",
"RowsCompared": 196,
"HasMismatch": false,
"Shards": "0",
"StartedAt": "2022-06-26 22:44:29",
"CompletedAt": "2022-06-26 22:44:31"
}
Please see the VDiff2 documentation for additional information.
New command line flags and behavior
vtctl GetSchema --table-schema-only
The new flag --table-schema-only skips column introspection. GetSchema only returns general schema analysis, and specifically it includes the CREATE TABLE|VIEW statement in the schema field.
Support for additional compressors and decompressors during backup & restore
Backup/Restore now allow you many more options for compression and decompression instead of relying on the default compressor(pgzip).
There are some built-in compressors which you can use out-of-the-box. Users will need to evaluate which option works best for their
use-case. Here are the flags that control this feature
- --compression-engine-name
- --external-compressor
- --external-decompressor
- --external-compressor-extension
- --compression-level
--compression-engine-name specifies the engine used for compression. It can have one of the following values
- pargzip (Default)
- pgzip
- lz4
- zstd
- external
where 'external' is set only when using a custom command or tool other than the ones that are already provided.
If you want to use any of the built-in compressors, simply set one of the above values for --compression-engine-name. The value
specified in --compression-engine-name is saved in the backup MANIFEST, which is later read by the restore process to decide which
engine to use for decompression. Default value for engine is 'pgzip'.
If you would like to use a custom command or external tool for compression/decompression then you need to provide the full command with
arguments to the --external-compressor and --external-decompressor flags. --external-compressor-extension flag also needs to be provided
so that compressed files are created with the correct extension. If the external command is not using any of the built-in compression engines
(i-e pgzip, pargzip, lz4 or zstd) then you need to set --compression-engine-name to value 'external'.
Please note that if you want the current production behavior then you don't need to change any of these flags.
You can read more about backup & restore [here] (https://vitess.io/docs/15.0/user-guides/operating-vitess/backup-and-restore/).
If you decided to switch from an external compressor to one of the built-in supported compressors (i-e pgzip, pargzip, lz4 or zstd) at any point
in the future, you will need to do it in two steps.
- step #1, set
--external-compressorand--external-compressor-extensionflag values to empty and change--compression-engine-nameto desired value. - Step #2, after at least one cycle of backup with new configuration, you can set
--external-decompressorflag value to empty.
The reason you cannot change all the values together is because the restore process will then have no way to find out which external decompressor
should be used to process the previous backup. Please make sure you have thought out all possible scenarios for restore before transitioning from one
compression engine to another.
Independent OLAP and OLTP transactional timeouts
--queryserver-config-olap-transaction-timeout specifies the timeout applied
to a transaction created within an OLAP workload. The default value is 30
seconds, but this can be raised, lowered, or set to zero to disable the timeout
altogether.
Until now, while OLAP queries would bypass the query timeout, transactions
created within an OLAP session would be rolled back
--queryserver-config-transaction-timeout seconds after the transaction was
started.
As of now, OLTP and OLAP transaction timeouts can be configured independently of each
other.
The main use case is to run queries spanning a long period of time which
require transactional guarantees such as consistency or atomicity.
Support for specifying group information in calls to VTGate
--grpc-use-effective-groups allows non-SSL callers to specify groups information for a caller.
Until now, you could only specify the caller-id for the security context used to authorize queries.
As of now, you can specify the principal of the caller, and any groups they belong to.
Online DDL changes
Concur...
Contributors
Assets 5
Vitess v14.0.3
Release of Vitess v14.0.3
Known Issues
- Corrupted results for non-full-group-by queries with JOINs. This can be resolved by using full-group-by queries.
Major Changes
Fix VTOrc Discovery
In patch releases prior to release 14.0.3, if VTOrc is unable to reach the MySQL instance of a vttablet, it was never able to read discover that tablet again.
This problem could be resolved by restarting the VTOrc so that it discovers all the tablets again, but in a kubernetes cluster where the pods are eviced
frequently, this posed a greater challenge, since some pods when evicted and rescheduled on a different node, would sometimes fail to be discovered by VTOrc.
This has problem has been addressed in this patch by the fix #10662.
The entire changelog for this release can be found here.
The release includes 12 commits (excluding merges)
Thanks to all our contributors: @GuptaManan100, @frouioui, @harshit-gangal, @mattlord, @vitess-bot[bot]
Contributors
Assets 5
Vitess v14.0.2
Release of Vitess v14.0.2
Known Issues
- Corrupted results for non-full-group-by queries with JOINs. This can be resolved by using full-group-by queries.
Major Changes
Upgrade to go1.18.5
Vitess v14.0.2 now runs on go1.18.5.
The patch release of Go, go1.18.5, was one of the main reasons for this release as it includes important security fixes to packages used by Vitess.
Below is a summary of this patch release. You can learn more here.
go1.18.4 (released 2022-07-12) includes security fixes to the compress/gzip, encoding/gob, encoding/xml, go/parser, io/fs, net/http, and path/filepath packages, as well as bug fixes to the compiler, the go command, the linker, the runtime, and the runtime/metrics package.
End of life of MariadDB 10.2
Since the end-of-life of MariaDB 10.2, its Docker image is unavailable, and we decided to remove the unit tests using this version of MariaDB. The Pull Request doing this change is available here.
You can find more information on the list of supported databases on our documentation website, here.
The entire changelog for this release can be found here.
The release includes 23 commits (excluding merges)
Thanks to all our contributors: @GuptaManan100, @dbussink, @frouioui, @mattlord, @shlomi-noach, @vitess-bot[bot]
Contributors
Assets 5
Vitess v13.0.3
Release of Vitess v13.0.3
Major Changes
Upgrade to go1.17.13
Vitess v13.0.3 now runs on go1.17.13.
The patch release of Go, go1.17.13, was one of the main reasons for this release as it includes important security fixes to packages used by Vitess.
Below is a summary of this patch release. You can learn more here.
go1.17.13 (released 2022-08-01) includes security fixes to the encoding/gob and math/big packages, as well as bug fixes to the compiler and the runtime.
End of life of MariadDB 10.2
Since the end-of-life of MariaDB 10.2, its Docker image is unavailable, and we decided to remove the unit tests using this version of MariaDB. The Pull Request doing this change is available here.
This change is documented on our website here.
The entire changelog for this release can be found here.
The release includes 17 commits(excluding merges)
Thanks to all our contributors: @GuptaManan100, @dbussink, @frouioui, @harshit-gangal, @mattlord, @rohit-nayak-ps, @vitess-bot[bot]
Contributors
Assets 5
Vitess v12.0.6
Release of Vitess v12.0.6
Major Changes
Upgrade to go1.17.13
Vitess v12.0.6 now runs on go1.17.13.
The patch release of Go, go1.17.13, was the main reason for this release as it includes important security fixes to packages used by Vitess.
Below is a summary of this patch release. You can learn more here.
go1.17.13 (released 2022-08-01) includes security fixes to the encoding/gob and math/big packages, as well as bug fixes to the compiler and the runtime.
End of life of MariadDB 10.2
Since the end-of-life of MariaDB 10.2, its Docker image is unavailable, and we decided to remove the unit tests using this version of MariaDB. The Pull Request doing this change is available here.
This change is documented on our website here.
The entire changelog for this release can be found here.
The release includes 11 commits (excluding merges)
Thanks to all our contributors: @GuptaManan100, @dbussink, @deepthi, @frouioui, @harshit-gangal, @rohit-nayak-ps
Contributors
Assets 2
Vitess v12.0.5
Release of Vitess v12.0.5
Major Changes
Upgrade to go1.17.12
Vitess v12.0.5 now runs on go1.17.12.
The patch release of Go, go1.17.12, is the reason for this release as it includes important security fixes to packages used by Vitess.
Below is a summary of this patch release. You can learn more here.
go1.17.12 (released 2022-07-12) includes security fixes to the compress/gzip, encoding/gob, encoding/xml, go/parser, io/fs, net/http, and path/filepath packages, as well as bug fixes to the compiler, the go command, the runtime, and the runtime/metrics package. See the Go 1.17.12 milestone on our issue tracker for details.
The entire changelog for this release can be found here.
The release includes 7 commits (excluding merges)