From 7f0d88dc5e5d8a2cbef0081998cc43c443154fad Mon Sep 17 00:00:00 2001 From: Sumeet Varma Date: Tue, 26 Mar 2024 14:22:11 -0700 Subject: [PATCH] Spec changes for vacuum protocol check --- PROTOCOL.md | 23 +++++++++++++++++++ protocol_rfcs/README.md | 7 +++--- .../{ => accepted}/vacuum-protocol-check.md | 0 3 files changed, 26 insertions(+), 4 deletions(-) rename protocol_rfcs/{ => accepted}/vacuum-protocol-check.md (100%) diff --git a/PROTOCOL.md b/PROTOCOL.md index 6098ed4c4d6..6c81737175e 100644 --- a/PROTOCOL.md +++ b/PROTOCOL.md @@ -55,6 +55,9 @@ - [Row Commit Versions](#row-commit-versions) - [Reader Requirements for Row Tracking](#reader-requirements-for-row-tracking) - [Writer Requirements for Row Tracking](#writer-requirements-for-row-tracking) +- [VACUUM Protocol Check](#vacuum-protocol-check) + - [Writer Requirements for Vacuum Protocol Check](#writer-requirements-for-vacuum-protocol-check) + - [Recommendations for Readers of Tables with Vacuum Protocol Check feature](#recommendations-for-readers-of-tables-with-vacuum-protocol-check-feature) - [Clustered Table](#clustered-table) - [Writer Requirements for Clustered Table](#writer-requirements-for-clustered-table) - [Requirements for Writers](#requirements-for-writers) @@ -1179,6 +1182,26 @@ When Row Tracking is enabled (when the table property `delta.enableRowTracking` In particular, writers should set `delta.rowTracking.preserved` in the `tags` of the `commitInfo` action to `true` if no rows are updated or copied. Writers should set that flag to false otherwise. +# VACUUM Protocol Check + +The `vacuumProtocolCheck` ReaderWriter feature ensures consistent application of reader and writer protocol checks during `VACUUM` operations, addressing potential protocol discrepancies and mitigating the risk of data corruption due to skipped writer checks. + +Enablement: +- The table must be on Writer Version 7 and Reader Version 3. +- The feature `vacuumProtocolCheck` must exist in the table `protocol`'s `writerFeatures` and `readerFeatures`. + +## Writer Requirements for Vacuum Protocol Check + +This feature affects only the VACUUM operations; standard commits remain unaffected. + +Before performing a VACUUM operation, writers must ensure that they check the table's write protocol. This is most easily implemented by adding an unconditional write protocol check for all tables, which removes the need to examine individual table properties. + +Writers that do not implement VACUUM do not need to change anything and can safely write to tables that enable the feature. + +## Recommendations for Readers of Tables with Vacuum Protocol Check feature + +For tables with Vacuum Protocol Check enabled, readers don’t need to understand or change anything new; they just need to acknowledge the feature exists. + # Clustered Table The Clustered Table feature facilitates the physical clustering of rows that share similar values on a predefined set of clustering columns. diff --git a/protocol_rfcs/README.md b/protocol_rfcs/README.md index 1e2e7d4fed3..08088e16cf0 100644 --- a/protocol_rfcs/README.md +++ b/protocol_rfcs/README.md @@ -22,13 +22,12 @@ Here is the history of all the RFCs propose/accepted/rejected since Feb 6, 2024, | 2023-02-09 | [type-widening.md](https://github.com/delta-io/delta/blob/master/protocol_rfcs/type-widening.md) | https://github.com/delta-io/delta/issues/2623 | Type Widening | | 2023-02-14 | [managed-commits.md](https://github.com/delta-io/delta/blob/master/protocol_rfcs/managed-commits.md) | https://github.com/delta-io/delta/issues/2598 | Managed Commits | | 2023-02-26 | [column-mapping-usage.tracking.md](https://github.com/delta-io/delta/blob/master/protocol_rfcs/column-mapping-usage-tracking.md)) | https://github.com/delta-io/delta/issues/2682 | Column Mapping Usage Tracking | -| 2023-02-28 | [vacuum-protocol-check.md](https://github.com/delta-io/delta/blob/master/protocol_rfcs/vacuum-protocol-check.md) | https://github.com/delta-io/delta/issues/2630 | Enforce Vacuum Protocol Check | ### Accepted RFCs -| Date proposed | Date accepted | RFC file | Github issue | RFC title | -|:-|:-|:-|:-|:-| -|...||||| +| Date proposed | Date accepted | RFC file | Github issue | RFC title | +|:--------------|:--------------|:-|:----------------------------------------------|:-------------------------------| +| 2023-02-28 | 2023-03-26 |[vacuum-protocol-check.md](https://github.com/delta-io/delta/blob/master/protocol_rfcs/vacuum-protocol-check.md)| https://github.com/delta-io/delta/issues/2630 | Enforce Vacuum Protocol Check | ### Rejected RFCs diff --git a/protocol_rfcs/vacuum-protocol-check.md b/protocol_rfcs/accepted/vacuum-protocol-check.md similarity index 100% rename from protocol_rfcs/vacuum-protocol-check.md rename to protocol_rfcs/accepted/vacuum-protocol-check.md