Skip to content

Commit

Permalink
Spec changes for vacuum protocol check
Browse files Browse the repository at this point in the history
  • Loading branch information
sumeet-db committed Apr 1, 2024
1 parent 537ed8e commit 7f0d88d
Show file tree
Hide file tree
Showing 3 changed files with 26 additions and 4 deletions.
23 changes: 23 additions & 0 deletions PROTOCOL.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,9 @@
- [Row Commit Versions](#row-commit-versions)
- [Reader Requirements for Row Tracking](#reader-requirements-for-row-tracking)
- [Writer Requirements for Row Tracking](#writer-requirements-for-row-tracking)
- [VACUUM Protocol Check](#vacuum-protocol-check)
- [Writer Requirements for Vacuum Protocol Check](#writer-requirements-for-vacuum-protocol-check)
- [Recommendations for Readers of Tables with Vacuum Protocol Check feature](#recommendations-for-readers-of-tables-with-vacuum-protocol-check-feature)
- [Clustered Table](#clustered-table)
- [Writer Requirements for Clustered Table](#writer-requirements-for-clustered-table)
- [Requirements for Writers](#requirements-for-writers)
Expand Down Expand Up @@ -1179,6 +1182,26 @@ When Row Tracking is enabled (when the table property `delta.enableRowTracking`
In particular, writers should set `delta.rowTracking.preserved` in the `tags` of the `commitInfo` action to `true` if no rows are updated or copied.
Writers should set that flag to false otherwise.

# VACUUM Protocol Check

The `vacuumProtocolCheck` ReaderWriter feature ensures consistent application of reader and writer protocol checks during `VACUUM` operations, addressing potential protocol discrepancies and mitigating the risk of data corruption due to skipped writer checks.

Enablement:
- The table must be on Writer Version 7 and Reader Version 3.
- The feature `vacuumProtocolCheck` must exist in the table `protocol`'s `writerFeatures` and `readerFeatures`.

## Writer Requirements for Vacuum Protocol Check

This feature affects only the VACUUM operations; standard commits remain unaffected.

Before performing a VACUUM operation, writers must ensure that they check the table's write protocol. This is most easily implemented by adding an unconditional write protocol check for all tables, which removes the need to examine individual table properties.

Writers that do not implement VACUUM do not need to change anything and can safely write to tables that enable the feature.

## Recommendations for Readers of Tables with Vacuum Protocol Check feature

For tables with Vacuum Protocol Check enabled, readers don’t need to understand or change anything new; they just need to acknowledge the feature exists.

# Clustered Table

The Clustered Table feature facilitates the physical clustering of rows that share similar values on a predefined set of clustering columns.
Expand Down
7 changes: 3 additions & 4 deletions protocol_rfcs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,12 @@ Here is the history of all the RFCs propose/accepted/rejected since Feb 6, 2024,
| 2023-02-09 | [type-widening.md](https://github.com/delta-io/delta/blob/master/protocol_rfcs/type-widening.md) | https://github.com/delta-io/delta/issues/2623 | Type Widening |
| 2023-02-14 | [managed-commits.md](https://github.com/delta-io/delta/blob/master/protocol_rfcs/managed-commits.md) | https://github.com/delta-io/delta/issues/2598 | Managed Commits |
| 2023-02-26 | [column-mapping-usage.tracking.md](https://github.com/delta-io/delta/blob/master/protocol_rfcs/column-mapping-usage-tracking.md)) | https://github.com/delta-io/delta/issues/2682 | Column Mapping Usage Tracking |
| 2023-02-28 | [vacuum-protocol-check.md](https://github.com/delta-io/delta/blob/master/protocol_rfcs/vacuum-protocol-check.md) | https://github.com/delta-io/delta/issues/2630 | Enforce Vacuum Protocol Check |

### Accepted RFCs

| Date proposed | Date accepted | RFC file | Github issue | RFC title |
|:-|:-|:-|:-|:-|
|...|||||
| Date proposed | Date accepted | RFC file | Github issue | RFC title |
|:--------------|:--------------|:-|:----------------------------------------------|:-------------------------------|
| 2023-02-28 | 2023-03-26 |[vacuum-protocol-check.md](https://github.com/delta-io/delta/blob/master/protocol_rfcs/vacuum-protocol-check.md)| https://github.com/delta-io/delta/issues/2630 | Enforce Vacuum Protocol Check |

### Rejected RFCs

Expand Down
File renamed without changes.

0 comments on commit 7f0d88d

Please sign in to comment.