Skip to content

Commit

Permalink
Update incremental insert overwrite docs (#331)
Browse files Browse the repository at this point in the history
* update README.md file to indicate that insert_overwrite is

* update CHANGELOG.md

* fix typo

* reframe the documentation
  • Loading branch information
BentsiLeviav authored Aug 21, 2024
1 parent 08ed52f commit 5ab6b00
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 2 deletions.
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
### Release XXXXX
#### Improvements
* Added a new *experimental* incremental strategy - `insert_overwrite` was added which replaces existing data in a target table partition by partition, ensuring only the specified partitions are overwritten with new data, which helps maintain performance and data consistency. This feature was not tested with distribution models and might not work with such materializations.


### Release [1.8.1], 2024-07-11
#### Bug Fix
* Refresh materialized_view table only if `--full-refresh` is specified.
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,8 +185,10 @@ This strategy replaces the `inserts_only` setting in previous versions of dbt-cl
As a result duplicate rows are not eliminated, and there is no temporary or intermediate table. It is the fastest approach if duplicates are either permitted
in the data or excluded by the incremental query WHERE clause/filter.

### The insert_overwrite Strategy

### The insert_overwrite Strategy (Experimental)
> [IMPORTANT]
> Currently, the insert_overwrite strategy is not fully functional with distributed materializations.
Performs the following steps:
1. Create a staging (temporary) table with the same structure as the incremental model relation: `CREATE TABLE <staging> AS <target>`.
2. Insert only new records (produced by `SELECT`) into the staging table.
Expand Down

0 comments on commit 5ab6b00

Please sign in to comment.