From 85ef0e8e359a1f5577277059d86fbf10ec2d9b41 Mon Sep 17 00:00:00 2001 From: Anders Swanson Date: Wed, 19 Jul 2023 12:59:52 -0400 Subject: [PATCH] snowflake dynamic tables: first draft --- .../resource-configs/snowflake-configs.md | 67 ++++++++++++++++++- 1 file changed, 66 insertions(+), 1 deletion(-) diff --git a/website/docs/reference/resource-configs/snowflake-configs.md b/website/docs/reference/resource-configs/snowflake-configs.md index 42ee3635089..154f8795eea 100644 --- a/website/docs/reference/resource-configs/snowflake-configs.md +++ b/website/docs/reference/resource-configs/snowflake-configs.md @@ -77,7 +77,7 @@ select ... ``` -In this example, you can set up a query tag to be applied to every query with the model's name. +In this example, you can set up a query tag to be applied to every query with the model's name. ```sql @@ -341,3 +341,68 @@ In the configuration format for the model SQL file: + + + +## Dynamic Tables + +[Dynamic Tables](https://docs.snowflake.com/en/user-guide/dynamic-tables-about) are Snowflake's flavor of Materialized Views. The `CREATE DYNAMIC TABLE` ([docs](https://docs.snowflake.com/en/sql-reference/sql/create-dynamic-table)) statement requires the following parameters `TARGET_LAG` and `WAREHOUSE`, so a dbt-snowflake user must also provide these. + +You can create a dynamic table by editing _one_ of these files: + +- the SQL file for your model +- the `dbt_project.yml` configuration file + +The following examples create a dynamic table: + + + +```sql +{{ + config( + materialized = 'dynamic_table', + warehouse = 'MY_WAREHOUSE', + target_lag = '10 min', + ) +}} +``` + + + + + +```yaml +models: + path: + materialized: dynamic_table + warehouse: MY_WAREHOUSE + target_lag: '10 min' +``` + + + +### Limitations + +#### Changing materialization to and from "dynamic_table" + +Swapping an already materialized model to be a dynamic table and vise versa. The workaround is the manually drop the existing materialization in the data warehouse before calling `dbt run` again. + +To illustrate, assume for the example model below, `my_model`, that it has already been materialized to the underlying data platform via `dbt run`. If a user then changes the model's config to be `materialized="dynamic_table"`, they will get an error. The workaround is to execute `DROP TABLE my_model` on the data warehouse before trying the model again. + + + +```yaml + +{{ config( + materialized="table" # or any model type eg view, incremental +) }} + +``` + + + +#### Altering `target_lag` or `warehouse` + +Currently, changing either the `target_lag` or `warehouse` parameters for a pre-existing Dynamic Table (DT) will result in the Dynamic Table being dropped and replaced, rather than altered. + +