Skip to content

Commit

Permalink
snowflake dynamic tables: first draft
Browse files Browse the repository at this point in the history
  • Loading branch information
dataders committed Jul 19, 2023
1 parent d84a897 commit 85ef0e8
Showing 1 changed file with 66 additions and 1 deletion.
67 changes: 66 additions & 1 deletion website/docs/reference/resource-configs/snowflake-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ select ...
```

In this example, you can set up a query tag to be applied to every query with the model's name.
In this example, you can set up a query tag to be applied to every query with the model's name.

```sql
Expand Down Expand Up @@ -341,3 +341,68 @@ In the configuration format for the model SQL file:
</File>

</VersionBlock>

<VersionBlock firstVersion="1.6">

## Dynamic Tables

[Dynamic Tables](https://docs.snowflake.com/en/user-guide/dynamic-tables-about) are Snowflake's flavor of Materialized Views. The `CREATE DYNAMIC TABLE` ([docs](https://docs.snowflake.com/en/sql-reference/sql/create-dynamic-table)) statement requires the following parameters `TARGET_LAG` and `WAREHOUSE`, so a dbt-snowflake user must also provide these.

You can create a dynamic table by editing _one_ of these files:

- the SQL file for your model
- the `dbt_project.yml` configuration file

The following examples create a dynamic table:

<File name='models/YOUR_MODEL_NAME.sql'>

```sql
{{
config(
materialized = 'dynamic_table',
warehouse = 'MY_WAREHOUSE',
target_lag = '10 min',
)
}}
```

</File>

<File name='dbt_project.yml'>

```yaml
models:
path:
materialized: dynamic_table
warehouse: MY_WAREHOUSE
target_lag: '10 min'
```

</File>

### Limitations

#### Changing materialization to and from "dynamic_table"

Swapping an already materialized model to be a dynamic table and vise versa. The workaround is the manually drop the existing materialization in the data warehouse before calling `dbt run` again.

To illustrate, assume for the example model below, `my_model`, that it has already been materialized to the underlying data platform via `dbt run`. If a user then changes the model's config to be `materialized="dynamic_table"`, they will get an error. The workaround is to execute `DROP TABLE my_model` on the data warehouse before trying the model again.

<File name='my_model.sql'>

```yaml
{{ config(
materialized="table" # or any model type eg view, incremental
) }}
```

</File>

#### Altering `target_lag` or `warehouse`

Currently, changing either the `target_lag` or `warehouse` parameters for a pre-existing Dynamic Table (DT) will result in the Dynamic Table being dropped and replaced, rather than altered.

</VersionBlock>

0 comments on commit 85ef0e8

Please sign in to comment.