Skip to content

Commit

Permalink
Update where.md to enhance the usability of the example
Browse files Browse the repository at this point in the history
The example on the page is useful; however, it is rigid in only allowing a 3 day look back. Adjusting the pattern to include a number value and a macro that performs a regex search to set the dateadd logic to whatever value is configured in the test. This allows for reusbaility of the macro for different lookback days.
  • Loading branch information
brian-franklin authored Oct 31, 2024
1 parent b7d1f96 commit 07fef64
Showing 1 changed file with 19 additions and 5 deletions.
24 changes: 19 additions & 5 deletions website/docs/reference/resource-configs/where.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ You can override this behavior by:

Within this macro definition, you can reference whatever custom macros you want, based on static inputs from the configuration. At simplest, this enables you to DRY up code that you'd otherwise need to repeat across many different `.yml` files. Because the `get_where_subquery` macro is resolved at runtime, your custom macros can also include [fetching the results of introspective database queries](https://docs.getdbt.com/reference/dbt-jinja-functions/run_query).

**Example:** Filter your test to the past three days of data, using dbt's cross-platform [`dateadd()`](https://docs.getdbt.com/reference/dbt-jinja-functions/cross-database-macros#dateadd) utility macro.
**Example:** Filter your test to the past N days of data, using dbt's cross-platform [`dateadd()`](https://docs.getdbt.com/reference/dbt-jinja-functions/cross-database-macros#dateadd) utility macro. You control how many days ago by setting the number in the placeholder string.

<File name='models/config.yml'>

Expand All @@ -147,7 +147,7 @@ models:
tests:
- unique:
config:
where: "date_column > __three_days_ago__" # placeholder string for static config
where: "date_column > __3_days_ago__" # placeholder string for static config
```

</File>
Expand All @@ -158,10 +158,9 @@ models:
{% macro get_where_subquery(relation) -%}
{% set where = config.get('where') %}
{% if where %}
{% if "__three_days_ago__" in where %}
{% if "_days_ago__" in where %}
{# replace placeholder string with result of custom macro #}
{% set three_days_ago = dbt.dateadd('day', -3, current_timestamp()) %}
{% set where = where | replace("__three_days_ago__", three_days_ago) %}
{% set where = replace_days_ago(where) %}
{% endif %}
{%- set filtered -%}
(select * from {{ relation }} where {{ where }}) dbt_subquery
Expand All @@ -171,6 +170,21 @@ models:
{% do return(relation) %}
{%- endif -%}
{%- endmacro %}
{% macro replace_days_ago(where_string) %}
{# Use regex to search the pattern for the number days #}
{# Default to 3 days when no number found #}
{% set re = modules.re %}
{% set days = 3 %}
{% set pattern = '__(\d+)_days_ago__' %}
{% set match = re.search(pattern, where_string) %}
{% if match %}
{% set days = match.group(1) | int %}
{% endif %}
{% set n_days_ago = dbt.dateadd('day', -days, current_timestamp()) %}
{% set result = re.sub(pattern, n_days_ago, where_string) %}
{{ return(result) }}
{% endmacro %}
```

</File>

0 comments on commit 07fef64

Please sign in to comment.