Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add dbt_valid_to_current #6308

Open
wants to merge 12 commits into
base: current
Choose a base branch
from
Open

add dbt_valid_to_current #6308

wants to merge 12 commits into from

Conversation

mirnawong1
Copy link
Contributor

@mirnawong1 mirnawong1 commented Oct 17, 2024

adding new config for dbt_valid_to_current, which allows users to set a future date for dbt_valid_to using the dbt_valid_to_current config in yaml/config/project.

Resolves #6275

Outstanding questions:

  • Can date be hardcoded using this syntax? "to_date('2024-05-10')" or can they use "to_date('2024, 05, 10')" ?
  • is the syntax just a string (e.g. dbt_valid_to_current: '9999, 12, 31' ) or must they use to_date?
  • can user use var to return a SQL statement '{{ var('my_future_date') }}'
  • can they use a macro that returns a SQL statement '{{ dbt.date(9999, 12, 31) }}'?
  • how does this new config work with deferral/state:modified? will we warn users that the config has been updated and they need to manually update their snapshot?

🚀 Deployment available! Here are the direct links to the updated files:

@mirnawong1 mirnawong1 requested a review from a team as a code owner October 17, 2024 15:07
Copy link

vercel bot commented Oct 17, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
docs-getdbt-com ✅ Ready (Inspect) Visit Preview Nov 4, 2024 4:41pm

@github-actions github-actions bot added content Improvements or additions to content Docs team Authored by the Docs team @dbt Labs size: medium This change will take up to a week to address labels Oct 17, 2024
@dbeatty10
Copy link
Contributor

@mirnawong1 grouping your outstanding questions into a handful of categories:

Hardcoded SQL expressions

Question:

  • Can date be hardcoded using this syntax? "to_date('2024-05-10')" or can they use "to_date('2024, 05, 10')" ?

👍 Hardcoded expressions that work within the data platform is the only option that is available currently. The precise syntax will vary per warehouse, so to_date('2024-05-10') may work for some, but others may require something like date(2024, 5, 10).

Jinja within dbt_valid_to_current

Questions:

  • is the syntax just a string (e.g. dbt_valid_to_current: '9999, 12, 31' ) or must they use to_date?
  • can user use var to return a SQL statement '{{ var('my_future_date') }}'
  • can they use a macro that returns a SQL statement '{{ dbt.date(9999, 12, 31) }}'?

👎 Jinja is not currently available for dbt_valid_to_current. So none of the above are possible, and the only option is to supply a full SQL expression that is specific to the users data warehouse.

Deferral / state:modified

Question:

  • how does this new config work with deferral/state:modified? will we warn users that the config has been updated and they need to manually update their snapshot?

👍 Based on hands-on experimentation, it does work as expected with deferral / state:modified. So if this config changes, than it will show up when using --select state:modified.

@dbeatty10 dbeatty10 added the blocked_by_dev Awaiting merge of PR with associated functionality label Oct 28, 2024
@dbeatty10
Copy link
Contributor

@mirnawong1 I added the blocked_by_dev label until this is resolved:

@mirnawong1
Copy link
Contributor Author

@mirnawong1 I added the blocked_by_dev label until this is resolved:

thank you @dbeatty10 ! it looks like this is resolved now right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked_by_dev Awaiting merge of PR with associated functionality content Improvements or additions to content Docs team Authored by the Docs team @dbt Labs size: medium This change will take up to a week to address
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Core] Allow custom date for dbt_valid_to in snapshots
2 participants