Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core] Update docs on dbt-databricks constraints #6250

Open
togobingi opened this issue Oct 5, 2024 · 2 comments
Open

[Core] Update docs on dbt-databricks constraints #6250

togobingi opened this issue Oct 5, 2024 · 2 comments
Labels
content Improvements or additions to content dbt Core The changes proposed in this issue relate to dbt Core

Comments

@togobingi
Copy link

Link to the page(s) on docs.getdbt.com requiring updates

https://docs.getdbt.com/reference/resource-properties/constraints#platform-specific-support

Tell us more about this update

The docs says: "As Databricks does not support transactions nor allows using create or replace table with a column schema, the table is first created without a schema, and alter statements are then executed to add the different constraints."

However, you can create and replace tables with a column schema as shown in the Databricks docs: https://docs.databricks.com/en/tables/constraints.html#set-a-not-null-constraint-in-databricks

Implementing this approach would allow dbt to enforce constraints in Databricks by ensuring 'bad' data never gets inserted into the tables in the first place, which isn't exactly possible now.

Reviewers/Stakeholders/SMEs

@mirnawong1

Related GitHub issues

No response

Additional information

No response

@togobingi togobingi added content Improvements or additions to content dbt Core The changes proposed in this issue relate to dbt Core labels Oct 5, 2024
@mirnawong1
Copy link
Contributor

thanks for opening this up @togobingi and appreciate the detailed description! I'm verifying internally and will get back to you here!

@mirnawong1
Copy link
Contributor

hey @togobingi , thanks so much for opening this issue up! can you say more about how you came to this conclusion? I've just checked internally with the team and it looks like dbt-databricks doesn't currently support create/replace table with constraints, but it's on the backlog as a high priority.

if you can clarify, i can work with the team to understand this more. many thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content Improvements or additions to content dbt Core The changes proposed in this issue relate to dbt Core
Projects
None yet
Development

No branches or pull requests

2 participants