Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CT-10785] Microbatch models should respect full_refresh model config #10788

Merged
merged 3 commits into from
Sep 26, 2024
Merged

[CT-10785] Microbatch models should respect full_refresh model config #10788

merged 3 commits into from
Sep 26, 2024

Commits on Sep 26, 2024

  1. Add tests to check how microbatch models respect full_refresh model… 

    … configs
    @QMalcolm
    QMalcolm committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    4bdbfbf View commit details
    Browse the repository at this point in the history

  2. Fix _is_incremental to properly respect full_refresh model config 

    In dbt-core, it is generally expected that values passed via CLI flags take
precedence over model level configs. However, `full_refresh` on a model is an
exception to this rule, where in the model config takes precedence. This
config exists specifically to _prevent_ accidental full refreshes of large
incremental models, as doing so can be costly. **_It is actually best
practice_** to set `full_refresh=False` on incremental models.

Prior to this commit, for microbatch models, the above was not happening. The
CLI flag `--full-refresh` was taking precedence over the model config
`full_refresh`. That meant that if `--full-refresh` was supplied, then the
microbatch model **_would full refresh_** even if `full_refresh=False` was
set on the model. This commit solves that problem.
    @QMalcolm
    QMalcolm committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    7fd546e View commit details
    Browse the repository at this point in the history

  3. Add changie doc for microbatch full_refresh config handling

    @QMalcolm
    QMalcolm committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    4d5381e View commit details
    Browse the repository at this point in the history