Skip to content
You're viewing an older version of this GitHub Action. Do you want to see the latest version instead?
play

GitHub Action

Run checks against DBT Models

v2.2.0-alpha.1 Pre-release

Run checks against DBT Models

play

Run checks against DBT Models

Compares columns and rows in BigQuery

Installation

Copy and paste the following snippet into your .yml file.

 
              
- name: Run checks against DBT Models

              
  uses: org-not-included/dbt_table_diff@v2.2.0-alpha.1
Learn more about this action in org-not-included/dbt_table_diff

Choose a version

Quick Start:

This example Workflow File shows how to configure the Github Action via Github Actions Inputs.
This example Pull Request shows the output of the Github Action.
Screen Shot 2022-08-11 at 3 42 04 PM

Github Actions Input Arguments:

Input Parameter Description
GCP_TOKEN for connecting to BQ (runs dbt compile and dbt_table_diff/sql_checks to compare tables)
GH_TOKEN for connecting to Github (ie. fetches modified models/*.sql in your PR, adds comment on your PR)
PR_NUMBER for fetching open PR from github (Pull Request ID [int])
GH_REPO for fetching open PR from github (Repository Name)
GH_ORG for fetching open PR from github (Repository owner/organization name)
DBT_PROFILE_FILE the local path in your repo to your profile.yml for dbt (this is necessary for compiling manifest.json during setup process)
dev_prefix the prefix used when running dbt locally (Your source schema/environment for comparison)
prod_prefix the prefix used when running dbt remotely (Your target schema/environment for comparison)
fallback_prefix useful if you have an overriden macro for generate_schema_name in your dbt project, which leverages a different prefix for some schemas in prod.
irregular_schemas comma separated string of schemas which use fallback_prefix
project_id for connecting to BQ (BigQuery Project ID)
ignored_schemas comma separated string of schemas to ignore (skip checking during github action)

Description:

This Github Action is intended for comparing models in dbt that have changed during an open PR.
It only currently supports BigQuery.

Step-By-Step Break Down of Process:

  • Fetches list of files modified in Pull Request
    • by CURLing github.api.com/repos/{organization}/{repository}/pulls/{pull_request_id}/files
  • Filters on relevant_files
    • which are files matching models/*.sql
  • Builds manifest.json
    • By running dbt deps; dbt compile
  • Parses manifest.json for relevant_models
    • using manifest-attribute original_file_path matching relevant_files
  • Runs all SQL files in dbt_table_diff/sql_checks
    • for each of the relevant_models, compare the two dbt targets (dev_prefix vs prod_prefix)
  • Saves output to file
    • in a format supported by Github comments
  • Posts comment on open PR
    • leveraging py-github-helper PyPi package