dbt-testing-examples

This repository contains examples of how to implement real unit tests for a dbt data application by using the dbt-unit-testing package. You can also check the articles I wrote on testing data pipelines and data products with dbt

• How to Improve The Code Quality of your Dbt Models with Unit Tests and TDD.
• A Complete Guide to Effectively Scale your Data Pipelines and Data Products with Contract Testing and dbt.

Features

• Unit test and mocking examples with the dbt-unit-testing package
• Katas to get started unit testing models
• Component test examples with the dbt-unit-testing package
• Sources contract test examples with the dbt-expectations package
• Model contracts example with dbt 1.5
• CI/CD pipeline example with Github Actions

Roadmap

• Add unit test examples
• Add katas and improve README
• Add component test examples
• Add support for Github Codespaces
• Add contract test examples
• Add data quality test examples

Sample dbt app

Our sample dbt app is called health-insights. It takes weight and height data from upstream sources and calculates the metric body mass index. It follows the usual layered architecture commonly found in dbt projects.

Types of tests

The repository contains examples for both unit and component tests.

Katas

The repo contains two katas (small exercises) to learn how to implement unit tests for dbt models with the dbt-unit-testing package:

• Kata 1 - Adding support for imperial units (Solution)
• Kata 2 - Categorize body mass index (BMI) following WHO guidelines

A code kata is a software development exercise in which the focus is not on solving a task or problem, but on learning new skills and developing successful routines.

Local setup

Install dbt following the official documentation

Spin up a Postgres database in a container

docker-compose up

Setup the environment variables

cp .env.example .env

Setup the dbt profile

cp profiles.example.yml profiles.yml

Install dbt dependencies

dbt deps

Check that everything works

dbt debug

Seed the database

dbt seed

Running the tests

All tests

dbt test

Unit tests

dbt test --select tag:unit-tests

Component tests

dbt test --select tag:component-test

Contract tests

dbt test --select tag:contract-test-source

Running data quality tests (Coming soon)

dbt test --select tag:data-quality

Setup data observability

Build Elementary models. It will create empty tables, that will be updated with artifacts, metrics and test results in future dbt executions.

dbt run --select elementary

Run the tests

dbt test

Generate data observability report

edr report