Implementing Pytest In dbt Packages

[callum-mcdata]

If We Had To Write A Guide

The developer experience team over at dbt labs is incredibly bullish on pytest - we believe that it represents the next logical step in package development and ensuring that tests can match the functionality of other package development frameworks. Also, it gives us unit testing and that is a capital B BIG DEAL.

What we're currently lacking is a best practice guide or principles around how this should be implemented:

• How do we group tests?
• What should live in fixtures vs the testing file?
• How should we recommend the python be written for testing?

These are all questions that we'll want to answer and I personally want to answer after working on adding pytest to dbt_metrics. Figured I'd start an issue and we can get some discussion going. Individuals here will probably have strong opinions and I'm here for it as I don't!

[dbeatty10]

Would love to see a best practice guide come together! 🤩

I didn't put any personal opinions down, but I did generate some possible variations here:

• Example testing a macro within a local dbt project dbt-tests-adapter-custom-tests#2

☝️ very simple and skin-deep.

[callum-mcdata]

Here are my thoughts after implementing a first draft of this with dbt_metrics:

• Assets referenced across multiple tests should be in the fixtures.py file. I didn't follow this practice 100% out of desire to have some metric definitions live in the text file and it wasn't as beneficial as I hoped. My v2 will fix move these definitions unless they are bespoke to the test
• Expected datasets (ie unit testing data) should live in the testing file. This makes it much easier to make changes during development if anything is wrong
• Each set of functionality should live within its own folder underneath functional. This helps logically group things and makes it much easier to navigate
• Again, learning from my implementation, I think the way we should account for differences in DWHs should be as follows. I didn't follow this method until later in development and created a lot of duplicate code.
  • define the exception behavior
  • else statement to define default behavior
• I personally like breaking each dbt run step into its own step in the final function. The error logs display the failing step quite well and make it "easy" to debug. If we compiled these into a single step we'd potentially lose that

[joellabes]

define the exception behavior
else statement to define default behavior

This is the opposite direction to most advice I've been given, which is to early exit on the happy path and only deal with weirdness in the else block. Can you say more about why doing the default in the main block and handling exceptions elsewhere causes code duplication?

[callum-mcdata]

This is probably hyper specific to the method I used to break apart the different DWH's (might be a better way as well) but BigQuery had a lot of exceptions that other DWHs didn't. So I defined that "exception" behavior and then defined the default after so that it could satisfy other behavior.

In the few places I didn't do that (and defined behavior for all DWHs) the code ended up being a lot longer