[Bug] Unit test compilation error 'var is undefined' when macro is used with project variables as input parameters

[canva-owen-yan]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When calling calling a macro that takes project variables as input, the configured unit tests fail with var is undefined error.
Overriding the macro or project variables doesn't help but removing the variables resolves the issue. No issue with running the model without unit test.

Expected Behavior

Expecting unit test to work with macro taking project variables as input, or at least end user can override the macro or the project variables used as inputs for the macro.

Steps To Reproduce

get_columns.sql

{%- macro get_columns(include=[]) -%}
    include
{%- endmacro -%}

my_model.sql

{{
  config(
  materialized='table'
)
}}
select 
  {{ get_columns(
    include=var('columns_list_one') + var('columns_list_two')
  )}}
from model

dbt_project.yml

columns_list_one:
  - column_a
  - column_b

columns_list_two:
  - column_c

unit_test__my_model.yml
Tried without override

Override macro:

    overrides:
      vars:
        get_columns: override

Override input:

    overrides:
      vars:
        columns: ['column_a','column_b','column_c']

Relevant log output

'var' is undefined. This can happen when calling a macro that does not exist. Check for typos and/or install package dependencies with "dbt deps".

Environment

- OS: Apple M1 Max
- Python: 3.11
- dbt: 1.8.2

Which database adapter are you using with dbt?

snowflake

Additional Context

No response

[dbeatty10]

Thanks for reporting this @canva-owen-yan !

The error message looks the same as in #10157, so I'm wondering if you are experiencing the same (or similar) thing.

When I tried to replicate this though, I wasn't able to. Could you try out the following files and try tweaking them as needed to reproduce the error you are seeing?

dbt_project.yml

vars:
  columns:
    - column_a
    - column_b
    - column_c

macros/get_columns.sql

{%- macro get_columns(include=[]) -%}
    {%- for col in include -%}
        {{ col }}{% if not loop.last %}, {% endif %}
    {%- endfor -%}
{%- endmacro -%}

models/upstream_model.sql

select
    1 as column_a,
    2 as column_b,
    3 as column_c

models/my_model.sql

select 
  {{ get_columns(
    include=var('columns')
  )}}
from {{ ref("upstream_model") }}

models/_unit_tests.yml

unit_tests:
  - name: test_10539
    model: my_model
    given:
      - input: ref('upstream_model') 
        format: dict
        rows:
          - {column_a: 11}
    expect:
        rows:
          - {column_a: 11}

Commands:

dbt build

Output:

(dbt_1.8) $ dbt build               
14:10:33  Running with dbt=1.8.3
14:10:33  Registered adapter: duckdb=1.8.1
14:10:33  Found 2 models, 407 macros, 1 unit test
14:10:33  
14:10:34  Concurrency: 1 threads (target='dev')
14:10:34  
14:10:34  1 of 3 START sql view model feature_456.upstream_model ......................... [RUN]
14:10:34  1 of 3 OK created sql view model feature_456.upstream_model .................... [OK in 0.08s]
14:10:34  2 of 3 START unit_test my_model::test_10539 .................................... [RUN]
14:10:34  2 of 3 PASS my_model::test_10539 ............................................... [PASS in 0.17s]
14:10:34  3 of 3 START sql view model feature_456.my_model ............................... [RUN]
14:10:34  3 of 3 OK created sql view model feature_456.my_model .......................... [OK in 0.08s]
14:10:34  
14:10:34  Finished running 2 view models, 1 unit test in 0 hours 0 minutes and 0.56 seconds (0.56s).
14:10:34  
14:10:34  Completed successfully
14:10:34  
14:10:34  Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3

[canva-owen-yan]

Thanks for reporting this @canva-owen-yan !

The error message looks the same as in #10157, so I'm wondering if you are experiencing the same (or similar) thing.

When I tried to replicate this though, I wasn't able to. Could you try out the following files and try tweaking them as needed to reproduce the error you are seeing?

dbt_project.yml

vars:
  columns:
    - column_a
    - column_b
    - column_c

macros/get_columns.sql

{%- macro get_columns(include=[]) -%}
    {%- for col in include -%}
        {{ col }}{% if not loop.last %}, {% endif %}
    {%- endfor -%}
{%- endmacro -%}

models/upstream_model.sql

select
    1 as column_a,
    2 as column_b,
    3 as column_c

models/my_model.sql

select 
  {{ get_columns(
    include=var('columns')
  )}}
from {{ ref("upstream_model") }}

models/_unit_tests.yml

unit_tests:
  - name: test_10539
    model: my_model
    given:
      - input: ref('upstream_model') 
        format: dict
        rows:
          - {column_a: 11}
    expect:
        rows:
          - {column_a: 11}

Commands:

dbt build

Output:

(dbt_1.8) $ dbt build               
14:10:33  Running with dbt=1.8.3
14:10:33  Registered adapter: duckdb=1.8.1
14:10:33  Found 2 models, 407 macros, 1 unit test
14:10:33  
14:10:34  Concurrency: 1 threads (target='dev')
14:10:34  
14:10:34  1 of 3 START sql view model feature_456.upstream_model ......................... [RUN]
14:10:34  1 of 3 OK created sql view model feature_456.upstream_model .................... [OK in 0.08s]
14:10:34  2 of 3 START unit_test my_model::test_10539 .................................... [RUN]
14:10:34  2 of 3 PASS my_model::test_10539 ............................................... [PASS in 0.17s]
14:10:34  3 of 3 START sql view model feature_456.my_model ............................... [RUN]
14:10:34  3 of 3 OK created sql view model feature_456.my_model .......................... [OK in 0.08s]
14:10:34  
14:10:34  Finished running 2 view models, 1 unit test in 0 hours 0 minutes and 0.56 seconds (0.56s).
14:10:34  
14:10:34  Completed successfully
14:10:34  
14:10:34  Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3

Thanks for looking into it @dbeatty10!
I can confirm that it's not reproducible with the provided files when only 1 variable is involved, could you please give it another try by providing 2 project variables together as the input params to the macro (sorry I should have made it clearer, will update description accordingly)?

dbt_project.yml

vars:
  columns_list_one:
    - column_a
    - column_b

  columns_list_two:
    - column_c

models/my_model.sql

select 
  {{ get_columns(
    include=var('columns_list_one') + var('columns_list_two')
  )}}
from {{ ref("upstream_model") }}

Commands:

dbt run -m my_model

Output:

05:01:28  Concurrency: 1 threads (target='dev')
05:01:28  
05:01:28  1 of 1 START sql view model dev.my_model ................................. [RUN]
05:01:31  1 of 1 OK created sql view model dev.my_model ............................ [SUCCESS 1 in 2.92s]
05:01:33  
05:01:33  Finished running 1 view model, 1 project hook in 0 hours 0 minutes and 18.22 seconds (18.22s).
05:01:38  
05:01:38  Completed successfully
05:01:38  
05:01:38  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1

dbt build -m my_model

Output:

05:09:26  Concurrency: 1 threads (target='dev')
05:09:26  
05:09:26  1 of 2 START unit_test dev.my_model::test_10539 ..................... [RUN]
05:09:26  1 of 2 ERROR dev.my_model::test_10539 ............................... [ERROR in 0.01s]
05:09:26  2 of 2 SKIP relation dev.my_model ............................. [SKIP]
05:09:29  
05:09:29  Finished running 1 unit test, 1 view model, 1 project hook in 0 hours 0 minutes and 20.20 seconds (20.20s).
05:09:35  
05:09:35  Completed with 1 error and 0 warnings:
05:09:35  
05:09:35    Compilation Error in unit_test test_10539 (models/dev/my_model/_unit_tests.yml)
  'var' is undefined. This can happen when calling a macro that does not exist. Check for typos and/or install package dependencies with "dbt deps".
05:09:35  
05:09:35  Done. PASS=0 WARN=0 ERROR=1 SKIP=1 TOTAL=2

[dbeatty10]

Thanks for updating that example @canva-owen-yan -- I was able to reproduce with the files and commands that you provided 👍

The error message looks the same / similar to #10353 and #10157, but it's not obvious if it is a duplicate issue or not, so I'll leave all three of those open as distinct issues for now.

[dbeatty10]

Reprex

Create these files:

dbt_project.yml

# project name, etc here

vars:
  columns_list_one:
    - column_a
    - column_b

  columns_list_two:
    - column_c

macros/get_columns.sql

{%- macro get_columns(include=[]) -%}
    {%- for col in include -%}
        {{ col }}{% if not loop.last %}, {% endif %}
    {%- endfor -%}
{%- endmacro -%}

models/my_model.sql

select
  {{ get_columns(
    include=var('columns_list_one') + var('columns_list_two')
  )}}
from {{ ref("upstream_model") }}

models/_unit_tests.yml

unit_tests:
  - name: test_10539
    model: my_model
    overrides:
      macros:
        get_columns: ['column_a','column_b','column_c']
      vars:
        columns_list_one:
          - column_a
          - column_b
        columns_list_two:
          - column_c
    given:
      - input: ref('upstream_model') 
        format: dict
        rows:
          - {column_a: 11}
    expect:
        rows:
          - {column_a: 11}

Run these commands:

dbt build --select my_model

Get this output:

$ dbt build -s my_model

19:18:29  Running with dbt=1.8.6
19:18:32  Registered adapter: duckdb=1.8.3
19:18:32  Found 2 models, 411 macros, 1 unit test
19:18:32  
19:18:32  Concurrency: 1 threads (target='dev')
19:18:32  
19:18:32  1 of 2 START unit_test my_model::test_10539 .................................... [RUN]
19:18:32  1 of 2 ERROR my_model::test_10539 .............................................. [ERROR in 0.01s]
19:18:32  2 of 2 SKIP relation feature_456.my_model ...................................... [SKIP]
19:18:32  
19:18:32  Finished running 1 unit test, 1 view model in 0 hours 0 minutes and 0.20 seconds (0.20s).
19:18:32  
19:18:32  Completed with 1 error and 0 warnings:
19:18:32  
19:18:32    Compilation Error in unit_test test_10539 (models/_unit_tests.yml)
  'var' is undefined. This can happen when calling a macro that does not exist. Check for typos and/or install package dependencies with "dbt deps".
19:18:32  
19:18:32  Done. PASS=0 WARN=0 ERROR=1 SKIP=1 TOTAL=2

Note

I got the same error message regardless if anything was supplied to overrides or not.