Using the Python dbt package

[tehe]

Suppose I wanted to interact (build, run, test) with my dbt project(s) from Python code, is it OK to use dbt.task.* Python modules?

I mean, I've got it to work with code like this:

            args = argparse.Namespace()
            args.project_dir = '<my_dbt_project_dir>'
            args.profiles_dir = args.project_dir + '/profiles'
            args.config_dir = True
            dbt.flags.PROFILES_DIR = args.profiles_dir
            user_config = read_user_config(dbt.flags.PROFILES_DIR)
            dbt.flags.set_from_args(args, user_config)
            args.which = 'debug'
            task = debug.DebugTask.from_args(args)
            dbt.tracking.initialize_from_flags()
            task.run()

But it feels a little hacky to me and I'm wondering if the dbt.task.* modules in the Python dbt package are public interfaces or a private one meant to be used just by the dbt CLI.

Thanks,
Tomas

[lostmygithubaccount]

hi @tehe, it is hacky and we don't recommend it! however, one of the major focuses for v1.4 and v1.5 of dbt Core is refactoring our Python APIs and enabling this kind of interactive use, both for ourselves and the community

you can find some detail on that in our roadmap post: https://github.com/dbt-labs/dbt-core/blob/main/docs/roadmap/2022-08-back-for-more.md#v14-january-for-us-for-you-for-core

you can follow along with much of the work in this issue: dbt-labs/dbt-core#5527

after this is complete, we will have a stable high-level Python API that we ourselves will leverage and we could recommend to you

[tehe]

Hi Cody. Thank you for the detailed answer and I'm looking forward to the new stable API. Regards, Tomas