Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure Snowflake account usage role, DB & Schema #18064

Open
SamJBentley opened this issue Oct 1, 2024 · 0 comments
Open

Configure Snowflake account usage role, DB & Schema #18064

SamJBentley opened this issue Oct 1, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request Ingestion

Comments

@SamJBentley
Copy link

Is your feature request related to a problem? Please describe.
I am trying to use the Profiler for Snowflake. My company stores account usage data in a separate (and custom-named) database and schema. We also need to configure the role for this too.

Describe the solution you'd like
I'd like to be able to configure the Snowflake DB, schema and role for profile queries. These configuration options are separate to data quality queries, which are in our project's Snowflake DB, schema and role. These work fine, but the profile queries do not (because of the situation descrbed, above)

Describe alternatives you've considered
N/A

Additional context
OpenMetadata defaults for Snowflake Profile data:
Database = snowflake
Schema = account_usage
Role = [role defined in Snowflake database services in OM]

My company's account usage location:
Database = abc_xyz
Schema = abc_account_usage
Role = my_account_usage_role

Names have been changed to protect company info.

The logs that show the failed profile ingestion are shown below:

Schema 'SNOWFLAKE.ACCOUNT_USAGE' does not exist or not authorized.
[SQL: /* {"app": "OpenMetadata", "version": "1.5.4.1"} */
    SELECT
        QUERY_ID,
        QUERY_TEXT,
        QUERY_TYPE,
        START_TIME,
        ROWS_INSERTED,
        ROWS_UPDATED,
        ROWS_DELETED
    FROM "SNOWFLAKE"."ACCOUNT_USAGE"."QUERY_HISTORY"
    WHERE
    start_time>= DATEADD('DAY', -1, CURRENT_TIMESTAMP)
    AND QUERY_TEXT ILIKE '%%my_table_name%%'
    AND QUERY_TYPE IN (
        'INSERT',
        'UPDATE',
        'DELETE',
        'MERGE'
    )
    AND EXECUTION_STATUS = 'SUCCESS';
]

It fails because the FROM should be FROM abc_xyz.abc_account_usage. The SQL queries for this are hard-coded here and I'd like these to be configurable to support my request.

@SamJBentley SamJBentley added the enhancement New feature or request label Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Ingestion
Projects
None yet
Development

No branches or pull requests

3 participants