Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Test ClickHouse 23.3 #4545

Closed
wants to merge 13 commits into from
Closed

ci: Test ClickHouse 23.3 #4545

wants to merge 13 commits into from

Conversation

lynnagara
Copy link
Member

@lynnagara lynnagara commented Jul 19, 2023

@github-actions
Copy link

github-actions bot commented Jul 20, 2023

This PR has a migration; here is the generated SQL

-- start migrations

-- forward migration functions : 0001_functions
Local op: CREATE TABLE IF NOT EXISTS functions_raw_local (project_id UInt64, transaction_name String, timestamp DateTime, depth UInt32, parent_fingerprint UInt64, fingerprint UInt64, name String, package String, path String, is_application UInt8, platform LowCardinality(String), environment LowCardinality(Nullable(String)), release LowCardinality(Nullable(String)), os_name LowCardinality(String), os_version LowCardinality(String), retention_days UInt16, durations Array(Float64), profile_id UUID, materialization_version UInt8) ENGINE ReplicatedMergeTree('/clickhouse/tables/functions/{shard}/default/functions_raw_local', '{replica}') ORDER BY (project_id, transaction_name, timestamp) PARTITION BY (toStartOfInterval(timestamp, INTERVAL 12 HOUR)) TTL timestamp + toIntervalDay(1);
Local op: CREATE TABLE IF NOT EXISTS functions_mv_local (project_id UInt64, transaction_name String, timestamp DateTime, depth UInt32, parent_fingerprint UInt64, fingerprint UInt64, name String, package String, path String, is_application UInt8, platform LowCardinality(String), environment LowCardinality(Nullable(String)), release LowCardinality(Nullable(String)), os_name LowCardinality(String), os_version LowCardinality(String), retention_days UInt16, count AggregateFunction(count, Float64), percentiles AggregateFunction(quantiles(0.5, 0.75, 0.9, 0.95, 0.99), Float64), min AggregateFunction(min, Float64), max AggregateFunction(max, Float64), avg AggregateFunction(avg, Float64), sum AggregateFunction(sum, Float64), worst AggregateFunction(argMax, UUID, Float64), examples AggregateFunction(groupUniqArray(5), UUID)) ENGINE ReplicatedAggregatingMergeTree('/clickhouse/tables/functions/{shard}/default/functions_mv_local', '{replica}') PRIMARY KEY (project_id, transaction_name, timestamp, depth, parent_fingerprint, fingerprint) ORDER BY (project_id, transaction_name, timestamp, depth, parent_fingerprint, fingerprint, name, package, path, is_application, platform, environment, release, os_name, os_version, retention_days) PARTITION BY (retention_days, toMonday(timestamp)) TTL timestamp + toIntervalDay(retention_days) SETTINGS index_granularity=2048;
Local op: CREATE MATERIALIZED VIEW IF NOT EXISTS functions_local TO functions_mv_local (project_id UInt64, transaction_name String, timestamp DateTime, depth UInt32, parent_fingerprint UInt64, fingerprint UInt64, name String, package String, path String, is_application UInt8, platform LowCardinality(String), environment LowCardinality(Nullable(String)), release LowCardinality(Nullable(String)), os_name LowCardinality(String), os_version LowCardinality(String), retention_days UInt16, count AggregateFunction(count, Float64), percentiles AggregateFunction(quantiles(0.5, 0.75, 0.9, 0.95, 0.99), Float64), min AggregateFunction(min, Float64), max AggregateFunction(max, Float64), avg AggregateFunction(avg, Float64), sum AggregateFunction(sum, Float64), worst AggregateFunction(argMax, UUID, Float64), examples AggregateFunction(groupUniqArray(5), UUID)) AS 
            SELECT
                project_id,
                transaction_name,
                name,
                package,
                path,
                fingerprint,
                parent_fingerprint,
                depth,
                is_application,
                platform,
                environment,
                release,
                os_name,
                os_version,
                toDateTime(3600 * intDiv(toUnixTimestamp(timestamp), 3600)) AS timestamp,
                retention_days,
                countState(arrayJoin(durations) AS duration) AS count,
                quantilesState(0.5, 0.75, 0.9, 0.95, 0.99)(duration) AS percentiles,
                minState(duration) AS min,
                maxState(duration) AS max,
                avgState(duration) AS avg,
                sumState(duration) AS sum,
                argMaxState(profile_id, duration) as worst,
                groupUniqArrayState(5)(profile_id) as examples
            FROM functions_raw_local
            WHERE materialization_version = 0
            GROUP BY
                project_id,
                transaction_name,
                name,
                package,
                path,
                fingerprint,
                parent_fingerprint,
                depth,
                is_application,
                platform,
                environment,
                release,
                os_name,
                os_version,
                timestamp,
                retention_days
        ;
Distributed op: CREATE TABLE IF NOT EXISTS functions_raw_dist (project_id UInt64, transaction_name String, timestamp DateTime, depth UInt32, parent_fingerprint UInt64, fingerprint UInt64, name String, package String, path String, is_application UInt8, platform LowCardinality(String), environment LowCardinality(Nullable(String)), release LowCardinality(Nullable(String)), os_name LowCardinality(String), os_version LowCardinality(String), retention_days UInt16, durations Array(Float64), profile_id UUID, materialization_version UInt8) ENGINE Distributed(cluster_one_sh, default, functions_raw_local);
Distributed op: CREATE TABLE IF NOT EXISTS functions_mv_dist (project_id UInt64, transaction_name String, timestamp DateTime, depth UInt32, parent_fingerprint UInt64, fingerprint UInt64, name String, package String, path String, is_application UInt8, platform LowCardinality(String), environment LowCardinality(Nullable(String)), release LowCardinality(Nullable(String)), os_name LowCardinality(String), os_version LowCardinality(String), retention_days UInt16, count AggregateFunction(count, Float64), percentiles AggregateFunction(quantiles(0.5, 0.75, 0.9, 0.95, 0.99), Float64), min AggregateFunction(min, Float64), max AggregateFunction(max, Float64), avg AggregateFunction(avg, Float64), sum AggregateFunction(sum, Float64), worst AggregateFunction(argMax, UUID, Float64), examples AggregateFunction(groupUniqArray(5), UUID)) ENGINE Distributed(cluster_one_sh, default, functions_mv_local);
-- end forward migration functions : 0001_functions




-- backward migration functions : 0001_functions
Distributed op: DROP TABLE IF EXISTS functions_raw_dist;
Distributed op: DROP TABLE IF EXISTS functions_mv_dist;
Local op: DROP TABLE IF EXISTS functions_raw_local;
Local op: DROP TABLE IF EXISTS functions_mv_local;
Local op: DROP TABLE IF EXISTS functions_local;
-- end backward migration functions : 0001_functions
-- forward migration querylog : 0001_querylog
Local op: CREATE TABLE IF NOT EXISTS querylog_local (request_id UUID, request_body String, referrer LowCardinality(String), dataset LowCardinality(String), projects Array(UInt64), organization Nullable(UInt64), timestamp DateTime, duration_ms UInt32, status Enum('success' = 0, 'error' = 1, 'rate-limited' = 2), clickhouse_queries Nested(sql String, status Enum('success' = 0, 'error' = 1, 'rate-limited' = 2), trace_id Nullable(UUID), duration_ms UInt32, stats String, final UInt8, cache_hit UInt8, sample Float32, max_threads UInt8, num_days UInt32, clickhouse_table LowCardinality(String), query_id String, is_duplicate UInt8, consistent UInt8)) ENGINE ReplicatedMergeTree('/clickhouse/tables/querylog/{shard}/default/querylog_local', '{replica}') ORDER BY (toStartOfDay(timestamp), request_id) PARTITION BY (toMonday(timestamp));
Distributed op: CREATE TABLE IF NOT EXISTS querylog_dist (request_id UUID, request_body String, referrer LowCardinality(String), dataset LowCardinality(String), projects Array(UInt64), organization Nullable(UInt64), timestamp DateTime, duration_ms UInt32, status Enum('success' = 0, 'error' = 1, 'rate-limited' = 2), clickhouse_queries Nested(sql String, status Enum('success' = 0, 'error' = 1, 'rate-limited' = 2), trace_id Nullable(UUID), duration_ms UInt32, stats String, final UInt8, cache_hit UInt8, sample Float32, max_threads UInt8, num_days UInt32, clickhouse_table LowCardinality(String), query_id String, is_duplicate UInt8, consistent UInt8)) ENGINE Distributed(cluster_one_sh, default, querylog_local);
-- end forward migration querylog : 0001_querylog




-- backward migration querylog : 0001_querylog
Distributed op: DROP TABLE IF EXISTS querylog_dist;
Local op: DROP TABLE IF EXISTS querylog_local;
-- end backward migration querylog : 0001_querylog

@lynnagara lynnagara added the skip-check-migrations skip checking the migration CI for coupled changes label Jul 20, 2023
@lynnagara lynnagara force-pushed the test-clickhouse-23.3 branch 2 times, most recently from 510b0c2 to 082c1c7 Compare July 25, 2023 23:20
@getsantry getsantry bot added the Stale label Oct 18, 2023
@lynnagara lynnagara closed this Oct 31, 2023
@lynnagara lynnagara deleted the test-clickhouse-23.3 branch October 31, 2023 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
migrations skip-check-migrations skip checking the migration CI for coupled changes Stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant