Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track parent partition size when child partitions are untracked #656

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Track parent partition size when child partitions are untracked #656

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion input/postgres/relation_stats.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,16 @@ locked_relids_with_parents AS (
UNION SELECT relid FROM locked_relids
)
SELECT c.oid,
COALESCE(pg_catalog.pg_table_size(c.oid), 0) AS size_bytes,
COALESCE(pg_catalog.pg_table_size(c.oid), 0) +
COALESCE((
SELECT pg_catalog.sum(pg_catalog.pg_table_size(inhrelid))
FROM pg_catalog.pg_inherits
JOIN pg_class t ON inhrelid = t.oid
WHERE inhparent = c.oid
-- Only include sizes from child partitions skipped by ignore_schema_regexp.
-- Child partitions tracked by the collector have their sizes added by mergePartitionSizes.
AND ($1 != '' AND (n.nspname || '.' || t.relname) ~* $1)
), 0) AS size_bytes,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The screenshot below is from local testing. 11:40 shows the detailed stats from #635. I then updated the collector config with ignore_schema_regexp = 35d_, and then 12:10 shows the updated behavior with this PR. Ignore the in-between timestamps where I was fixing bugs.

Screenshot 2024-12-19 at 12 12 08 PM

CASE c.reltoastrelid WHEN NULL THEN 0 ELSE COALESCE(pg_catalog.pg_total_relation_size(c.reltoastrelid), 0) END AS toast_bytes,
COALESCE(pg_stat_get_numscans(c.oid), 0) AS seq_scan,
COALESCE(pg_stat_get_tuples_returned(c.oid), 0) AS seq_tup_read,
Expand Down
6 changes: 6 additions & 0 deletions output/transform/merge_partition_sizes.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ import (
"github.com/pganalyze/collector/state"
)

// Since Postgres reports parent partition tables as being zero-sized,
// this backfills stats for the parent table as a summation of all child tables.
//
// When ignore_schema_regexp is set, GetRelationStats bypasses this for tables not tracked by the collector.
//
// TODO: recursively build up stats when nested partitioning is used
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This TODO is an existing issue with the code.

func mergePartitionSizes(s snapshot.FullSnapshot, newState state.PersistedState, ts state.TransientState, databaseOidToIdx OidToIdx) snapshot.FullSnapshot {
for idx, rel := range s.RelationInformations {
if !rel.HasParentRelation || rel.PartitionBoundary == "" {
Expand Down
Loading