-
Notifications
You must be signed in to change notification settings - Fork 319
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: warehouse router tracker #5407
Conversation
4b17736
to
1f6a706
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## release/1.40.x #5407 +/- ##
==================================================
- Coverage 74.80% 74.73% -0.07%
==================================================
Files 440 440
Lines 61442 61412 -30
==================================================
- Hits 45963 45899 -64
- Misses 12950 12972 +22
- Partials 2529 2541 +12 ☔ View full report in Codecov by Sentry. |
SELECT EXISTS ( | ||
SELECT 1 | ||
FROM ` + whutils.WarehouseUploadsTable + ` | ||
WHERE source_id = $1 AND destination_id = $2 AND | ||
(status = $3 OR status = $4 OR status LIKE $5) AND | ||
updated_at > $6 | ||
);` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was just checking indexes for uploads table.
- Do we keep deleting old entries from UploadsTable else without proper index on time columns this query can keep getting slower with time.
- Not relevant to this PR.
CREATE INDEX wh_uploads_status_index ON wh_uploads(status text_ops);
index on status column with very low cardinality could have more performance drawbacks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we keep deleting old entries from UploadsTable else without proper index on time columns this query can keep getting slower with time.
Yes.
rudder-server/warehouse/archive/archiver.go
Line 510 in b51605a
func (a *Archiver) Delete(ctx context.Context) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not relevant to this PR. CREATE INDEX wh_uploads_status_index ON wh_uploads(status text_ops); index on status column with very low cardinality could have more performance drawbacks.
True, For columns with low cardinality, PostgreSQL might opt for a sequential scan over using the index, as scanning the table can be more efficient than traversing an index with low selectivity.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tracking Query-related optimizations/discussions in this doc.
d208c7e
to
b436b44
Compare
Description
warehouse_track_upload_missing
Guage metrics twice. First with value0
and then if missing upload exists we send value with 1.1
is continuously fired for2H
then the alert is being sent. But if the value comes as0
it will get resolved. For calculating the value it uses whatever it being present at that instant.Linear Ticket
Security