Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(generic-metrics): Add generic metrics interface #46957
feat(generic-metrics): Add generic metrics interface #46957
Changes from 11 commits
a880b35
6332cdc
a195e70
05952a2
8558b04
525049f
2d7b9b6
cf44caf
f342ded
ce1e05c
5fa2aaf
0c21461
565411e
c6032fd
dd51b80
076c2f3
faa5db4
73cb703
5931748
63af343
4fb4c29
f2dc617
b0d389b
1cc5cda
7486191
240f90e
5db95d7
e6e01f2
7f20f17
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is quite an opinionated interface. Why is org and project ID needed at all? Seems like it can't be used for any use cases that aren't within the context of a single org and project then?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is (a required) part of the back-end schema so I don't think we should drop these.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's because the backend schema is a multi-tenant system and we need to keep individual customer data separate from other customers. For Sentry, this seems like it should always be org_id = 1 and project_id = 1. Why asking the user to pass it every time here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This wouldn't be the Sentry org_id and project_id since this would be customer data. @onewland am I missing anything?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So just to clarify, the metrics platform is for aggregating customer data received during ingest only? I think this restriction wasn't super clear to me before, and I assumed it was valid to use metrics for anything at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think from both a query efficiency (the back-end tables are ordered first by org_id and project_id) and cost accounting perspective we should aim to include customer project IDs and org IDs where they make sense, and we can pass Sentry org IDs and project IDs everywhere else.
For example, on the escalating issues project, we're trying to keep a count of error by group ID on ingest. This has an obvious correspondence to a customer's project_id and org_id, and therefore that data should propagate through.
If we wanted to track something like the number of times a celery task is executed, and this is independent from customer data/behavior, then I think it would make more sense to do org_id = 1 and project_id = 1.
Lyn, how would you feel if these parameters were optional and the backends did some resolution if org_id/project_id aren't passed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can't just hardcode org_id 1 and and project_id 1 though. Is there a plan for how these values will get resolved?
Does this interface make using Relay a non-starter? Or does there need to be other changes there to make this happen? The implication of this interface seems to be that a project can send data onto some other org/project not just itself. I am guessing this is not something generally allowed in Relay as it might have all kinds of implications in terms of security, quotas, etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that fallback values could be configured in Sentry options so they could vary by deployment.
I don't see why this interface makes Relay a non-starter, nor why you think it implies that projects would send data on behalf of other projects. Keep in mind that this interface will only be supported by an internal Relay endpoint, and we would not be exposing this interface to anybody outside Sentry. If we ever reach the point where we want to accept metrics externally, it will be through a DSN which does auth and so there will always be a customer-specific org/project_id attached.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What changes are planned on Relay-side? Without any, you won't be able to implement this interface for Relay. If you only want to implement this interface for Kafka all is well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
right, so just to clarify - the internal Relay endpoint mentioned above would allow us to skip authenticating as Sentry, and would allow us to send customer org_id and project_id to Relay via this interface