fix(hc): Support split db monolith mode via smarter delegators #53365
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
Scope: Backend
corps
commented
Jul 21, 2023
| for model, constructor in self._constructors.items(): | ||
| # Shared memory reference for the transaction.on_commit | ||
| switch = _Switch() | ||
| transaction.on_commit(switch, router.db_for_read(model)) |
There was a problem hiding this comment.
Actualy, there may be an easier way to make this check, may improve.
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## master #53365 +/- ##
==========================================
- Coverage 79.54% 78.23% -1.32%
==========================================
Files 4940 4917 -23
Lines 208184 208298 +114
Branches 35483 35490 +7
==========================================
- Hits 165600 162960 -2640
- Misses 37545 40154 +2609
- Partials 5039 5184 +145
|
mrduncan
approved these changes
Jul 22, 2023
|
|
||
| from sentry.db.postgres.transactions import django_test_transaction_water_mark | ||
| from sentry.models import AuditLogEntry, OutboxCategory, OutboxScope, RegionOutbox, UserIP | ||
| from sentry.services.hybrid_cloud.log import AuditLogEvent, LogService, UserIpEvent | ||
| from sentry.utils import metrics | ||
|
|
||
|
|
||
| class DatabaseBackedLogService(LogService): |
There was a problem hiding this comment.
Awesome that the service implementations themselves get simpler because of this
| } | ||
| ) | ||
|
|
||
| with override_settings(SILO_MODE=SiloMode.CONTROL): |
There was a problem hiding this comment.
pytest.mark.parametrize is sometimes useful for these types of matrices, not sure it'd necessarily make things more clear here though
| ServiceInterface, | ||
| DelegatedByOpenTransaction( | ||
| { | ||
| User: mapping[SiloMode.CONTROL], |
There was a problem hiding this comment.
Thinking out loud - if more models are added here, could the silo mode be inferred from the model?
dashed
approved these changes
Jul 22, 2023
corps
deleted the
zc/ensure-monolith-services-align-with-open-transactions
branch
|
PR reverted: 97c76f2 |
armenzg
pushed a commit
that referenced
this pull request
Jul 24, 2023
Monolith mode delegators now *also* pick their implementation by attempting to examine any open transactions and selecting for an implementation that would align with the currently open tranaction. This fixes the subtle issue for hybrid cloud services like the log_service, as well as some helper methods on the organization_service, that do not invoke RPC, but rather, *pick different models* locally based on the silo mode. For Monolith mode, this means picking one model that works on one db, even if being invoked inside the transaction of the other db, threatening the transactional atomicity that is desired. (`schedule_signal`, for instance, hopes to leave an outbox behind in the same transaction as invocation). The Sentry Options backend *may* also need this, but unclear. Going to analyze split db tests after this lands and follow up.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Monolith mode delegators now also pick their implementation by attempting to examine any open transactions and selecting for an implementation that would align with the currently open tranaction.
This fixes the subtle issue for hybrid cloud services like the log_service, as well as some helper methods on the organization_service, that do not invoke RPC, but rather, pick different models locally based on the silo mode. For Monolith mode, this means picking one model that works on one db, even if being invoked inside the transaction of the other db, threatening the transactional atomicity that is desired. (
schedule_signal, for instance, hopes to leave an outbox behind in the same transaction as invocation).The Sentry Options backend may also need this, but unclear. Going to analyze split db tests after this lands and follow up.