Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: reduce usage of PreProcess task #811

Merged
merged 1 commit into from
Sep 11, 2024

Conversation

giovanni-guidini
Copy link
Contributor

Originally I wanted to delete PreProcess task completely, but after reviewing
certain usage patterns I realized the task is necessary (for empty-upload and
static-analysis for example).

So the alternative is to reduce the amount of times it is called, to only be called
when we effectively create a new report. Similar to #804

Purpose/Motivation

What is the feature? Why is this being done?

Links to relevant tickets

What does this PR do?

Include a brief description of the changes in this PR. Bullet points are your friend.

Notes to Reviewer

Anything to note to the team? Any tips on how to review, or where to start?

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

Originally I wanted to delete PreProcess task completely, but after reviewing
certain usage patterns I realized the task is necessary (for `empty-upload` and
`static-analysis` for example).

So the alternative is to reduce the amount of times it is called, to only be called
when we effectively create a new report. Similar to #804
@giovanni-guidini giovanni-guidini requested a review from a team as a code owner September 10, 2024 20:38
@codecov-notifications
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

Copy link

codecov bot commented Sep 10, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.22%. Comparing base (4055e66) to head (5a383f4).
Report is 2 commits behind head on main.

✅ All tests successful. No failed tests found.

Additional details and impacted files
@@             Coverage Diff             @@
##               main       #811   +/-   ##
===========================================
  Coverage   96.22000   96.22000           
===========================================
  Files           812        812           
  Lines         18538      18536    -2     
===========================================
- Hits          17839      17836    -3     
- Misses          699        700    +1     
Flag Coverage Δ
unit 92.05% <100.00%> (-0.03%) ⬇️
unit-latest-uploader 92.05% <100.00%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Swatinem
Copy link
Contributor

One question though:
As these tasks are asynchronous, can we end up in a situation where the Upload task is being run before the PreProcessUpload task finishes?

Also, why is this an async task in the first place, and why not synchronously do whatever needs to be done within the API request?

@giovanni-guidini
Copy link
Contributor Author

One question though: As these tasks are asynchronous, can we end up in a situation where the Upload task is being run before the PreProcessUpload task finishes?

Also, why is this an async task in the first place, and why not synchronously do whatever needs to be done within the API request?

For question 1:
yes. But the PreProcess and Upload tasks are redundant in terms of "carryforwarding reports" (both do it, but the Upload task does other things too).

However there are scenarios - empty_upload being an example, apparently - in which there is no Upload task.

For question 2: I think 2 reasons:
(1) code instrumentation. The worker has the code that is used (currently) to handle report processing, and carryforwarding reports is part of that. Just for this reason we naturally turned to making it an async task

(2) the operation takes too long. It can take up to several seconds to actually do the carryforwarding of reports. That's a bit too much (I think) do do in the request cycle

@giovanni-guidini giovanni-guidini added this pull request to the merge queue Sep 11, 2024
Merged via the queue into main with commit f6ed141 Sep 11, 2024
17 of 19 checks passed
@giovanni-guidini giovanni-guidini deleted the gio/soft-kill-preprocess-task branch September 11, 2024 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants