Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move org storage recalculation into background job #2138

Merged
merged 3 commits into from
Nov 19, 2024

Conversation

tw4l
Copy link
Member

@tw4l tw4l commented Nov 11, 2024

Fixes #2112

  • Moves org storage recalculation to background job, modify endpoint to return job id as part of response
  • Updates crawl + QA backend tests that broke due to https://webrecorder.net website changes

Not moving org import over to a background job yet as we hadn't firmly decided to do that and it seems likely to take more time to figure out mounting the JSON to import into the background job container than I wanted to spend on this for this sprint.

oid, backend_image, pull_policy, job_id, job_type=BgJobType.DELETE_ORG.value
)

async def run_recalculate_org_stats_job(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should perhaps make a more generic run_bg_job here if most of the params are the same - but can be added in a later refactor as we add more jobs, so shouldn't hold off merge.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see, I guess there already is a more generic method, maybe not needed at all, we'll see!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes there's a generic private method already, the public methods here really just set the job id to what we want it to be.

@tw4l tw4l merged commit ba5ca3f into main Nov 19, 2024
4 checks passed
@tw4l tw4l deleted the issue-2112-additional-bg-job-tasks branch November 19, 2024 22:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move additional long-running tasks to background jobs
2 participants