Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Admin S3 Object Deletion - Build a process to populate the Storage image field value for existing assets #1593

Closed
jkueloc opened this issue Jan 24, 2022 · 0 comments
Assignees
Labels

Comments

@jkueloc
Copy link
Collaborator

jkueloc commented Jan 24, 2022

This ticket continues work of #1575 (also related to #1573 and #1574).

How to handle populating the Asset Storage image new field?

  • Using a "data" migration to populate the field kept getting killed by the OS less than 1.5 hours into the process - most likely due to memory trying to update ~700k records. zsh: killed pipenv run ./manage.py migrate.

    • After consultation - even if the this code was modified to chunk the updates - as a "data" migration - the way Django handles it - may still not run fast enough for the application start up process to not loose connection.
  • For local development - built sql and directly update set the value of the asset storage_image database field. This approach is possible but not recommended for production.

  • Looking at creating a management command and/or "one off" celery task based on "data" migration logic @ queryset level.

@jkueloc jkueloc self-assigned this Jan 24, 2022
@lalgee lalgee added the database label Feb 2, 2022
@elainekamlley elainekamlley changed the title Build a process to populate the Storage image field value for existing assets Admin S3 Object Deletion - Build a process to populate the Storage image field value for existing assets Feb 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants