Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find a better way to update parents size #46

Open
unmade opened this issue Oct 19, 2021 · 2 comments
Open

Find a better way to update parents size #46

unmade opened this issue Oct 19, 2021 · 2 comments
Labels
✪ tech Refactoring, code quality, etc...

Comments

@unmade
Copy link
Owner

unmade commented Oct 19, 2021

We need to find a better way to store folder size.

Current flow is to update all parents on moving/uploading a file to folder. This however leads to concurrent transactions updating the same parents.

@unmade unmade added the ✪ tech Refactoring, code quality, etc... label Oct 19, 2021
@unmade unmade added this to the MVP milestone Oct 19, 2021
@unmade
Copy link
Owner Author

unmade commented Jan 30, 2022

Current reconciliation doesn't calculate folder sizes. This is because of performance optimisation. However, after reconciliation we can restore folder size with query like this:

WITH Parent := File,
UPDATE File
FILTER
    .mediatype.name = 'application/directory'
SET {
    size := (SELECT sum((
        SELECT Parent { size }
        FILTER
            Parent.namespace = File.namespace
            AND
            Parent.path LIKE File.path ++ '/%'
            AND
            Parent.mediatype.name != 'application/directory'
    ).size))
};

@unmade
Copy link
Owner Author

unmade commented Jul 13, 2022

One way to solve the problem is to store folder size in Redis using incr_by. That way we can avoid problems with transactions and concurrent updates.

Initial data for cache can be calculated using query above.

@unmade unmade removed this from the MVP milestone Aug 6, 2022
@unmade unmade changed the title Keep folder size consistent Find a better way to update parents size Aug 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✪ tech Refactoring, code quality, etc...
Projects
None yet
Development

No branches or pull requests

1 participant