Skip to content

Commit

Permalink
write working with cache
Browse files Browse the repository at this point in the history
  • Loading branch information
azuline committed Oct 26, 2023
1 parent 6c8f8b0 commit 49386b2
Show file tree
Hide file tree
Showing 2 changed files with 67 additions and 27 deletions.
19 changes: 11 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -305,14 +305,17 @@ For additional documentation, please read the following files:
```
Copyright 2023 blissful <blissful@sunsetglow.net>
Licensed under the Apache License, Version 2.0 (the "License"); you may not use
this file except in compliance with the License. You may obtain a copy of the
License at http://www.apache.org/licenses/LICENSE-2.0.
Unless required by applicable law or agreed to in writing, software distributed
under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
```

## Contributions
Expand Down
75 changes: 56 additions & 19 deletions docs/CACHE_MAINTENANCE.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,58 @@
# Maintaining the Cache

For performance, Rosé stores a copy of every source file's metadata in a SQLite
read cache. The read cache does not accept writes; thus it can always be fully
recreated from the source files. It can be freely deleted and recreated without
consequence.

The cache can be updated with the command `rose cache update`. By default, the
cache updater will only recheck files that have changed since the last run. To
override this behavior and always re-read file tags, run `rose cache update
--force`.

By default, the cache is updated on `rose fs mount` and when files are changed
through the virtual filesystem. However, if the `music_source_dir` is changed
directly, Rosé does not automatically update the cache, which can lead to cache
drifts.

You can solve this problem by running `rose cache watch`. This starts a watcher
that triggers a cache update whenever a source file changes. This can be useful
if you synchronize your music library between two computers, or use another
tool to directly modify the `music_source_dir`.
The read cache is a SQLite database that replicates the metadata in
`music_source_dir`. The read cache exists to improve performance: it can be
read from far more performantly than the `music_source_dir` can.

The read cache is never written to directly, outside of the `update_cache_*`
functions, which re-read the source files and write their metadata into the
cache. All mutations in the system occur directly to the source files, not to
the read cache.

So what's the problem?

The read cache has the possibility of _drifting_ from the source directory. For
example, let's say that I move files around in the source directory, modify
tags directly, or even delete a release. After those actions, the read cache
will still reflect the previous state, and thus we say that the cache has
_drifted_.

This is problematic because some operations in the virtual filesystem will
begin to fail. For example, if a file is moved in the source directory, and the
virtual filesystem then attempts to read from its previous path, it will hit a
FileNotFound (ENOENT) error.

Thus, after changes to the source directory, we need to update the cache so
that it _synchronizes_ with the source directory. Note that this
synchronization is entirely _one-way_: the source files update the read cache.
The opposite direction never occurs: the read cache never updates the source
files.

The cache can be updated with the command `rose cache update`. By default, this
command only checks files which have changed since the last cache update. It
uses the mtime (last modified) field for this purpose. However, sometimes we
want to refresh the cache regardless of mtimes. In that case, we can run `rose
cache update --force`.

It would be pretty annoying if you had to run this command by hand after each
metadata update. So Rosé will automatically run this command whenever an update
happens _through_ Rosé. That means:

- If a file is modified in the virtual filesystem, a cache update is
triggered when the file handle closes.
- If a release is modified by the command line, a cache update is triggered at
the end of the command.

Rosé will also update the cache on virtual filesystem mount.

However, there is one class of updates that this does not update in response
to, and that is updates made by external tools directly to the source
directory. If another system updates your source directory directly, the read
cache will drift.

To update in response to those external changes, you can run `rose cache
watch`. This command starts a watcher that triggers a cache update whenever a
file changes in the source directory. `rose cache watch` runs in the
foreground, so it is recommended that you manage it with a service manager like
systemd. See [Configuration](./CONFIGURATION.md) for example systemd unit
files.

0 comments on commit 49386b2

Please sign in to comment.