expand on architecture docs for vfs

.github/workflows/build.yaml

@@ -11,11 +11,11 @@ jobs:
       - uses: cachix/cachix-action@v12
         with:
           name: rose
-          authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
+          authToken: "${{ secrets.CACHIX_AUTH_TOKEN }}"
       - name: Build Nix
         run: nix build -j8 .#devShells.x86_64-linux.default
       - name: Typecheck
-        if: success() || failure()  # Means that we run all steps even if one fails.
+        if: success() || failure() # Means that we run all steps even if one fails.
         run: nix develop --command make typecheck
       - name: Test
         if: success() || failure()

docs/ARCHITECTURE.md

@@ -81,6 +81,72 @@ queries to batch the writes.
 The update process is also parallelizable, so we shard workloads across
 multiple processes.
 
+# Virtual Filesystem
+
+We use the `llfuse` library for the virtual filesystem, as it's the only
+reasonable choice for Python. `llfuse` is a fairly low-level library for FUSE
+though, leaving us to, for example, work with inodes and all of their problems.
+
+Because of this, we split the Virtual Filesystem code into two main layers:
+
+1. `RoseLogicalFS`: A logical core of Rosé's domain logic, which exposes
+   syscall-like functions at a higher semantic level.
+2. `VirtualFS`: A wrapper around `RoseLogicalFS` that translates syscalls into
+   Rosé's semantics, manages inodes, caches for performance, and "lies" a
+   little so that user tools work transparently.
+
+There are a few other useful classes as well. Because the virtual filesystem
+interface is object oriented, the entire virtual filesystem module is organized
+following object orientation principles. We favor composition over inheritance
+^.~
+
+## Ghost Files
+
+While Rosé converts a music library into a virtual filesystem, some of its
+behaviors do not truly follow the conventions and expectations of a filesystem.
+For example, filenames are very rigid.
+
+For the most part, Rosé is a read only hierarchy, which still aligns with the
+expectations of other tools. However, there are two operations which diverge
+significantly from conventional expectations:
+
+1. Adding a release to a collage
+2. Adding a track to a playlist
+
+Let's start with adding a release to a collage. When we add a release to a
+collage, we take the `mkdir` call and translate that to an
+`add_release_to_collage` call. Afterwards, the release is in the collage. And
+the tracks of the release magically appear inside the directory we just
+`mkdir`-ed, since that directory is simply the release.
+
+But we don't usually call `mkdir`. We call `cp -r "1. Releases/x" "8. Collages/Road Trip/"`.
+And so we don't stop at a `mkdir`, we continue to attempt to `create` every
+single track in the release. Which is problematic, as after the `mkdir` call,
+all the files already exist in the directory.
+
+Strictly speaking, nothing's wrong. `cp -r` errors out, but the intended effect
+has been achieved. However, it's quite jarring for the conventional tools to
+produce errors and warnings when undertaking a happy path action. So, how can
+we silence those errors and warnings?
+
+We do so with "Ghost Files." We pretend that files exist (or don't exist!) for
+2-5 seconds after one of the above two operations occur. In the case of
+collages, the `VirtualFS` layer simply pretends that the directory is empty for
+5 seconds, and redirects any incoming writes into the directory to `/dev/null`.
+
+And in the case of playlists, where we copy a track into a playlist directory,
+we do not have a directory magically appearing. We actually store the data
+written into an in-memory dictionary, and upon release, we read that data to
+parse out the `roseid` tag. However, once we invoke `add_track_to_playlist`,
+the filename and inode of the new track will differ from the one passed to
+the `create` call. To the tool interacting with Rosé, it appears as if the file
+they just wrote vanished.
+
+We also solve playlists with Ghost Files: We pretend that a written playlist
+track exists for 2 seconds before we delete it. This way, a command like
+`cp --preserve=mode` will have time to replicate file attributes after copying
+(though we do nothing with them :p).
+
 # Logging
 
 Logs are written to stderr and to `${XDG_STATE_HOME:-$HOME/.local/state}/rose/rose.log`.

docs/METADATA_MANAGEMENT.md

@@ -250,4 +250,3 @@ from additional fields.
 | Track Number  | `tracknumber`  |                                                                                                                 |
 | Disc Number   | `discnumber`   |                                                                                                                 |
 | Rose ID       | `roseid`       |                                                                                                                 |
-

docs/PLAYLISTS_COLLAGES.md

@@ -133,12 +133,6 @@ $ rose playlists add-track "Evening" "018b6514-6fb7-7cc6-9d23-8eaf0b1beee8"
 
 Virtual filesystem:
 
-_When copying a release directory, there will be errors if `cp` is used. They
-are safe to ignore. The error happens because the action of creating the
-directory adds the release to the collage. After that point, all files are
-already part of the release directory, yet `cp` attempts to copy the files over
-too. We will try to fix this later._
-
 ```bash
 $ cd $fuse_mount_dir
 

docs/README.md

@@ -8,3 +8,5 @@ For more detailed documentation, please visit the following files:
 - [Using Playlists & Collages](./PLAYLISTS_COLLAGES.md)
 - [Maintaining the Cache](./CACHE_MAINTENANCE.md)
 - [Architecture](./ARCHITECTURE.md)
+
+The code is also commented somewhat decently, if you'd like to read the source.

docs/VIRTUAL_FILESYSTEM.md

@@ -158,7 +158,6 @@ _Deletion will move the release into the trashbin, following the
 [freedesktop spec](https://freedesktop.org/wiki/Specifications/trash-spec/).
 The release can be restored later if the deletion was accidental._
 
-
 Command line:
 
 ```bash