expand on architecture docs for vfs

.github/workflows/build.yaml

@@ -11,11 +11,11 @@ jobs:
       - uses: cachix/cachix-action@v12
         with:
           name: rose
-          authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
+          authToken: "${{ secrets.CACHIX_AUTH_TOKEN }}"
       - name: Build Nix
         run: nix build -j8 .#devShells.x86_64-linux.default
       - name: Typecheck
-        if: success() || failure()  # Means that we run all steps even if one fails.
+        if: success() || failure() # Means that we run all steps even if one fails.
         run: nix develop --command make typecheck
       - name: Test
         if: success() || failure()

docs/ARCHITECTURE.md

@@ -81,6 +81,60 @@ queries to batch the writes.
 The update process is also parallelizable, so we shard workloads across
 multiple processes.
 
+# Virtual Filesystem
+
+We use the `llfuse` library for the virtual filesystem. `llfuse` is a fairly
+low-level library for FUSE, which leaves us to work with system-level concepts
+such as inodes.
+
+Though these concepts have much to do with filesystems, they have little to do
+with Rosé. Thus, we have split the Virtual Filesystem code into two main layers:
+
+1. `RoseLogicalCore`: A logical core of Rosé's domain logic, which exposes
+   syscall-like functions at a higher semantic level.
+2. `VirtualFS`: A wrapper around `RoseLogicalFS` that translates syscalls into
+   Rosé's semantics, manages inodes, caches for performance, and "lies" a
+   little so that user tools work transparently.
+
+There are a few other useful classes as well. Because the virtual filesystem
+interface is object oriented, the entire virtual filesystem module is organized
+following object orientation principles. We favor composition over inheritance
+^.~
+
+## Ghost Files
+
+Some of Rosé operations break the conventional expectations of a filesystem.
+Most operations are still _decent_, but the following two diverge pretty
+heavily:
+
+1. Adding a release to a collage
+2. Adding a track to a playlist
+
+When we add a release to a collage, we take the `mkdir` call and translate that
+to an `add_release_to_collage` call. Afterwards, the release is in the collage.
+And the tracks of the release magically appear inside the directory afterwards,
+since that directory is simply the release.
+
+However, a command like `cp -r` immediately attempts to then replicate files
+into a supposedly empty directory, and errors. Strictly speaking, nothing's
+wrong. Though `cp` errors, the release has been added to the collage, and the
+next `ls` command shows that the release has been successfully added to a
+collage. But it is quite nice when basic tools work without erroring!
+
+Thus, "Ghost Files." In order to keep tools like `cp` happy, we pretend that
+files exist (or don't exist!) for 2-5 seconds after one of the above two
+operations occur. In the case of collages, the `VirtualFS` layer pretends that
+the new directory is empty for 5 seconds, and redirects any incoming writes
+into the directory to `/dev/null`.
+
+For playlists, the problem arises in that the copied file vanishes immediately
+once the file handle is released. A command like `cp --preserve=mode`, which
+attempts to replicate file attributes after releasing the file, fails.
+
+So in the playlists case, we pretend that the written file exists for 2 seconds
+after we wrote it. This way, `cp --preserve=mode` can replicate its attributes
+onto a ghost file and happily exit without errors.
+
 # Logging
 
 Logs are written to stderr and to `${XDG_STATE_HOME:-$HOME/.local/state}/rose/rose.log`.

docs/METADATA_MANAGEMENT.md

@@ -250,4 +250,3 @@ from additional fields.
 | Track Number  | `tracknumber`  |                                                                                                                 |
 | Disc Number   | `discnumber`   |                                                                                                                 |
 | Rose ID       | `roseid`       |                                                                                                                 |
-

docs/PLAYLISTS_COLLAGES.md

@@ -133,12 +133,6 @@ $ rose playlists add-track "Evening" "018b6514-6fb7-7cc6-9d23-8eaf0b1beee8"
 
 Virtual filesystem:
 
-_When copying a release directory, there will be errors if `cp` is used. They
-are safe to ignore. The error happens because the action of creating the
-directory adds the release to the collage. After that point, all files are
-already part of the release directory, yet `cp` attempts to copy the files over
-too. We will try to fix this later._
-
 ```bash
 $ cd $fuse_mount_dir
 

docs/README.md

@@ -8,3 +8,5 @@ For more detailed documentation, please visit the following files:
 - [Using Playlists & Collages](./PLAYLISTS_COLLAGES.md)
 - [Maintaining the Cache](./CACHE_MAINTENANCE.md)
 - [Architecture](./ARCHITECTURE.md)
+
+The code is also commented somewhat decently, if you'd like to read the source.

docs/VIRTUAL_FILESYSTEM.md

@@ -158,7 +158,6 @@ _Deletion will move the release into the trashbin, following the
 [freedesktop spec](https://freedesktop.org/wiki/Specifications/trash-spec/).
 The release can be restored later if the deletion was accidental._
 
-
 Command line:
 
 ```bash

rose/virtualfs.py

@@ -336,8 +336,15 @@ def __init__(self) -> None:
         # Fake sentinel for file handler. The VirtualFS class implements this file handle as a black
         # hole.
         self.dev_null = 9
-        # We translate Rose's Virtual Filesystem file handles to the host machine file handles. We
-        # don't simply pass file handles through in order to avoid collision problems.
+        # We translate Rose's Virtual Filesystem file handles to the host machine file handles. This
+        # means that every file handle from the host system has a corresponding "wrapper" file
+        # handle in Rose, and we only return Rose's file handles from the virtual fs.
+        #
+        # When we receive a Rose file handle that maps to a host filesystem operation, we "unwrap"
+        # the file handle back to the host file handle, and then use it.
+        #
+        # This prevents any accidental collisions, where Rose generates a file handle that ends up
+        # being the same number as a file handle that the host system generates.
         self._rose_to_host_map: dict[int, int] = {}
 
     def next(self) -> int:
@@ -370,7 +377,7 @@ def unwrap_host(self, rose_fh: int) -> int:
 ]
 
 
-class RoseLogicalFS:
+class RoseLogicalCore:
     def __init__(self, config: Config, fhandler: FileHandleManager):
         self.config = config
         self.fhandler = fhandler
@@ -933,7 +940,7 @@ class VirtualFS(llfuse.Operations):  # type: ignore
 
     def __init__(self, config: Config):
         self.fhandler = FileHandleManager()
-        self.rose = RoseLogicalFS(config, self.fhandler)
+        self.rose = RoseLogicalCore(config, self.fhandler)
         self.inodes = INodeManager(config)
         self.default_attrs = {
             # Well, this should be ok for now. I really don't want to track this... we indeed change