diff --git a/README.md b/README.md index 4fadc80..8e91009 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,62 @@ # Rosé -_Work in Progress. See [Issue #1](https://github.com/azuline/rose/issues/1) for -the current state._ +> [!IMPORTANT] +> Rosé is under active development. See [Issue #1](https://github.com/azuline/rose/issues/1) +> for progress updates. -**WARNING: Rosé modifies your audio files. If you do not want to modify your -audio files, you should not use Rosé.** +Rosé is a music manager for Unix-based systems. Rosé provides a virtual FUSE +filesystem for managing your music library and various functions for editing +and improving your music library's metadata and tags. -A virtual filesystem for music and metadata improvement tooling. +> [!NOTE] +> Rosé modifies the managed audio files. If you do not want to modify your +> audio files, for example because they are seeding in a bittorrent client, you +> should not use Rosé. + +TODO: Video + +## Installation + +Install Rosé with Nix Flakes. If you do not have Nix Flakes, you can install it +with [this installer](https://github.com/DeterminateSystems/nix-installer). + +```bash +$ nix profile install github:azuline/rose#rose +``` + +In the future, other packaging systems may be considered. However, I strongly +dislike Python's packaging story, hence: Nix. + +## Quickstart + +TODO + +## Features + +TODO + +## Requirements + +TODO + +## License + +Copyright 2023 blissful + +Licensed under the Apache License, Version 2.0 (the "License"); you may not use +this file except in compliance with the License. You may obtain a copy of the +License at http://www.apache.org/licenses/LICENSE-2.0. + +Unless required by applicable law or agreed to in writing, software distributed +under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR +CONDITIONS OF ANY KIND, either express or implied. See the License for the +specific language governing permissions and limitations under the License. + +## Contributions + +TODO + +# OLD ## The Virtual Filesystem @@ -92,16 +142,6 @@ manager! I have yet to write this part of the tool. Please check back later! -# Installation - -Install with Nix Flakes: - -```bash -$ nix profile install github:azuline/rose#rose -``` - -You can install Nix and Nix Flakes with -[this installer](https://github.com/DeterminateSystems/nix-installer). # Usage @@ -123,75 +163,6 @@ Commands: playlists Manage playlists. ``` -## Configuration - -Rosé must be configured prior to use. Rosé is configured via a TOML file -located at `${XDG_CONFIG_HOME:-$HOME/.config}/rose/config.toml`. - -The configuration parameters, with examples, are: - -```toml -# === Required values === - -# The directory containing the music to manage. This source directory WILL be -# modified by Rosé; if you do not want the files in directory to be modified, -# use another tool! -music_source_dir = "~/.music-source" -# The directory to mount the library's virtual filesystem on. This is the -# primary "user interface" of Rosé, and the directory in which you will be able -# to browse your music library. -fuse_mount_dir = "~/music" - -# === Optional values === - -# The directory to write the cache to. Defaults to -# `${XDG_CACHE_HOME:-$HOME/.cache}/rose`. -cache_dir = "~/.cache/rose" -# Maximum parallel processes that the cache updater can spawn. Defaults to -# nproc/2. The higher this number is; the more performant the cache update will -# be. -max_proc = 4 -# Artist aliases: Releases belonging to an alias will also "belong" to the main -# artist. This option improves the Artist browsing view by showing the aliased -# releases in the main artist's releases list. -artist_aliases = [ - { artist = "Abakus", aliases = ["Cinnamon Chasers"] }, - { artist = "tripleS", aliases = ["EVOLution", "LOVElution", "+(KR)ystal Eyes", "Acid Angel From Asia", "Acid Eyes"] }, -] -# Artists, genres, and labels to show in the virtual filesystem navigation. By -# default, all artists, genres, and labels are shown. However, these values can -# be used to filter the listed values to a specific few. This is useful e.g. if -# you only care to browse your favorite genres and labels. -fuse_artists_whitelist = [ "xxx", "yyy" ] -fuse_genres_whitelist = [ "xxx", "yyy" ] -fuse_labels_whitelist = [ "xxx", "yyy" ] -# Artists, genres, and labels to hide from the virtual filesystem navigation. -# These options remove specific entities from the default policy of listing all -# entities. These options are mutually exclusive with the fuse_*_whitelist -# options; if both are specified for a given entity type, the configuration -# will not validate. -fuse_artists_blacklist = [ "xxx" ] -fuse_genres_blacklist = [ "xxx" ] -fuse_labels_blacklist = [ "xxx" ] -``` - -The `--config/-c` flag overrides the config location. - -## Music Source Dir - -The `music_source_dir` must be a flat directory of releases, meaning all releases -must be top-level directories inside `music_source_dir`. Each release should also -be a single directory in `music_source_dir`. - -Every directory should follow the format: `$music_source_dir/$release_name/**/$track.mp3`. -A release can have arbitrarily many nested subdirectories. - -So for example: `$music_source_dir/BLACKPINK - 2016. SQUARE ONE/*.mp3`. - -Rosé writes playlist and collage files to the `$music_source_dir/.playlists` -and `$music_source_dir/.collages` directories. Each file is a human-readable -TOML file. - ## Supported Filetypes Rosé supports `.mp3`, `.m4a`, `.ogg` (vorbis), `.opus`, and `.flac` audio files. @@ -213,116 +184,6 @@ TODO TODO -## Data Querying - -There are several commands that print out data from the read cache in a -JSON-encoded format (e.g. `rose releases print` and `rose collages print`). The -command output can be piped into tools like `jq`, `fx`, and others. - -## Tagging Conventions - -Rosé is lenient in the tags it ingests, but has opinionated conventions for the -tags it writes. - -### Multi-Valued Tags - -Rosé supports multiple values for the artists, genres, and labels tags. Rosé -uses the `;` character as a tag delimiter. For example, `genre=Deep House;Techno`. - -Rosé also preserves the artists' role in the artist tag by using specialized -delimiters. for example, `artist=Pyotr Ilyich Tchaikovsky performed by André Previn;London Symphony Orchestra feat. Barack Obama`. - -The artist tag is described by the following grammar: - -``` -artist-tag ::= composer dj main guest remixer producer -composer ::= name ' performed by ' -dj ::= name ' pres. ' -main ::= name -guest ::= ' feat. ' name -remixer ::= ' remixed by ' name -producer ::= ' produced by ' name -name ::= string ';' name | string -``` - -## New Releases - -TODO - -## Artist Aliases - -TODO - -## The Read Cache - -For performance, Rosé stores a copy of every source file's metadata in a SQLite -read cache. The read cache does not accept writes; thus it can always be fully -recreated from the source files. It can be freely deleted and recreated without -consequence. - -The cache can be updated with the command `rose cache update`. By default, the -cache updater will only recheck files that have changed since the last run. To -override this behavior and always re-read file tags, run `rose cache update ---force`. - -By default, the cache is updated on `rose fs mount` and when files are changed -through the virtual filesystem. However, if the `music_source_dir` is changed -directly, Rosé does not automatically update the cache, which can lead to cache -drifts. - -You can solve this problem by running `rose cache watch`. This starts a watcher -that triggers a cache update whenever a source file changes. This can be useful -if you synchronize your music library between two computers, or use another -tool to directly modify the `music_source_dir`. - ## Systemd Unit Files TODO; example unit files to schedule Rosé with systemd. - -## Logging - -Logs are written to stderr and to `${XDG_STATE_HOME:-$HOME/.local/state}/rose/rose.log`. - -# Architecture - -Rosé has a simple uni-directional looping architecture. - -1. The source files: audio+playlists+collages, are the single source of truth - for your music library. -2. The read cache is transient and deterministically derived from source - files. It can always be deleted and fully recreated from source files. -3. The virtual filesystem uses the read cache (for performance). Writes to the - virtual filesystem update the source files and then refresh the read cache. -4. The metadata tooling writes to the source files directly, which in turn - refreshes the read cache. The metadata tooling reads from the read cache for - performance. - -```mermaid -flowchart BT - S[Source Files] - C[Read Cache] - M[Metadata Tooling] - V[Virtual Filesystem] - S -->|Populates| C - M -->|Reads| C - M -->|Writes| S - V -->|Writes| S - V -->|Reads| C -``` - -This architecture takes care to ensure that there is a single source of truth -and uni-directional mutations. This has a few benefits: - -- Rosé and the source files always have the same metadata. If they drift, `rose - cache update` will rebuild the cache such that it fully matches the source - files. And if `rose cache watch` is running, there should not be any drift. -- Rosé is easily synchronized across machines. As long as the source - files are synchronized, Rosé will rebuild the exact same cache regardless of - machine. - -Rosé writes `.rose.{uuid}.toml` files into each release's directory as a way to -preserve release-level state and keep release UUIDs consistent across full -cache rebuilds. - -Tracks are uniquely identified by the `(release_uuid, discnumber, tracknumber)` -tuple, which are also consistent across rebuilds. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..28eda49 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,66 @@ +# Architecture + +Rosé has a simple uni-directional looping architecture. + +```mermaid +flowchart BT + S[Source Files] + C[Read Cache] + M[Metadata Tooling] + V[Virtual Filesystem] + S -->|Populates| C + M -->|Reads| C + M -->|Writes| S + V -->|Writes| S + V -->|Reads| C +``` + +1. The source audio files, playlist files, and collage files are single sources + of truth. All writes are made directly to the sources files. +2. The read cache is transient and deterministically derived from source + files. It can always be deleted and fully recreated from source files. It + updates in response to changes in the source files. +3. The virtual filesystem uses the read cache for performance. All writes made + via the virtual filesystem are made to the Source Files, which in turn + refreshes the read cache. +4. The metadata tooling uses the read cache for performance, but always writes + to the source files directly, which in turn refreshes the read cache. + +This architecture takes care to ensure that there is a single source of truth +and uni-directional mutations. This means that Rosé and the source files always +have the same metadata. If the source files change, `rose cache update` is +guaranteed to rebuild the cache such that it fully matches the source files. +And if `rose cache watch` is running, the cache updates should happen +automatically. + +This has some nice consequences: + +- External tag editing tools do not disrupt Rosé. If an external tool modifies + the audio tags, Rosé's cache can always update to match the newly modified + source files. +- Rosé is easily synchronized across machines, for example with a tool like + Syncthing. As long as the source files are in-sync, Rosé's read cache will + match. +- An inconsistent state between source files and Rosé is trivially resolved. + This is different from music managers that retain a separate database, + because conflict resolution is then ambiguous. Whereas in Rosé, there are no + conflicts. + +## Stable Release & Track Identifiers + +Rosé assigns UUIDs to each release and track in order to identify them across +arbitrarily large metadata changes. These UUIDs are persisted to the source +files. + +- Each release has a `.rose.{uuid}.toml` file, which preserves release-level + state, such as `New`. The UUID is in the filename instead of the file + contents for improved performance: we can collect the UUID via a `readdir` + call instead of an expensive file read. +- Each track has a custom `roseid` tag. This tag is written to the source audio + file. Read the `tagger.py` file for the exact field name used. + +## Logging + +Logs are written to stderr and to `${XDG_STATE_HOME:-$HOME/.local/state}/rose/rose.log`. +Debug logging can be turned on with the `--verbose/-v` option. Rosé is heavily +instrumented with debug logging. diff --git a/docs/CACHE_MAINTENANCE.md b/docs/CACHE_MAINTENANCE.md new file mode 100644 index 0000000..9939e98 --- /dev/null +++ b/docs/CACHE_MAINTENANCE.md @@ -0,0 +1,21 @@ +# Maintaining the Cache + +For performance, Rosé stores a copy of every source file's metadata in a SQLite +read cache. The read cache does not accept writes; thus it can always be fully +recreated from the source files. It can be freely deleted and recreated without +consequence. + +The cache can be updated with the command `rose cache update`. By default, the +cache updater will only recheck files that have changed since the last run. To +override this behavior and always re-read file tags, run `rose cache update +--force`. + +By default, the cache is updated on `rose fs mount` and when files are changed +through the virtual filesystem. However, if the `music_source_dir` is changed +directly, Rosé does not automatically update the cache, which can lead to cache +drifts. + +You can solve this problem by running `rose cache watch`. This starts a watcher +that triggers a cache update whenever a source file changes. This can be useful +if you synchronize your music library between two computers, or use another +tool to directly modify the `music_source_dir`. diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md new file mode 100644 index 0000000..13672d9 --- /dev/null +++ b/docs/CONFIGURATION.md @@ -0,0 +1,69 @@ +# Configuration + +Rosé must be configured prior to use. Rosé is configured via a TOML file +located at `${XDG_CONFIG_HOME:-$HOME/.config}/rose/config.toml`. + +The configuration parameters, with examples, are: + +```toml +# === Required values === + +# The directory containing the music to manage. This source directory WILL be +# modified by Rosé; if you do not want the files in directory to be modified, +# use another tool! +music_source_dir = "~/.music-source" +# The directory to mount the library's virtual filesystem on. This is the +# primary "user interface" of Rosé, and the directory in which you will be able +# to browse your music library. +fuse_mount_dir = "~/music" + +# === Optional values === + +# The directory to write the cache to. Defaults to +# `${XDG_CACHE_HOME:-$HOME/.cache}/rose`. +cache_dir = "~/.cache/rose" +# Maximum parallel processes that the cache updater can spawn. Defaults to +# nproc/2. The higher this number is; the more performant the cache update will +# be. +max_proc = 4 +# Artist aliases: Releases belonging to an alias will also "belong" to the main +# artist. This option improves the Artist browsing view by showing the aliased +# releases in the main artist's releases list. +artist_aliases = [ + { artist = "Abakus", aliases = ["Cinnamon Chasers"] }, + { artist = "tripleS", aliases = ["EVOLution", "LOVElution", "+(KR)ystal Eyes", "Acid Angel From Asia", "Acid Eyes"] }, +] +# Artists, genres, and labels to show in the virtual filesystem navigation. By +# default, all artists, genres, and labels are shown. However, these values can +# be used to filter the listed values to a specific few. This is useful e.g. if +# you only care to browse your favorite genres and labels. +fuse_artists_whitelist = [ "xxx", "yyy" ] +fuse_genres_whitelist = [ "xxx", "yyy" ] +fuse_labels_whitelist = [ "xxx", "yyy" ] +# Artists, genres, and labels to hide from the virtual filesystem navigation. +# These options remove specific entities from the default policy of listing all +# entities. These options are mutually exclusive with the fuse_*_whitelist +# options; if both are specified for a given entity type, the configuration +# will not validate. +fuse_artists_blacklist = [ "xxx" ] +fuse_genres_blacklist = [ "xxx" ] +fuse_labels_blacklist = [ "xxx" ] +``` + +The `--config/-c` flag overrides the config location. + +## Music Source Dir + +The `music_source_dir` must be a flat directory of releases, meaning all releases +must be top-level directories inside `music_source_dir`. Each release should also +be a single directory in `music_source_dir`. + +Every directory should follow the format: `$music_source_dir/$release_name/**/$track.mp3`. +A release can have arbitrarily many nested subdirectories. + +So for example: `$music_source_dir/BLACKPINK - 2016. SQUARE ONE/*.mp3`. + +Rosé writes playlist and collage files to the `$music_source_dir/.playlists` +and `$music_source_dir/.collages` directories. Each file is a human-readable +TOML file. + diff --git a/docs/METADATA_MANAGEMENT.md b/docs/METADATA_MANAGEMENT.md new file mode 100644 index 0000000..6ddf652 --- /dev/null +++ b/docs/METADATA_MANAGEMENT.md @@ -0,0 +1,41 @@ +# Managing Your Music Metadata + +## Data Querying + +There are several commands that print out data from the read cache in a +JSON-encoded format (e.g. `rose releases print` and `rose collages print`). The +command output can be piped into tools like `jq`, `fx`, and others. + +## Tagging Conventions + +Rosé is lenient in the tags it ingests, but has opinionated conventions for the +tags it writes. + +### Fields + +TODO + +### Multi-Valued Tags + +Rosé supports multiple values for the artists, genres, and labels tags. Rosé +writes a single tag field and with fields concatenated together with a `;` +delimiter. For example, `genre=Deep House;Techno`. Rosé does not write one tag +per frame due to inconsistent support by other useful programs. + +### Artist Tags + +Rosé preserves the artists' role in the artist tag by using specialized +delimiters. An example artist tag is: `Pyotr Ilyich Tchaikovsky performed by André Previn;London Symphony Orchestra feat. Barack Obama`. + +The artist tag is described by the following grammar: + +``` + ::=
+ ::= ' performed by ' + ::= ' pres. ' +
::= + ::= ' feat. ' + ::= ' remixed by ' + ::= ' produced by ' + ::= string ';' | string +``` diff --git a/docs/PLAYLISTS_COLLAGES.md b/docs/PLAYLISTS_COLLAGES.md new file mode 100644 index 0000000..bdfc5fa --- /dev/null +++ b/docs/PLAYLISTS_COLLAGES.md @@ -0,0 +1 @@ +# Using Playlists & Collages diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..b52935c --- /dev/null +++ b/docs/README.md @@ -0,0 +1,10 @@ +For an introduction to Rosé, please visit the [README](../README.md). + +For more detailed documentation, please visit the following files: + +- [Configuration](./CONFIGURATION.md) +- [Browsing with the Virtual Filesystem](./VIRTUAL_FILESYSTEM.md) +- [Managing Your Music Metadata](./METADATA_MANAGEMENT.md) +- [Using Playlists & Collages](./PLAYLISTS_COLLAGES.md) +- [Maintaining the Cache](./CACHE_MAINTENANCE.md) +- [Architecture](./ARCHITECTURE.md) diff --git a/docs/VIRTUAL_FILESYSTEM.md b/docs/VIRTUAL_FILESYSTEM.md new file mode 100644 index 0000000..bc9bdf1 --- /dev/null +++ b/docs/VIRTUAL_FILESYSTEM.md @@ -0,0 +1 @@ +# Browsing the Virtual Filesystem