Skip to content

Commit

Permalink
additional edits and language improvements to the docs
Browse files Browse the repository at this point in the history
  • Loading branch information
azuline committed Oct 27, 2023
1 parent 0e7f9f6 commit 34054bf
Show file tree
Hide file tree
Showing 7 changed files with 167 additions and 151 deletions.
68 changes: 42 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,14 +53,14 @@ virtual/
│ │   ├── 02. BLACKPINK - BOOMBAYAH.opus
│ │   └── cover.jpg
│   ├── BLACKPINK - 2016. SQUARE TWO - Single [Dance-Pop;K-Pop]/...
│   ├── LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [K-Pop]/...
│   ├── LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [Dance-Pop;Future Bass;K-Pop]/...
│   ├── NewJeans - 2022. Ditto - Single [Contemporary R&B;K-Pop]/...
│   └── {NEW} LOOΠΔ - 2017. Kim Lip - Single [Contemporary R&B;Dance-Pop;K-Pop]/...
├── 2. Releases - New/
│   └── {NEW} LOOΠΔ - 2017. Kim Lip - Single [Contemporary R&B;Dance-Pop;K-Pop]/...
├── 3. Releases - Recently Added/
│   ├── [2023-10-25] LOOΠΔ - 2017. Kim Lip - Single [Contemporary R&B;Dance-Pop;K-Pop]/...
│   ├── [2023-10-01] LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [K-Pop]/...
│   ├── [2023-10-01] LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [Dance-Pop;Future Bass;K-Pop]/...
│   └── [2023-02-28] NewJeans - 2022. Ditto - Single [Contemporary R&B;K-Pop]/...
│   ├── [2022-08-22] BLACKPINK - 2016. SQUARE TWO - Single [Dance-Pop;K-Pop]/...
│   ├── [2022-08-10] BLACKPINK - 2016. SQUARE ONE - Single [Big Room House;Dance-Pop;K-Pop]/...
Expand All @@ -70,9 +70,9 @@ virtual/
│   │   └── BLACKPINK - 2016. SQUARE TWO - Single [Dance-Pop;K-Pop]/...
│   ├── LOOΠΔ/
│   │   ├── {NEW} LOOΠΔ - 2017. Kim Lip - Single [Contemporary R&B;Dance-Pop;K-Pop]/...
│   │   └── LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [K-Pop]/...
│   │   └── LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [Dance-Pop;Future Bass;K-Pop]/...
│   ├── LOOΠΔ ODD EYE CIRCLE/
│   │   └── LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [K-Pop]/...
│   │   └── LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [Dance-Pop;Future Bass;K-Pop]/...
│   └── NewJeans/
│   └── NewJeans - 2022. Ditto - Single [Contemporary R&B;K-Pop]/...
├── 5. Genres/
Expand All @@ -81,28 +81,31 @@ virtual/
│   ├── Contemporary R&B/
│   │   ├── NewJeans - 2022. Ditto - Single [Contemporary R&B;K-Pop]/...
│   │   └── {NEW} LOOΠΔ - 2017. Kim Lip - Single [Contemporary R&B;Dance-Pop;K-Pop]/...
│   ├── Future Bass/
│   │ └── LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [Dance-Pop;Future Bass;K-Pop]/...
│   ├── Dance-Pop/
│   │   ├── BLACKPINK - 2016. SQUARE ONE - Single [Big Room House;Dance-Pop;K-Pop]/...
│   │   ├── BLACKPINK - 2016. SQUARE TWO - Single [Dance-Pop;K-Pop]/...
│   │ ├── LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [Dance-Pop;Future Bass;K-Pop]/...
│   │   └── {NEW} LOOΠΔ - 2017. Kim Lip - Single [Contemporary R&B;Dance-Pop;K-Pop]/...
│   └── K-Pop/
│   ├── BLACKPINK - 2016. SQUARE ONE - Single [Big Room House;Dance-Pop;K-Pop]/...
│   ├── BLACKPINK - 2016. SQUARE TWO - Single [Dance-Pop;K-Pop]/...
│   ├── LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [K-Pop]/...
│   ├── LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [Dance-Pop;Future Bass;K-Pop]/...
│   └── {NEW} LOOΠΔ - 2017. Kim Lip - Single [Contemporary R&B;Dance-Pop;K-Pop]/...
├── 6. Labels/
│   ├── ADOR/
│   │ └── NewJeans - 2022. Ditto - Single [Contemporary R&B;K-Pop]/...
│   ├── BlockBerry Creative/
│   │   ├── LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [K-Pop]/...
│   │   ├── LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [Dance-Pop;Future Bass;K-Pop]/...
│   │ └── {NEW} LOOΠΔ - 2017. Kim Lip - Single [Contemporary R&B;Dance-Pop;K-Pop]/...
│   └── YG Entertainment/
│   ├── BLACKPINK - 2016. SQUARE ONE - Single [Big Room House;Dance-Pop;K-Pop]/...
│   └── BLACKPINK - 2016. SQUARE TWO - Single [Dance-Pop;K-Pop]/...
├── 7. Collages/
│   └── Road Trip/
│   ├── 1. BLACKPINK - 2016. SQUARE TWO - Single [Dance-Pop;K-Pop]/...
│   └── 2. LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [K-Pop]/...
│   └── 2. LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [Dance-Pop;Future Bass;K-Pop]/...
└── 8. Playlists/
└── Shower/
├── 1. LOOΠΔ ODD EYE CIRCLE - Chaotic.opus
Expand All @@ -111,17 +114,19 @@ virtual/
└── 4. LOOΠΔ - Eclipse.opus
```

In addition to a flat directory of all releases, Rosé creates
additional directories based on Date Added, Artist, Genre, and Label. Rosé also
provides a few other features designed to keep your music library organized:
In addition to a flat directory of all releases, Rosé creates directories based
on Date Added, Artist, Genre, and Label. Rosé also provides a few other
concepts for organizing your music library:

- **Collages:** Collections of releases.
- **Playlists:** Collections of tracks.
- **New release tracking:** Track new unlistened additions to the library.

Rosé's virtual filesystem organizes your music library by the metadata in the
music tags. Thus, the quality of the virtual filesystem depends on the quality
of the tags, Rosé also provides functions for improving the tags of your music
music tags. The quality of the virtual filesystem depends on the quality of the
tags.

Thus, Rosé also provides functions for improving the tags of your music
library. Rosé provides an easy text-based interface for manually modifying
metadata, automatic metadata importing from third-party sources, and a rules
engine to automatically apply metadata changes based on patterns.
Expand Down Expand Up @@ -162,20 +167,31 @@ _Demo Video TBD_
- Support for `.mp3`, `.m4a`, `.ogg` (vorbis), `.opus`, and `.flac` audio
files.

# Is Rosé Right for Me?
# Is Rosé For You?

Rosé expects users to be comfortable with the shell and general software
concepts. Rosé assumes general technical competency in its documentation and
user interface.

Rosé provides several parts of a complete music system, but not a complete
music system. The user can then construct their own complete music system by
composing multiple programs. For example, Rosé+nnn+mpv.

Rosé creates the most value when used with a large music library. For small
libraries of several hundred releases or less, Rosé's organization is
unnecessary, as the entire library is easily browsed via a flat view over all
releases.

Rosé also makes some opinionated decisions. Please make sure that Rosé's
opinions are acceptable for you:
Rosé modifies files in the source directory, even as early as the first library
scan. All mutations in Rosé are persisted by writing to the source directory;
Rosé maintains no state of its own outside of the source directory. If your
music library is immutable, or should be treated as immutable, Rosé will not
work.

- Rosé modifies files in the source directory, even as early as the first
library scan. All mutations in Rosé are persisted by writing to the source
directory; Rosé maintains no state of its own outside of the source
directory. This makes Rosé work very poorly with torrents.
- Rosé expects all releases to be immediate child directories of the
source directory. And Rosé expects that all tracks belong to a "release"
(meaning an album, single, EP, etc.). This means that loose audio files at
the root of the source directory will be ignored. Thus, Rosé works very
poorly with collections of individual track files.
Rosé expects all releases to be immediate child directories of the source
directory. And Rosé expects that all tracks belong to a "release" (meaning an
album, single, EP, etc.). Rosé is designed for release-oriented libraries, not
track-oriented libraries.

# Installation

Expand Down Expand Up @@ -284,7 +300,7 @@ finally (3) play music!
$ ls -1 "1. Releases/"
'BLACKPINK - 2016. SQUARE ONE - Single [Big Room House;Dance-Pop;K-Pop]'
'BLACKPINK - 2016. SQUARE TWO - Single [Dance-Pop;K-Pop]'
'LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [K-Pop]'
'LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [Dance-Pop;Future Bass;K-Pop]'
'NewJeans - 2022. Ditto - Single [Contemporary R&B;K-Pop]'
'{NEW} LOOΠΔ - 2017. Kim Lip - Single [Contemporary R&B;Dance-Pop;K-Pop]'
```
Expand All @@ -295,7 +311,7 @@ finally (3) play music!
Mine is `mpv`:

```bash
$ mpv "1. Releases/LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [K-Pop]/04. LOOΠΔ ODD EYE CIRCLE - Chaotic.opus"
$ mpv "1. Releases/LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [Dance-Pop;Future Bass;K-Pop]/04. LOOΠΔ ODD EYE CIRCLE - Chaotic.opus"
(+) Audio --aid=1 'Chaotic' (opus 2ch 48000Hz)
File tags:
Artist: LOOΠΔ ODD EYE CIRCLE
Expand Down
16 changes: 12 additions & 4 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,12 @@ This has some nice consequences:
drifts from source files, that drift can always be automatically resolved
simply by rebuilding the cache.

# Stable Release & Track Identifiers
# Release & Track Identifiers

Rosé assigns UUIDs to each release and track in order to identify them across
arbitrarily large metadata changes. These UUIDs are persisted to the source
files.
Rosé assigns a stable UUID to each release and track in order to identify them.
The UUIDs are also used to track membership in collages and playlists.

These UUIDs are persisted to the source files:

- Each release has a `.rose.{uuid}.toml` file, which preserves release-level
state, such as `New`. The UUID is in the filename instead of the file
Expand All @@ -58,6 +59,13 @@ files.
- Each track has a custom `roseid` tag. This tag is written to the source audio
file.

Therefore, provided that other programs do not erase the UUID, Rosé will be
able to identify releases and tracks across arbitrarily drastic directory and
file renames and tag changes. Rosé does not depend on the source directory's
filenames or tags remaining static; the source directory can be freely
modified. The only constraint is that each release must be a directory in the
root of the `music_source_dir`.

# Read Cache Update

The read cache update is optimized to minimize the number of disk accesses, as
Expand Down
104 changes: 46 additions & 58 deletions docs/CACHE_MAINTENANCE.md
Original file line number Diff line number Diff line change
@@ -1,67 +1,60 @@
# Maintaining the Cache

The read cache is a SQLite database that replicates the metadata in
`music_source_dir`. The read cache exists to improve performance: it can be
read from far more performantly than the `music_source_dir` can.
`music_source_dir`. The read cache exists solely to improve performance: it can
be read from far more performantly than the `music_source_dir` can.

The read cache is never written to directly, outside of the `update_cache_*`
functions, which re-read the source files and write their metadata into the
cache. All mutations in the system occur directly to the source files, not to
the read cache.
The read cache does not have any state of its own. _All_ of the data in the read
cache is replicated from the `music_source_dir`. Hence, we never write to the
read cache. Instead, all updates write to the source files, which then triggers
the cache update function. The cache update function updates the cache to match
the source directory's state.

# Cache Drift
> [!NOTE]
> To better understand how the read cache fits into Rosé, we recommend reading
> [Architecture](./ARCHITECTURE.md).
So what's the problem?
# Cache Drift

The read cache has the possibility of _drifting_ from the source directory. For
example, let's say that I move files around in the source directory, modify
tags directly, or even delete a release. After those actions, the read cache
will still reflect the previous state, and thus we say that the cache has
_drifted_.
Assuming you only modify your music library through Rosé, the cache will always
remain up to date, since Rosé triggers a cache update whenever it is aware of
an update to the source directory.

This is problematic because some operations in the virtual filesystem will
begin to fail. For example, if a file is moved in the source directory, and the
virtual filesystem then attempts to read from its previous path, it will hit a
FileNotFound (ENOENT) error.
However, that assumption does not hold. If changes are made directly to the
source directory, and Rosé is not "informed," Rosé's cache will contain the
previous state of the source directory. We call this a _cache drift_.

Thus, after changes to the source directory, we need to update the cache so
that it _synchronizes_ with the source directory. Note that this
synchronization is entirely _one-way_: the source files update the read cache.
The opposite direction never occurs: the read cache never updates the source
files.
This is problematic because Rosé may attempt to read files that no longer exist
or display old metadata. Thus, we should inform Rosé whenever a change is made
to the source directory.

# Updating the Cache

The cache can be updated with the command `rose cache update`. By default, this
command only checks files which have changed since the last cache update. It
uses the mtime (last modified) field for this purpose. However, sometimes we
want to refresh the cache regardless of mtimes. In that case, we can run `rose
cache update --force`.

It would be pretty annoying if you had to run this command by hand after each
metadata update. So Rosé will automatically run this command whenever an update
happens _through_ Rosé. That means:
A cache update can be performed manually with the `rose cache update` command.
In this command, Rosé will identify any files that changed and update the read
cache accordingly. In other words, this command informs Rosé that something
changed in the source directory.

- If a file is modified in the virtual filesystem, a cache update is
triggered when the file handle closes.
- If a release is modified by the command line, a cache update is triggered at
the end of the command.
For performance reasons, the `rose cache update` command only checks files with
a different Last Modified (mtime) from the last cache update. To disable this
behavior and recheck every file, pass the `--force/-f` flag.

Rosé will also update the cache on virtual filesystem mount.
It would be annoying if you had to run `rose cache upate` by hand after each
metadata change. Rosé thus automatically updates the cache in response to
changes made _through_ Rosé. Any updates made through the virtual filesystem or
command line automatically trigger a cache update for the changed files. Rosé
will also update the cache when the virtual filesystem is mounted.

# Updating on External Changes
However, even with that improvement, you would still need to run `rose cache
update` automatically whenever changes are made directly to the source
directory. Thus, Rosé provides the `rose cache watch` command, which runs a
watcher that listens for file update events in the source directory. This
watcher will trigger a cache update whenever a file in the source directory
changes.

However, there is one class of updates that this does not update in response
to, and that is updates made by external tools directly to the source
directory. If another system updates your source directory directly, the read
cache will drift.

To update in response to those external changes, you can run `rose cache
watch`. This command starts a watcher that triggers a cache update whenever a
file changes in the source directory. `rose cache watch` runs in the
foreground, so it is recommended that you manage it with a service manager like
systemd. See [Configuration](./CONFIGURATION.md) for example systemd unit
files.
By default, the watcher runs in the foreground. We recommend backgrounding it
with a service manager, such as systemd. See
[Configuration](./CONFIGURATION.md) for sample systemd unit files.

# Cache Resets

Expand All @@ -71,14 +64,9 @@ When Rosé detects that:
2. The configuration file has changed,
3. Or the cache database schema has changed,

Rosé will delete the read cache and rebuild it from scratch.

A full cache rebuild is fairly performant, though an order of magnitude slower
than a cache scan that results in no changes.

Deleting the read cache is a supported operation, and it is a viable solution
in case your cache ends up in a bad state (e.g. due to a bug or other issue).
Rosé will delete the read cache and rebuild it from scratch. A full cache
rebuild is fairly performant, though an order of magnitude slower than a cache
scan that results in no changes.

Since Rosé stores all of its state in the source directory, and merely
replicates that state into the cache, deleting the cache will never result in a
loss of data.
Deleting the read cache does not result in any loss of data, and is a viable
solution if your cache ends up in a bad state (e.g. due to a bug).
8 changes: 4 additions & 4 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ The configuration parameters, with examples, are:
# The directory containing the music to manage. Rosé has strong expectations
# towards the organization of this directory.
#
# All releases are expected to be immediate child directories of the
# music_source_dir. And Rosé expects that all tracks belong to a "release"
# (meaning an album, single, EP, etc.). Therefore, loose audio files at the
# top-level of the source directory will be ignored.
# All releases must be immediate child directories of the music_source_dir. And
# Rosé expects that all tracks belong to a "release" (meaning an album, single,
# EP, etc.). Therefore, loose audio files at the top-level of the source
# directory will be ignored.
#
# Rosé also writes collages and playlists to this directory, as `!collages` and
# `!playlists` subdirectories.
Expand Down
13 changes: 7 additions & 6 deletions docs/METADATA_MANAGEMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ The artist tag is described by the following grammar:
<name> ::= string ';' <name> | string
```

Rosé only supports the artist roles:
Rosé supports the following artist roles:

- `main`
- `guest`
Expand Down Expand Up @@ -111,16 +111,17 @@ Virtual Filesystem Directory Name.
So for example:

```bash
$ rose releases edit "LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [K-Pop]"
$ rose releases edit "LOOΠΔ ODD EYE CIRCLE - 2017. Mix & Match - EP [Dance-Pop;Future Bass;K-Pop]"
$ rose releases edit "018b4ff1-acdf-7ff1-bcd6-67757aea0fed"
```

This command opens up a TOML representation of the release's metadata in your
`$EDITOR`. Upon save and exit, the TOML's metadata is written to the file tags.

Rosé validates the Artist Role and Release Type fields. The values provided
must be one of the supported values. The supported values are documented in
[Artist Tags](#artist-tags) and [Release Type Tags](#release-type-tags).
> [!NOTE]
> Rosé validates the Artist Role and Release Type fields upon metadata edit.
> The values provided must be one of the supported values. The supported values
> are documented in [Artist Tags](#artist-tags) and [Release Type Tags](#release-type-tags).
An example of the TOML representation is:

Expand Down Expand Up @@ -197,7 +198,7 @@ Rosé supports three tag container formats:
- MP4: `.m4a` files
- Vorbis: `.ogg`, `.opus`, and `.flac` files

In this section, we will list out the per-container fields that we read/write.
In this section, we list out the per-container field names that we read/write.
Rosé will only write to a single field for each tag; however, for tags with
multiple conventions out in the rest of the world, Rosé will support reading
from additional fields.
Expand Down
Loading

0 comments on commit 34054bf

Please sign in to comment.