Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Lidarr support #30

Merged
merged 28 commits into from
Jan 15, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
fec366f
updated test data
Flaminel Jan 1, 2025
a1fcdb6
updated appsettings
Flaminel Jan 1, 2025
fd106bb
updated test data
Flaminel Jan 1, 2025
4a16fd2
added support for Lidarr
Flaminel Jan 1, 2025
8c23e8c
updated test data
Flaminel Jan 1, 2025
aa38a97
updated README
Flaminel Jan 1, 2025
e540fbf
changed blocklist code to accommodate more arrs
Flaminel Jan 3, 2025
20be1bc
updated README
Flaminel Jan 3, 2025
1f9094a
streamlined content deletion when all files are unwanted
Flaminel Jan 8, 2025
77bc456
renamed var
Flaminel Jan 9, 2025
6fbfeaf
added some comments
Flaminel Jan 9, 2025
2579090
removed unused method
Flaminel Jan 9, 2025
e18dec3
fixed test docker-compose
Flaminel Jan 9, 2025
b864ae8
fixed blocklist check
Flaminel Jan 9, 2025
611b84b
updated README
Flaminel Jan 9, 2025
14b0fe7
enabled dev queue cleaner
Flaminel Jan 9, 2025
9b14f1a
updated test data
Flaminel Jan 10, 2025
a8d1c7b
updated some logs to be more descriptive
Flaminel Jan 10, 2025
21c77fa
fixed dummy download client
Flaminel Jan 11, 2025
2f09e88
updated test data
Flaminel Jan 13, 2025
1fead83
Merge branch 'main' into add_lidarr_support
Flaminel Jan 14, 2025
4135378
fixed merge
Flaminel Jan 14, 2025
1858290
Merge branch 'main' into add_lidarr_support
Flaminel Jan 14, 2025
1c3669c
fixed README
Flaminel Jan 14, 2025
5bc58c9
changed defaults for some env vars
Flaminel Jan 15, 2025
b972825
changed log templates to include instance type
Flaminel Jan 15, 2025
88d9bde
fixed README
Flaminel Jan 15, 2025
8d6a111
fixed id data type
Flaminel Jan 15, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,6 @@ _NCrunch_*
_TeamCity*

# Sonarr
config.xml
nzbdrone.log*txt
UpdateLogs/
*workspace.xml
Expand Down
115 changes: 80 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,9 @@ Only the **latest versions** of the following apps are supported, or earlier ver
- Transmission
- Sonarr
- Radarr
- Lidarr

This tool is actively developed and still a work in progress. Join the Discord server if you want to reach out to me quickly (or just stay updated on new releases) so we can squash those pesky bugs together:
This tool is actively developed and still a work in progress, so using the `latest` Docker tag may result in breaking changes. Join the Discord server if you want to reach out to me quickly (or just stay updated on new releases) so we can squash those pesky bugs together:

> https://discord.gg/sWggpnmGNY

Expand All @@ -35,8 +36,12 @@ This tool is actively developed and still a work in progress. Join the Discord s
- Mark the files that were found in the queue as **unwanted/skipped** if:
- They **are listed in the blacklist**, or
- They **are not included in the whitelist**.
- If **all files** of a download **are unwanted**:
- It will be removed from the *arr's queue and blocked.
- It will be deleted from the download client.
- A new search will be triggered for the *arr item.
2. **Queue cleaner** will:
- Run every 5 minutes (or configured cron).
- Run every 5 minutes (or configured cron, or right after `content blocker`).
- Process all items in the *arr queue.
- Check each queue item if it is **stalled (download speed is 0)**, **stuck in matadata downloading** or **failed to be imported**.
- If it is, the item receives a **strike** and will continue to accumulate strikes every time it meets any of these conditions.
Expand All @@ -63,8 +68,8 @@ This tool is actively developed and still a work in progress. Join the Discord s

## Using cleanuperr's blocklist (works with all supported download clients)

1. Set both `QUEUECLEANER__ENABLED` and `CONTENTBLOCKER_ENABLED` to `true` in your environment variables.
2. Configure and enable either a **blacklist** or a **whitelist** as described in the [Environment variables](#Environment-variables) section.
1. Set both `QUEUECLEANER__ENABLED` and `CONTENTBLOCKER__ENABLED` to `true` in your environment variables.
2. Configure and enable either a **blacklist** or a **whitelist** as described in the [Arr variables](#Arr-variables) section.
3. Once configured, cleanuperr will perform the following tasks:
- Execute the **content blocker** job, as explained in the [How it works](#how-it-works) section.
- Execute the **queue cleaner** job, as explained in the [How it works](#how-it-works) section.
Expand Down Expand Up @@ -108,16 +113,13 @@ services:

- CONTENTBLOCKER__ENABLED=true
- CONTENTBLOCKER__IGNORE_PRIVATE=true
- CONTENTBLOCKER__BLACKLIST__ENABLED=true
- CONTENTBLOCKER__BLACKLIST__PATH=https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/blacklist
# OR
# - CONTENTBLOCKER__WHITELIST__ENABLED=true
# - CONTENTBLOCKER__WHITELIST__PATH=https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/whitelist

- DOWNLOAD_CLIENT=qBittorrent
- QBITTORRENT__URL=http://localhost:8080
- QBITTORRENT__USERNAME=user
- QBITTORRENT__PASSWORD=pass
- DOWNLOAD_CLIENT=none
# OR
# - DOWNLOAD_CLIENT=qBittorrent
# - QBITTORRENT__URL=http://localhost:8080
# - QBITTORRENT__USERNAME=user
# - QBITTORRENT__PASSWORD=pass
# OR
# - DOWNLOAD_CLIENT=deluge
# - DELUGE__URL=http://localhost:8112
Expand All @@ -127,26 +129,40 @@ services:
# - TRANSMISSION__URL=http://localhost:9091
# - TRANSMISSION__USERNAME=test
# - TRANSMISSION__PASSWORD=testing
# OR
# - DOWNLOAD_CLIENT=none

- SONARR__ENABLED=true
- SONARR__SEARCHTYPE=Episode
- SONARR__BLOCK__TYPE=blacklist
- SONARR__BLOCK__PATH=https://example.com/path/to/file.txt
- SONARR__INSTANCES__0__URL=http://localhost:8989
- SONARR__INSTANCES__0__APIKEY=secret1
- SONARR__INSTANCES__1__URL=http://localhost:8990
- SONARR__INSTANCES__1__APIKEY=secret2

- RADARR__ENABLED=true
- RADARR__BLOCK__TYPE=blacklist
- RADARR__BLOCK__PATH=https://example.com/path/to/file.txt
- RADARR__INSTANCES__0__URL=http://localhost:7878
- RADARR__INSTANCES__0__APIKEY=secret3
- RADARR__INSTANCES__1__URL=http://localhost:7879
- RADARR__INSTANCES__1__APIKEY=secret4

- LIDARR__ENABLED=true
- LIDARR__BLOCK__TYPE=blacklist
- LIDARR__BLOCK__PATH=https://example.com/path/to/file.txt
- LIDARR__INSTANCES__0__URL=http://radarr:8686
- LIDARR__INSTANCES__0__APIKEY=secret5
- LIDARR__INSTANCES__1__URL=http://radarr:8687
- LIDARR__INSTANCES__1__APIKEY=secret6
image: ghcr.io/flmorg/cleanuperr:latest
restart: unless-stopped
```

### Environment variables
## Environment variables

### General variables
<details>
<summary>Click here</summary>

| Variable | Required | Description | Default value |
|---|---|---|---|
Expand All @@ -168,12 +184,15 @@ services:
|||||
| CONTENTBLOCKER__ENABLED | No | Enable or disable the content blocker | false |
| CONTENTBLOCKER__IGNORE_PRIVATE | No | Whether to ignore downloads from private trackers | false |
| CONTENTBLOCKER__BLACKLIST__ENABLED | Yes if content blocker is enabled and whitelist is not enabled | Enable or disable the blacklist | false |
| CONTENTBLOCKER__BLACKLIST__PATH | Yes if blacklist is enabled | Path to the blacklist (local file or url)<br>Needs to be json compatible | empty |
| CONTENTBLOCKER__WHITELIST__ENABLED | Yes if content blocker is enabled and blacklist is not enabled | Enable or disable the whitelist | false |
| CONTENTBLOCKER__WHITELIST__PATH | Yes if whitelist is enabled | Path to the whitelist (local file or url)<br>Needs to be json compatible | empty |
|||||
| DOWNLOAD_CLIENT | No | Download client that is used by *arrs<br>Can be `qbittorrent`, `deluge`, `transmission` or `none` | `qbittorrent` |
</details>

### Download client variables
<details>
<summary>Click here</summary>

| Variable | Required | Description | Default value |
|---|---|---|---|
| DOWNLOAD_CLIENT | No | Download client that is used by *arrs<br>Can be `qbittorrent`, `deluge`, `transmission` or `none` | `none` |
| QBITTORRENT__URL | No | qBittorrent instance url | http://localhost:8112 |
| QBITTORRENT__USERNAME | No | qBittorrent user | empty |
| QBITTORRENT__PASSWORD | No | qBittorrent password | empty |
Expand All @@ -184,42 +203,68 @@ services:
| TRANSMISSION__URL | No | Transmission instance url | http://localhost:9091 |
| TRANSMISSION__USERNAME | No | Transmission user | empty |
| TRANSMISSION__PASSWORD | No | Transmission password | empty |
|||||
| SONARR__ENABLED | No | Enable or disable Sonarr cleanup | true |
</details>

### Arr variables
<details>
<summary>Click here</summary>

| Variable | Required | Description | Default value |
|---|---|---|---|
| SONARR__ENABLED | No | Enable or disable Sonarr cleanup | false |
| SONARR__BLOCK__TYPE | No | Block type<br>Can be `blacklist` or `whitelist` | `blacklist` |
| SONARR__BLOCK__PATH | No | Path to the blocklist (local file or url)<br>Needs to be json compatible | empty |
| SONARR__SEARCHTYPE | No | What to search for after removing a queue item<br>Can be `Episode`, `Season` or `Series` | `Episode` |
| SONARR__INSTANCES__0__URL | No | First Sonarr instance url | http://localhost:8989 |
| SONARR__INSTANCES__0__APIKEY | No | First Sonarr instance API key | empty |
|||||
| RADARR__ENABLED | No | Enable or disable Radarr cleanup | false |
| RADARR__BLOCK__TYPE | No | Block type<br>Can be `blacklist` or `whitelist` | `blacklist` |
| RADARR__BLOCK__PATH | No | Path to the blocklist (local file or url)<br>Needs to be json compatible | empty |
| RADARR__INSTANCES__0__URL | No | First Radarr instance url | http://localhost:8989 |
| RADARR__INSTANCES__0__APIKEY | No | First Radarr instance API key | empty |
|||||
| LIDARR__ENABLED | No | Enable or disable LIDARR cleanup | false |
| LIDARR__BLOCK__TYPE | No | Block type<br>Can be `blacklist` or `whitelist` | `blacklist` |
| LIDARR__BLOCK__PATH | No | Path to the blocklist (local file or url)<br>Needs to be json compatible | empty |
| LIDARR__INSTANCES__0__URL | No | First LIDARR instance url | http://localhost:8989 |
| LIDARR__INSTANCES__0__APIKEY | No | First LIDARR instance API key | empty |
</details>

### Advanced variables
<details>
<summary>Click here</summary>

| Variable | Required | Description | Default value |
|---|---|---|---|
| HTTP_MAX_RETRIES | No | The number of times to retry a failed HTTP call (to *arrs, download clients etc.) | 0 |
| HTTP_TIMEOUT | No | The number of seconds to wait before failing an HTTP call (to *arrs, download clients etc.) | 100 |
</details>

#

### To be noted

1. The blacklist and the whitelist can not be both enabled at the same time.
2. The queue cleaner and content blocker can be enabled or disabled separately, if you want to run only one of them.
3. Only one download client can be enabled at a time. If you have more than one download client, you should deploy multiple instances of cleanuperr.
4. The blocklists (blacklist/whitelist) should have a single pattern on each line and supports the following:
1. The queue cleaner and content blocker can be enabled or disabled separately, if you want to run only one of them.
2. Only one download client can be enabled at a time. If you have more than one download client, you should deploy multiple instances of cleanuperr.
3. The blocklists (blacklist/whitelist) should have a single pattern on each line and supports the following:
```
*example // file name ends with "example"
example* // file name starts with "example"
*example* // file name has "example" in the name
example // file name is exactly the word "example"
*example // file name ends with "example"
example* // file name starts with "example"
*example* // file name has "example" in the name
example // file name is exactly the word "example"
regex:<ANY_REGEX> // regex that needs to be marked at the start of the line with "regex:"
```
5. Multiple Sonarr/Radarr instances can be specified using this format, where `<NUMBER>` starts from 0:
4. Multiple Sonarr/Radarr/Lidarr instances can be specified using this format, where `<NUMBER>` starts from `0`:
```
SONARR__INSTANCES__<NUMBER>__URL
SONARR__INSTANCES__<NUMBER>__APIKEY
```
6. Multiple failed import patterns can be specified using this format, where `<NUMBER>` starts from 0:
5. Multiple failed import patterns can be specified using this format, where `<NUMBER>` starts from 0:
```
QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__<NUMBER>
```

6. [This blacklist](https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/blacklist) and [this whitelist](https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/whitelist) can be used for Sonarr and Radarr, but they are not suitable for other *arrs.
#

### Binaries (if you're not using Docker)
Expand Down
11 changes: 11 additions & 0 deletions code/Common/Configuration/Arr/ArrConfig.cs
Original file line number Diff line number Diff line change
@@ -1,8 +1,19 @@
using Common.Configuration.ContentBlocker;

namespace Common.Configuration.Arr;

public abstract record ArrConfig
{
public required bool Enabled { get; init; }

public Block Block { get; init; } = new();

public required List<ArrInstance> Instances { get; init; }
}

public record Block
{
public BlocklistType Type { get; set; }

public string? Path { get; set; }
}
6 changes: 6 additions & 0 deletions code/Common/Configuration/Arr/LidarrConfig.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
namespace Common.Configuration.Arr;

public sealed record LidarrConfig : ArrConfig
{
public const string SectionName = "Lidarr";
}
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
namespace Domain.Enums;
namespace Common.Configuration.ContentBlocker;

public enum BlocklistType
{
Expand Down
30 changes: 1 addition & 29 deletions code/Common/Configuration/ContentBlocker/ContentBlockerConfig.cs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Configuration;

namespace Common.Configuration.ContentBlocker;

Expand All @@ -11,35 +11,7 @@ public sealed record ContentBlockerConfig : IJobConfig
[ConfigurationKeyName("IGNORE_PRIVATE")]
public bool IgnorePrivate { get; init; }

public PatternConfig? Blacklist { get; init; }

public PatternConfig? Whitelist { get; init; }

public void Validate()
{
if (!Enabled)
{
return;
}

if (Blacklist is null && Whitelist is null)
{
throw new Exception("content blocker is enabled, but both blacklist and whitelist are missing");
}

if (Blacklist?.Enabled is true && Whitelist?.Enabled is true)
{
throw new Exception("only one exclusion (blacklist/whitelist) list is allowed");
}

if (Blacklist?.Enabled is true && string.IsNullOrEmpty(Blacklist.Path))
{
throw new Exception("blacklist path is required");
}

if (Whitelist?.Enabled is true && string.IsNullOrEmpty(Whitelist.Path))
{
throw new Exception("blacklist path is required");
}
}
}
8 changes: 0 additions & 8 deletions code/Common/Configuration/ContentBlocker/PatternConfig.cs

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@ namespace Common.Configuration.DownloadClient;
public sealed record DownloadClientConfig
{
[ConfigurationKeyName("DOWNLOAD_CLIENT")]
public Enums.DownloadClient DownloadClient { get; init; } = Enums.DownloadClient.QBittorrent;
public Enums.DownloadClient DownloadClient { get; init; } = Enums.DownloadClient.None;
}
8 changes: 8 additions & 0 deletions code/Domain/Models/Arr/Blocking/BlockedItem.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
namespace Domain.Models.Arr.Blocking;

public record BlockedItem
{
public required string Hash { get; init; }

public required Uri InstanceUrl { get; init; }
}
8 changes: 8 additions & 0 deletions code/Domain/Models/Arr/Blocking/LidarrBlockedItem.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
namespace Domain.Models.Arr.Blocking;

public sealed record LidarrBlockedItem : BlockedItem
{
public required long AlbumId { get; init; }

public required long ArtistId { get; init; }
}
6 changes: 6 additions & 0 deletions code/Domain/Models/Arr/Blocking/RadarrBlockedItem.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
namespace Domain.Models.Arr.Blocking;

public sealed record RadarrBlockedItem : BlockedItem
{
public required long MovieId { get; init; }
}
10 changes: 10 additions & 0 deletions code/Domain/Models/Arr/Blocking/SonarrBlockedItem.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
namespace Domain.Models.Arr.Blocking;

public sealed record SonarrBlockedItem : BlockedItem
{
public required long EpisodeId { get; init; }

public required long SeasonNumber { get; init; }

public required long SeriesId { get; init; }
}
20 changes: 15 additions & 5 deletions code/Domain/Models/Arr/Queue/QueueRecord.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,26 @@ namespace Domain.Models.Arr.Queue;

public sealed record QueueRecord
{
public int SeriesId { get; init; }
public int EpisodeId { get; init; }
public int SeasonNumber { get; init; }
public int MovieId { get; init; }
// Sonarr
public long SeriesId { get; init; }
public long EpisodeId { get; init; }
public long SeasonNumber { get; init; }

// Radarr
public long MovieId { get; init; }

// Lidarr
public long ArtistId { get; init; }

public long AlbumId { get; init; }

// common
public required string Title { get; init; }
public string Status { get; init; }
public string TrackedDownloadStatus { get; init; }
public string TrackedDownloadState { get; init; }
public List<TrackedDownloadStatusMessage>? StatusMessages { get; init; }
public required string DownloadId { get; init; }
public required string Protocol { get; init; }
public required int Id { get; init; }
public required long Id { get; init; }
}
12 changes: 12 additions & 0 deletions code/Domain/Models/Lidarr/Album.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
namespace Domain.Models.Lidarr;

public sealed record Album
{
public long Id { get; set; }

public string Title { get; set; }

public long ArtistId { get; set; }

public Artist Artist { get; set; }
}
Loading
Loading