Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bump] introduce new version: 0.6 #459

Merged
merged 67 commits into from
Nov 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
7844a8e
fix: improve external api for redis
geofmureithi May 22, 2024
fd6c110
fix: improve exports for redis
geofmureithi May 23, 2024
6203fa0
fix: expose redis codec
geofmureithi May 23, 2024
aedb34c
Merge branch 'master' into chore/v0.6.0-alpha.0
geofmureithi Jun 26, 2024
59ed0f4
Feature: v0.6.0-alpha.0 version of apalis
geofmureithi Jun 26, 2024
7756b37
fix: minor fixes on some failures
geofmureithi Jun 26, 2024
fcce659
lint: cargo fmt
geofmureithi Jun 26, 2024
009f577
fix: remove Job impl
geofmureithi Jun 26, 2024
7bbde7f
lint: cargo fmt
geofmureithi Jun 26, 2024
b0af55c
bench: improve polling
geofmureithi Jun 26, 2024
2fec565
fix: introduce namespace and codec config (#339)
geofmureithi Jun 27, 2024
bcbb015
Version: 0.6.0-alpha.1 (#342)
geofmureithi Jul 2, 2024
732553a
bump: to v0.6.0-alpha.1 (#343)
geofmureithi Jul 2, 2024
dd9570c
fix: allow cd for prereleases (#349)
geofmureithi Jul 4, 2024
5861e84
Remove `Clone` constraints and buffer the service (#348)
geofmureithi Jul 4, 2024
7b953fd
bump: to 0.6.0-rc.1 (#350)
geofmureithi Jul 4, 2024
2fc018d
feat: add rsmq example (#353)
geofmureithi Jul 9, 2024
8bc2899
Fix: load layer from poller (#354)
geofmureithi Jul 9, 2024
a6998a6
Fix: mq example (#355)
geofmureithi Jul 9, 2024
adf401e
fix: handle unwraps in storages (#356)
geofmureithi Jul 9, 2024
d5496ff
fix: better apalis deps allowing tree shaking for backends (#357)
geofmureithi Jul 10, 2024
fc72bc7
standardize backend for storage and mq (#358)
geofmureithi Jul 10, 2024
7159edb
feat: standardize cron as backend (#359)
geofmureithi Jul 10, 2024
8ae48dc
fix: remove non-working restapi example (#360)
geofmureithi Jul 10, 2024
f1daab3
fix: expose the missing apis (#361)
geofmureithi Jul 10, 2024
9e71559
bump: to new version (#362)
geofmureithi Jul 10, 2024
369d8ec
Make Config accessible publicly (#364)
geofmureithi Jul 12, 2024
d70e479
fix: die if retries is zero (#365)
geofmureithi Jul 12, 2024
97ff348
Feature: Add a layer that catches panics (#366)
geofmureithi Jul 13, 2024
4ad94d2
Feature: Save results for storages (#369)
geofmureithi Jul 13, 2024
144d7ef
Bump: to 0.6.0-rc.3 (#370)
geofmureithi Jul 13, 2024
95618ff
fix: serde for sql request (#371)
geofmureithi Jul 13, 2024
1238fb0
fix: handle attempts in storages (#373)
geofmureithi Jul 16, 2024
04f4987
add: test utils that allow backend polling during tests (#374)
geofmureithi Jul 19, 2024
6d2a1e2
bump: to 0.6.0-rc.4 (#377)
geofmureithi Jul 19, 2024
cc58602
fix: handle 0 retries (#378)
geofmureithi Jul 19, 2024
be1674f
fix: ack api to allow backend to handle differently (#383)
geofmureithi Jul 25, 2024
b790332
fix(deps): update rust crate sqlx to 0.8.0 (#380)
renovate[bot] Jul 25, 2024
649b384
bump: to v0.6.0-rc.5 (#385)
geofmureithi Jul 25, 2024
e7a751c
chore: standardize codec usage (#388)
geofmureithi Aug 2, 2024
39fd007
Chore/more examples (#389)
geofmureithi Aug 3, 2024
914bb51
fix: improve on benches (#379)
geofmureithi Aug 5, 2024
873a2ca
Merge branch 'master' into chore/v0.6.0
geofmureithi Aug 5, 2024
6b85ac4
fix(deps): update rust crate sqlx to 0.8.1 [security] (#400)
renovate[bot] Sep 2, 2024
83f0e81
fix: add some missing data required for dependency injection (#409)
geofmureithi Sep 5, 2024
b1f79bd
remove: benchmarks (#410)
geofmureithi Sep 5, 2024
1c19c7a
bump: to 0.6.0-rc.6 (#412)
geofmureithi Sep 6, 2024
4ec676f
Update async-std to 1.13 (#413)
jayvdb Sep 9, 2024
7a496ad
Feature: Introducing Request Context (#416)
geofmureithi Sep 17, 2024
d62281f
bump: to 0.6.0-rc.7 (#418)
geofmureithi Sep 17, 2024
3166d7c
fix: apply `FromRequest` for items in `Parts` (#425)
geofmureithi Oct 3, 2024
e1d7e6b
fix:[bug] include backend provided layer in service layers. (#426)
geofmureithi Oct 4, 2024
f6442fb
chore: bump to 0.6.0-rc.8 (#430)
geofmureithi Oct 11, 2024
ae9f221
fix: apply max_attempts set via SqlContext (#447)
mlafeldt Nov 13, 2024
638242a
Bump redis (#442)
zakstucke Nov 19, 2024
31b6858
feat: re-export sqlx (#451)
mlafeldt Nov 21, 2024
68276ee
feat: Improve Worker management and drop Executor (#428)
geofmureithi Nov 21, 2024
e7dad8c
Chore/better api (#452)
geofmureithi Nov 21, 2024
995be7c
feat: add recovery of abandoned jobs to backend heartbeats (#453)
geofmureithi Nov 22, 2024
9863a60
feat: allow backends to emit errors (#454)
geofmureithi Nov 23, 2024
24275eb
Feat: Introduce simple ability to pipe cron jobs to any backend (#455)
geofmureithi Nov 23, 2024
c70e5c6
chore/dev-branch
geofmureithi Nov 26, 2024
c4ec3d3
feat: return of exposing backends to help in building apis (#457) (#458)
geofmureithi Nov 26, 2024
f09eea4
fmt: Cargo.toml
geofmureithi Nov 27, 2024
f2e5bfb
Merge branch 'main' into develop
geofmureithi Nov 27, 2024
6641691
fix: removed features
geofmureithi Nov 27, 2024
3250bea
fix: run only specific tests
geofmureithi Nov 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 0 additions & 39 deletions .github/workflows/bench.yaml

This file was deleted.

5 changes: 1 addition & 4 deletions .github/workflows/cd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,7 @@ jobs:
# vX.Y.Z-foo is pre-release version
VERSION=${GITHUB_REF#refs/tags/v}
VERSION_NUMBER=${VERSION%-*}
PUBLISH_OPTS="--dry-run"
if [[ $VERSION == $VERSION_NUMBER ]]; then
PUBLISH_OPTS=""
fi
PUBLISH_OPTS=""
echo VERSION=${VERSION} >> $GITHUB_ENV
echo PUBLISH_OPTS=${PUBLISH_OPTS} >> $GITHUB_ENV
echo VERSION_NUMBER=${VERSION_NUMBER} >> $GITHUB_ENV
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
- uses: actions-rs/cargo@v1
with:
command: check
args: --features tokio-comp
args: --all
- uses: actions-rs/cargo@v1
with:
command: check
Expand All @@ -39,7 +39,7 @@ jobs:
- uses: actions-rs/cargo@v1
with:
command: test
args: --features tokio-comp
# args: --all


fmt:
Expand Down
7 changes: 0 additions & 7 deletions .github/workflows/mysql.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,4 @@
on:
push:
paths:
- "packages/apalis-sql/src/lib.rs"
- "packages/apalis-sql/mysql.rs"
- "packages/apalis-sql/src/migrations/mysql/**"
- "packages/apalis-sql/src/Cargo.toml"
- ".github/workflows/mysql.yaml"
pull_request:
paths:
- "packages/apalis-sql/src/lib.rs"
Expand Down
9 changes: 1 addition & 8 deletions .github/workflows/postgres.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,4 @@
on:
push:
paths:
- "packages/apalis-sql/src/lib.rs"
- "packages/apalis-sql/postgres.rs"
- "packages/apalis-sql/src/migrations/postgres/**"
- "packages/apalis-sql/src/Cargo.toml"
- ".github/workflows/postgres.yaml"
pull_request:
paths:
- "packages/apalis-sql/src/lib.rs"
Expand Down Expand Up @@ -37,4 +30,4 @@ jobs:
toolchain: stable
override: true
- run: cargo test --no-default-features --features postgres,migrate,tokio-comp -- --test-threads=1
working-directory: packages/apalis-sql
working-directory: packages/apalis-sql
8 changes: 2 additions & 6 deletions .github/workflows/redis.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,4 @@
on:
push:
paths:
- "packages/apalis-redis/**"
- ".github/workflows/redis.yaml"
pull_request:
paths:
- "packages/apalis-redis/**"
Expand All @@ -26,11 +22,11 @@ jobs:
profile: minimal
toolchain: stable
override: true
- run: cargo test --features tokio-comp -- --test-threads=1
- run: cargo test -- --test-threads=1
working-directory: packages/apalis-redis
env:
REDIS_URL: redis://127.0.0.1/
- run: cargo test --features async-std-comp -- --test-threads=1
- run: cargo test -- --test-threads=1
working-directory: packages/apalis-redis
env:
REDIS_URL: redis://127.0.0.1/
9 changes: 1 addition & 8 deletions .github/workflows/sqlite.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,4 @@
on:
push:
paths:
- "packages/apalis-sql/src/lib.rs"
- "packages/apalis-sql/src/sqlite.rs"
- "packages/apalis-sql/src/migrations/sqlite/**"
- "packages/apalis-sql/src/Cargo.toml"
- ".github/workflows/sqlite.yaml"
pull_request:
paths:
- "packages/apalis-sql/src/lib.rs"
Expand All @@ -28,4 +21,4 @@ jobs:
toolchain: stable
override: true
- run: cargo test --no-default-features --features sqlite,migrate,tokio-comp -- --test-threads=1
working-directory: packages/apalis-sql
working-directory: packages/apalis-sql
94 changes: 30 additions & 64 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ repository = "https://github.com/geofmureithi/apalis"

[package]
name = "apalis"
version = "0.5.5"
version = "0.6.0"
authors = ["Geoffrey Mureithi <mureithinjuguna@gmail.com>"]
description = "Simple, extensible multithreaded background job processing for Rust"
edition.workspace = true
Expand All @@ -19,18 +19,8 @@ categories = ["database"]
bench = false

[features]
default = ["tracing", "tokio-comp"]

## Include redis storage
redis = ["apalis-redis"]
## Include Postgres storage
postgres = ["apalis-sql/postgres"]
## Include SQlite storage
sqlite = ["apalis-sql/sqlite"]
## Include MySql storage
mysql = ["apalis-sql/mysql"]
## Include Cron functionality
cron = ["apalis-cron"]
default = ["tracing"]


## Support Tracing 👀
tracing = ["dep:tracing", "dep:tracing-futures"]
Expand All @@ -47,20 +37,8 @@ timeout = ["tower/timeout"]
limit = ["tower/limit"]
## Support filtering jobs based on a predicate
filter = ["tower/filter"]
## Compatibility with async-std and smol runtimes
async-std-comp = [
"apalis-sql?/async-std-comp",
"apalis-redis?/async-std-comp",
"apalis-cron?/async-std-comp",
"async-std",
]
## Compatibility with tokio and actix runtimes
tokio-comp = [
"apalis-sql?/tokio-comp",
"apalis-redis?/tokio-comp",
"apalis-cron?/tokio-comp",
"tokio",
]
## Captures panics in executions and convert them to errors
catch-panic = []

layers = [
"sentry",
Expand All @@ -70,35 +48,16 @@ layers = [
"timeout",
"limit",
"filter",
"catch-panic",
]

docsrs = ["document-features"]

[dependencies.apalis-redis]
version = "0.5.5"
optional = true
path = "./packages/apalis-redis"
default-features = false

[dependencies.apalis-sql]

version = "0.5.5"
features = ["migrate"]
optional = true
default-features = false
path = "./packages/apalis-sql"

[dependencies.apalis-core]
version = "0.5.5"
version = "0.6.0"
default-features = false
path = "./packages/apalis-core"

[dependencies.apalis-cron]
version = "0.5.5"
optional = true
default-features = false
path = "./packages/apalis-cron"

[dependencies.document-features]
version = "0.2"
optional = true
Expand All @@ -116,22 +75,24 @@ pprof = { version = "0.14", features = ["flamegraph"] }
paste = "1.0.14"
serde = "1"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
apalis = { path = ".", features = ["redis", "sqlite", "postgres", "mysql"] }
redis = { version = "0.25.3", default-features = false, features = [
"script",
"aio",
"connection-manager",
apalis = { path = ".", features = ["limit"] }
apalis-redis = { path = "./packages/apalis-redis" }
apalis-sql = { path = "./packages/apalis-sql", features = [
"postgres",
"mysql",
"sqlite",
] }
redis = { version = "0.27", default-features = false, features = [
"tokio-comp",
"script",
"aio",
"connection-manager",
] }

[dev-dependencies.sqlx]
version = "0.8.1"
default-features = false
features = ["chrono", "mysql", "sqlite", "postgres"]


[[bench]]
name = "storages"
harness = false
features = ["chrono", "mysql", "sqlite", "postgres", "runtime-tokio"]

[workspace]
members = [
Expand All @@ -150,17 +111,22 @@ members = [
"examples/axum",
"examples/prometheus",
"examples/tracing",
# "examples/rest-api",
"examples/async-std-runtime",
"examples/basics",
"examples/redis-with-msg-pack",
"examples/redis-deadpool",
"examples/redis-mq-example",
"examples/cron",
"examples/catch-panic",
"examples/graceful-shutdown",
"examples/unmonitored-worker",
"examples/fn-args",
"examples/persisted-cron",
"examples/rest-api",
]


[dependencies]
tokio = { version = "1", features = [
"rt",
], default-features = false, optional = true }
async-std = { version = "1", optional = true }
tower = { version = "0.5", features = ["util"], default-features = false }
tracing-futures = { version = "0.2.5", optional = true, default-features = false }
sentry-core = { version = "0.34.0", optional = true, default-features = false }
Expand Down
31 changes: 13 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@

- Simple and predictable job handling model.
- Jobs handlers with a macro free API.
- Take full advantage of the [tower] ecosystem of
- Take full advantage of the [`tower`] ecosystem of
middleware, services, and utilities.
- Runtime agnostic - Use tokio, smol etc.
- Optional Web interface to help you manage your jobs.
Expand All @@ -59,43 +59,42 @@ To get started, just add to Cargo.toml

```toml
[dependencies]
apalis = { version = "0.5", features = ["redis"] } # Backends available: postgres, sqlite, mysql, amqp
apalis = { version = "0.6" }
apalis-redis = { version = "0.6" }
# apalis-sql = { version = "0.6", features = ["postgres"] } # or mysql, sqlite
```

## Usage

```rust
use apalis::prelude::*;
use apalis::redis::RedisStorage;
use apalis_redis::{RedisStorage, Config};
use serde::{Deserialize, Serialize};
use anyhow::Result;

#[derive(Debug, Deserialize, Serialize)]
struct Email {
to: String,
}

impl Job for Email {
const NAME: &'static str = "apalis::Email";
}

/// A function that will be converted into a service.
async fn send_email(job: Email, data: Data<usize>) -> Result<(), Error> {
/// execute job
Ok(())
}

#[tokio::main]
async fn main() -> Result<()> {
async fn main() -> {
std::env::set_var("RUST_LOG", "debug");
env_logger::init();
let redis_url = std::env::var("REDIS_URL").expect("Missing env variable REDIS_URL");
let storage = RedisStorage::new(redis).await?;
let conn = apalis_redis::connect(redis_url).await.expect("Could not connect");
let storage = RedisStorage::new(conn);
Monitor::new()
.register_with_count(2, {
.register({
WorkerBuilder::new(format!("email-worker"))
.concurrency(2)
.data(0usize)
.with_storage(storage)
.backend(storage)
.build_fn(send_email)
})
.run()
Expand All @@ -122,17 +121,13 @@ async fn produce_route_jobs(storage: &RedisStorage<Email>) -> Result<()> {
## Feature flags

- _tracing_ (enabled by default) — Support Tracing 👀
- _redis_ — Include redis storage
- _postgres_ — Include Postgres storage
- _sqlite_ — Include SQlite storage
- _mysql_ — Include MySql storage
- _cron_ — Include cron job processing
- _sentry_ — Support for Sentry exception and performance monitoring
- _prometheus_ — Support Prometheus metrics
- _retry_ — Support direct retrying jobs
- _timeout_ — Support timeouts on jobs
- _limit_ — 💪 Limit the amount of jobs
- _filter_ — Support filtering jobs based on a predicate
- _catch-panic_ - Catch panics that occur during execution

## Storage Comparison

Expand Down Expand Up @@ -214,5 +209,5 @@ See also the list of [contributors](https://github.com/geofmureithi/apalis/contr
This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details

[`tower::Service`]: https://docs.rs/tower/latest/tower/trait.Service.html
[tower]: https://crates.io/crates/tower
[`tower`]: https://crates.io/crates/tower
[`actix`]: https://crates.io/crates/actix
Loading
Loading