Skip to content

Commit

Permalink
feat: added milestone dispatch feature
Browse files Browse the repository at this point in the history
  • Loading branch information
Tbaile committed Nov 20, 2024
1 parent 2b7337b commit fc890d5
Show file tree
Hide file tree
Showing 10 changed files with 215 additions and 11 deletions.
38 changes: 28 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,8 @@ While some of the values are self-explanatory, there are a few that you need to
- `APP_URL`: The full URL where the application is reached from, while most of the functionalities will work with a
wrong value, the url generation is based off this value.
- `FILESYSTEM_DISK`: Disk to use during production, works same as development, more info in the development setup.
- `REPOSITORY_MILESTONE_TOKEN`: Token used to trigger from remote the milestone creation, you can set this to a random
value, it's used to avoid unwanted requests.

### Container Configuration

Expand All @@ -213,14 +215,8 @@ otherwise you will lose reference to endpoints and snapshots (or files, if you'r

### First deploy

To deploy the service, you can find the related files needed in the `deploy` directory:

- `compose`: Contains the `docker-compose.yml` file, this file can deploy the full stack once copied to the server and
then using `docker compose up -d`, remember to copy the `.env` production file to the same directory where you store
the compose file.
- `systemd`: Uses `systemd` and `podman` to deploy the service, you can find the service files in the `systemd`. Use
the `v4` or `v5` files, depending on the podman version you want to deploy. Remember to copy the `.env` production
file to the `%S/parceler.env` directory.
An example of a deployment can be found under the `deploy` directory, you can use the `deploy/docker-compose.yml` file
to deploy the full stack. And replicate the same structure in your server.

### Additional Configuration

Expand Down Expand Up @@ -267,8 +263,8 @@ The command will guide you through the process of adding a repository, here's th
- `command`: the command the worker will run to sync the repository it can be anything available in the container.
Save the content of the repository under the path `source/{repository_name}` in the disk you're using.
(e.g. if you're using the local disk, save the content of the repository
under `storage/app/source/repository_name`). `rclone` binary is available in the container, to add configuration file
follow the [Additional Configuration](#additional-configuration) section.
under `storage/app/source/{repository_name}`). `rclone` binary is available in the container, to add configuration
file follow the [Additional Configuration](#additional-configuration) section.
- `source_folder`: if repository files are stored in a subfolder, you can specify it here, otherwise leave it empty.
- `delay`: how many days the upstream files are delayed.

Expand Down Expand Up @@ -360,6 +356,28 @@ be provided the folder that are snapshotted and which one is currently being ser
php artisan repository:snapshots {repository_name}
```

### Milestone release

A Milestone Release is a process that wipes all the snapshots of a repository and then creates one with the latest sync.
This is useful when you want to release a new version of a repository, or when you want to force the release of a
specific set of packages.

To trigger a milestone release, this can be done by both of the following:

- CLI

```bash
php artisan repository:milestone {repository_name}
```

- CURL

Additional authentication must be provided, the token is set in the `.env` file under the `REPOSITORY_MILESTONE_TOKEN`.

```bash
curl -X POST -H Accept:application/json -H Authorization:Bearer <token> <url>/repository/<repository_name>/milestone
```

## List of behaviours in case of distribution of faulty packages

The following list is a guide on how to handle the distribution of faulty packages, to find which of the snapshots has a
Expand Down
42 changes: 42 additions & 0 deletions app/Console/Commands/MilestoneRepository.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
<?php

namespace App\Console\Commands;

use App\Jobs\MilestoneRelease;
use App\Models\Repository;
use Illuminate\Console\Command;
use Illuminate\Database\Eloquent\ModelNotFoundException;
use Throwable;

class MilestoneRepository extends Command
{
/**
* The name and signature of the console command.
*
* @var string
*/
protected $signature = 'repository:milestone {repository}';

/**
* The console command description.
*
* @var string
*/
protected $description = 'Dispatch a milestone reset job for the given repository.';

/**
* Execute the console command.
*
* @throws Throwable
*/
public function handle(): void
{
try {
$repository = Repository::where('name', $this->argument('repository'))->firstOrFail();
MilestoneRelease::dispatch($repository);
$this->info("Milestone release for $repository->name dispatched.");
} catch (ModelNotFoundException) {
$this->fail("Repository '{$this->argument('repository')}' not found.");
}
}
}
26 changes: 26 additions & 0 deletions app/Http/Middleware/MilestoneAuth.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
<?php

namespace App\Http\Middleware;

use Closure;
use Illuminate\Http\Request;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\HttpKernel\Exception\UnauthorizedHttpException;

class MilestoneAuth
{
/**
* Using the Authorization header to authenticate the user, the token is passed as a Bearer token.
* Check against the configuration milestone_token to allow the request.
*
* @param Closure(Request): (Response) $next
*/
public function handle(Request $request, Closure $next): Response
{
if ($request->header('Authorization') !== 'Bearer '.config('repositories.milestone_token')) {
throw new UnauthorizedHttpException('Bearer', 'Unauthenticated');
}

return $next($request);
}
}
39 changes: 39 additions & 0 deletions app/Jobs/MilestoneRelease.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
<?php

//
// Copyright (C) 2024 Nethesis S.r.l.
// SPDX-License-Identifier: AGPL-3.0-or-later
//

namespace App\Jobs;

use App\Models\Repository;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Storage;

class MilestoneRelease implements ShouldQueue
{
use Queueable;

/**
* Sync the milestone release.
*/
public function __construct(public readonly Repository $repository) {}

/**
* Execute the job.
*/
public function handle(): void
{
Log::debug("Releasing milestone for {$this->repository->name}.");
$toPurge = Storage::directories($this->repository->snapshotDir());
SyncRepository::dispatchSync($this->repository);
foreach ($toPurge as $dir) {
Log::debug("Purging directory $dir.");
Storage::deleteDirectory($dir);
}
Log::debug("Milestone released for {$this->repository->name}.");
}
}
4 changes: 3 additions & 1 deletion bootstrap/app.php
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@
health: '/up',
)
->withMiddleware(function (Middleware $middleware) {
//
$middleware->validateCsrfTokens(except: [
'/repository/*/milestone',
]);
})
->withExceptions(function (Exceptions $exceptions) {
//
Expand Down
5 changes: 5 additions & 0 deletions config/repositories.php
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,9 @@
* This directory contains the snapshots of synced repositories.
*/
'snapshots' => env('REPOSITORY_BASE_FOLDER', 'snapshots'),

/*
* Milestone release authentication token
*/
'milestone_token' => env('REPOSITORY_MILESTONE_TOKEN'),
];
1 change: 1 addition & 0 deletions phpunit.xml
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,6 @@
<env name="QUEUE_CONNECTION" value="sync"/>
<env name="SESSION_DRIVER" value="array"/>
<env name="TELESCOPE_ENABLED" value="false"/>
<env name="REPOSITORY_MILESTONE_TOKEN" value="testing" />
</php>
</phpunit>
10 changes: 10 additions & 0 deletions routes/web.php
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,22 @@
use App\Http\Middleware\CommunityLicenceCheck;
use App\Http\Middleware\EnterpriseLicenceCheck;
use App\Http\Middleware\ForceBasicAuth;
use App\Http\Middleware\MilestoneAuth;
use App\Jobs\MilestoneRelease;
use App\Models\Repository;
use Illuminate\Support\Facades\Route;

Route::get('/', function () {
return view('welcome');
});

Route::middleware(MilestoneAuth::class)
->post('/repository/{repository:name}/milestone', function (Repository $repository) {
MilestoneRelease::dispatch($repository);

return response()->json(['message' => 'Milestone release job dispatched.']);
});

Route::middleware(ForceBasicAuth::class)->group(function () {
Route::get('/repository/community/{repository:name}/{path}', RepositoryController::class)
->where('path', '.*')
Expand Down
27 changes: 27 additions & 0 deletions tests/Feature/Console/MilestoneReleaseTest.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
<?php

use App\Jobs\MilestoneRelease;
use App\Models\Repository;

use function Pest\Laravel\artisan;

it('cannot release milestone without a name', function () {
artisan('repository:milestone')
->assertFailed();
})->throws(RuntimeException::class);

it('cannot release milestone for a non-existing repository', function () {
artisan('repository:milestone', ['repository' => 'non-existing-repo'])
->assertFailed();
});

it('ensure MilestoneRelease job is dispatched', function () {
$repo = Repository::factory()->create();
Queue::fake();
artisan('repository:milestone', ['repository' => $repo->name])
->expectsOutput("Milestone release for $repo->name dispatched.")
->assertExitCode(0);
Queue::assertPushed(function (MilestoneRelease $job) use ($repo): bool {
return $job->repository->is($repo);
});
});
34 changes: 34 additions & 0 deletions tests/Feature/Http/MilestoneJobTest.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
<?php

use App\Jobs\MilestoneRelease;
use App\Models\Repository;

use function Pest\Laravel\post;
use function Pest\Laravel\withToken;

it('cannot access milestone route without auth', function () {
$repo = Repository::factory()->create();
post("repository/$repo->name/milestone")
->assertUnauthorized()
->assertHeader('WWW-Authenticate', 'Bearer');
withToken('random')->post("repository/$repo->name/milestone")
->assertUnauthorized()
->assertHeader('WWW-Authenticate', 'Bearer');
});

it('will dispatch a MilestoneRelease job', function () {
$repo = Repository::factory()->create();
Queue::fake();
withToken(config('repositories.milestone_token'))
->post("repository/$repo->name/milestone")
->assertOk();
Queue::assertPushed(function (MilestoneRelease $job) use ($repo): bool {
return $job->repository->is($repo);
});
});

it('cannot dispatch a MilestoneRelease job for a non-existent repository', function () {
withToken(config('repositories.milestone_token'))
->post('repository/non-existent/milestone')
->assertNotFound();
});

0 comments on commit fc890d5

Please sign in to comment.