Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement TextToImage OCP API #40326

Merged
merged 66 commits into from
Oct 26, 2023
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
c8cab9d
Implement TextToImage OCP API
marcelklehr Sep 7, 2023
2d44c7c
Small fixes
marcelklehr Sep 7, 2023
666f7b3
Small fixes
marcelklehr Sep 7, 2023
dc8cba6
cs:Fix
marcelklehr Sep 8, 2023
1e36d74
Update core/Controller/TextToImageApiController.php
marcelklehr Oct 16, 2023
e199d1a
Update lib/public/TextToImage/IManager.php
marcelklehr Oct 16, 2023
53678a8
Update lib/public/TextToImage/Events/TaskSuccessfulEvent.php
marcelklehr Oct 16, 2023
41847c9
Update lib/private/TextToImage/TaskBackgroundJob.php
marcelklehr Oct 16, 2023
e5efbc8
enh(TextToImage): Address review comments
marcelklehr Oct 16, 2023
3e6a8b3
fix(TextToImage): Fix psalm errors
marcelklehr Oct 16, 2023
207c958
fix(TextToImage): Fix coding style
marcelklehr Oct 16, 2023
c59861a
enh(TextToImage): Implement removal of stale images and change Task#l…
marcelklehr Oct 17, 2023
f52d763
enh(TextToImage): Add routes
marcelklehr Oct 17, 2023
e8faaeb
enh(TextToImage): Allow anonymous access to IManager#getUserTasksByApp
marcelklehr Oct 17, 2023
1d07dcc
Update lib/private/TextToImage/Db/Task.php
marcelklehr Oct 18, 2023
4737238
Update lib/public/TextToImage/Exception/TaskNotFoundException.php
marcelklehr Oct 18, 2023
5ddf3c3
Update lib/public/TextToImage/Exception/TextToImageException.php
marcelklehr Oct 18, 2023
c5fbe5a
enh(TextToImage): Add bruteforce protection for anonymous API usage
marcelklehr Oct 18, 2023
d3da49d
fix(TextToImage): Fix docblock of getImage route
marcelklehr Oct 18, 2023
ca9a28a
fix(TextToImage): Fix notnull column to allow for empty strings on or…
marcelklehr Oct 18, 2023
9b7f639
fix(TextToImage): Fix psalm issues
marcelklehr Oct 18, 2023
ab856a5
fix(TextToImage): Fix psalm issues
marcelklehr Oct 18, 2023
e57e94e
fix(TextToImage): Add bruteforce protection to API
marcelklehr Oct 18, 2023
6238aca
fix(TextToImage): Fix bruteforce protection
marcelklehr Oct 18, 2023
3d11ab7
fix(TextToImage): Add autoloader changes and registerAlias
marcelklehr Oct 19, 2023
92cc171
fix(TextToImage): Fix OpenAPI definitions
marcelklehr Oct 19, 2023
8968573
enh(TextToImage): Add getExpectedRuntime to IProvider and run tasks d…
marcelklehr Oct 20, 2023
b7fd518
enh(TextToImage): Allow generating multiple images with one task
marcelklehr Oct 20, 2023
8ab47b6
enh(testing app): Add fake text2image provider
marcelklehr Oct 20, 2023
37c4ccc
fix(Text2Image): Fix Task#lastUpdated initialization to use DateTime …
marcelklehr Oct 20, 2023
7b7f552
fix(Text2Image): Fix OpenAPI types
marcelklehr Oct 20, 2023
bc85acf
fix(Text2Image): Fix psalm issues
marcelklehr Oct 20, 2023
73da7f2
fix(Text2Image): Add number_of_images to migration
marcelklehr Oct 20, 2023
4c58edc
fix(Text2Image): Fix psalm error
marcelklehr Oct 20, 2023
9ee7263
enh(Text2Image): Expose expected completion time
marcelklehr Oct 20, 2023
cee5aa8
fix(Text2Image): Fix psalm errors
marcelklehr Oct 20, 2023
9787f9d
enh(Text2Image): Add AI settings section to text2image
marcelklehr Oct 22, 2023
4055a90
fix(Text2Image): Fix $completionExpectedAt default value
marcelklehr Oct 22, 2023
b101859
fix(Text2Image): Fix appdata folder creation
marcelklehr Oct 22, 2023
6b6ac72
fix(Text2Image): Fix FakeText2ImageProvider in testing app
marcelklehr Oct 22, 2023
71a06b6
fix(Folder): Allow filename to be '0'
marcelklehr Oct 22, 2023
fa2fa47
fix(TextToImage/Manager): Appease psalm
marcelklehr Oct 22, 2023
bfc205f
fix(settings/AdminAI): Fix no-providers-installed view
marcelklehr Oct 22, 2023
6007b1b
fix(settings/AdminAI): Fix eslint
marcelklehr Oct 22, 2023
497559b
fix(RemoveOldTasksBackgroundJob): Update after change to support mult…
marcelklehr Oct 22, 2023
37419a9
fix: Update openapi.json
marcelklehr Oct 22, 2023
4624748
Update lib/public/TextToImage/Task.php
marcelklehr Oct 23, 2023
1f090a3
Update lib/public/TextToImage/Task.php
marcelklehr Oct 23, 2023
9144888
Update lib/private/TextToImage/RemoveOldTasksBackgroundJob.php
marcelklehr Oct 23, 2023
5d36c4c
Update lib/private/TextToImage/Manager.php
marcelklehr Oct 23, 2023
f9d209c
Update lib/private/TextToImage/Db/TaskMapper.php
marcelklehr Oct 23, 2023
ded7729
Update lib/private/TextToImage/Db/TaskMapper.php
marcelklehr Oct 23, 2023
5f815bd
fix(TextToImage\Db\Task): Inject ITimeFactory
marcelklehr Oct 23, 2023
68bb4a0
fix(TextToImage\Migration): Fix typo
marcelklehr Oct 23, 2023
4b8a58c
fix: appease linters and psalm
marcelklehr Oct 23, 2023
0c1bd84
fix: appease linters and psalm
marcelklehr Oct 23, 2023
8339b5b
fix: Minor copypasta
marcelklehr Oct 23, 2023
47e13cd
en(TextToImage): Use specific exception class instead of generic Runt…
marcelklehr Oct 26, 2023
14d1c18
en(TextToImage): Add getId method to IProvider
marcelklehr Oct 26, 2023
4e625f6
fix(TextToImage): Update testing provider to new interface
marcelklehr Oct 26, 2023
154bb53
Update core/Controller/TextToImageApiController.php
marcelklehr Oct 26, 2023
35bf7fc
fix(TextToImage): Fix copypasta
marcelklehr Oct 26, 2023
b9985bf
fix(TextToImage): Add missing task status update
marcelklehr Oct 26, 2023
9658d9c
Update lib/public/TextToImage/IManager.php
marcelklehr Oct 26, 2023
1823a8e
Update lib/public/TextToImage/IProvider.php
marcelklehr Oct 26, 2023
cef069e
fix(TextToImage): Update autoloaders
marcelklehr Oct 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
234 changes: 234 additions & 0 deletions core/Controller/TextToImageApiController.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,234 @@
<?php

declare(strict_types=1);

/**
* @copyright Copyright (c) 2023 Marcel Klehr <mklehr@gmx.net>
*
* @author Marcel Klehr <mklehr@gmx.net>
*
* @license GNU AGPL version 3 or any later version
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/


namespace OC\Core\Controller;

use OC\Files\AppData\AppData;
use OCA\Core\ResponseDefinitions;
use OCP\AppFramework\Http;
use OCP\AppFramework\Http\Attribute\AnonRateLimit;
use OCP\AppFramework\Http\Attribute\BruteForceProtection;
use OCP\AppFramework\Http\Attribute\NoAdminRequired;
use OCP\AppFramework\Http\Attribute\PublicPage;
use OCP\AppFramework\Http\Attribute\UserRateLimit;
use OCP\AppFramework\Http\DataResponse;
use OCP\AppFramework\Http\FileDisplayResponse;
use OCP\Files\NotFoundException;
use OCP\IL10N;
use OCP\IRequest;
use OCP\TextToImage\Exception\TaskNotFoundException;
use OCP\TextToImage\Task;
use OCP\TextToImage\IManager;
use OCP\PreConditionNotMetException;

/**
* @psalm-import-type CoreTextToImageTask from ResponseDefinitions
*/
class TextToImageApiController extends \OCP\AppFramework\OCSController {
public function __construct(
string $appName,
IRequest $request,
private IManager $textToImageManager,
private IL10N $l,
private ?string $userId,
private AppData $appData,
) {
parent::__construct($appName, $request);
}

/**
* Check whether this feature is available
*
* @return DataResponse<Http::STATUS_OK, array{isAvailable: bool}, array{}>
*
* 200: Returns availability status
*/
#[PublicPage]
public function isAvailable(): DataResponse {
return new DataResponse([
'isAvailable' => $this->textToImageManager->hasProviders(),
]);
}

/**
* This endpoint allows scheduling a text to image task
*
* @param string $input Input text
* @param string $appId ID of the app that will execute the task
* @param string $identifier An arbitrary identifier for the task
*
* @return DataResponse<Http::STATUS_OK, array{task: CoreTextToImageTask}, array{}>|DataResponse<Http::STATUS_PRECONDITION_FAILED, array{message: string}, array{}>
Fixed Show fixed Hide fixed
*
* 200: Task scheduled successfully
* 412: Scheduling task is not possible
*/
#[PublicPage]
#[UserRateLimit(limit: 20, period: 120)]
#[AnonRateLimit(limit: 5, period: 120)]
public function schedule(string $input, string $appId, string $identifier = ''): DataResponse {
$task = new Task($input, $appId, $this->userId, $identifier);
try {
$this->textToImageManager->scheduleTask($task);

$json = $task->jsonSerialize();

return new DataResponse([
'task' => $json,
]);
} catch (PreConditionNotMetException) {
return new DataResponse(['message' => $this->l->t('No text to image provider is available')], Http::STATUS_PRECONDITION_FAILED);
}
}

/**
* This endpoint allows checking the status and results of a task.
* Tasks are removed 1 week after receiving their last update.
*
* @param int $id The id of the task
*
* @return DataResponse<Http::STATUS_OK, array{task: CoreTextToImageTask}, array{}>|DataResponse<Http::STATUS_NOT_FOUND|Http::STATUS_INTERNAL_SERVER_ERROR, array{message: string}, array{}>
*
* 200: Task returned
* 404: Task not found
*/
#[PublicPage]
marcelklehr marked this conversation as resolved.
Show resolved Hide resolved
#[BruteForceProtection(action: 'text2image')]
public function getTask(int $id): DataResponse {
try {
$task = $this->textToImageManager->getUserTask($id, $this->userId);

$json = $task->jsonSerialize();

return new DataResponse([
'task' => $json,
]);
} catch (TaskNotFoundException) {
$res = new DataResponse(['message' => $this->l->t('Task not found')], Http::STATUS_NOT_FOUND);
$res->throttle(['action' => 'text2image']);
return $res;
} catch (\RuntimeException) {
return new DataResponse(['message' => $this->l->t('Internal error')], Http::STATUS_INTERNAL_SERVER_ERROR);
}
}

/**
* This endpoint allows downloading the resulting image of a task
*
* @param int $id The id of the task
*
* @return FileDisplayResponse<Http::STATUS_OK, array{'Content-Type': string}>|DataResponse<Http::STATUS_NOT_FOUND|Http::STATUS_INTERNAL_SERVER_ERROR, array{message: string}, array{}>
*
* 200: Image returned
* 404: Task or image not found
*/
#[PublicPage]
#[BruteForceProtection(action: 'text2image')]
public function getImage(int $id): DataResponse|FileDisplayResponse {
try {
$task = $this->textToImageManager->getUserTask($id, $this->userId);
try {
$folder = $this->appData->getFolder('text2image');
} catch(NotFoundException) {
$folder = $this->appData->newFolder('text2image');
marcelklehr marked this conversation as resolved.
Show resolved Hide resolved
}
$file = $folder->getFile((string)$task->getId());
$info = getimagesizefromstring($file->getContent());

return new FileDisplayResponse($file, Http::STATUS_OK, ['Content-Type' => image_type_to_mime_type($info[2])]);
} catch (TaskNotFoundException) {
$res = new DataResponse(['message' => $this->l->t('Task not found')], Http::STATUS_NOT_FOUND);
$res->throttle(['action' => 'text2image']);
return $res;
} catch (\RuntimeException) {
return new DataResponse(['message' => $this->l->t('Internal error')], Http::STATUS_INTERNAL_SERVER_ERROR);
} catch (NotFoundException) {
$res = new DataResponse(['message' => $this->l->t('Image not found')], Http::STATUS_NOT_FOUND);
$res->throttle(['action' => 'text2image']);
return $res;
}
}

/**
* This endpoint allows to delete a scheduled task for a user
*
* @param int $id The id of the task
*
* @return DataResponse<Http::STATUS_OK, array{task: CoreTextToImageTask}, array{}>|DataResponse<Http::STATUS_NOT_FOUND|Http::STATUS_INTERNAL_SERVER_ERROR, array{message: string}, array{}>
*
* 200: Task returned
* 404: Task not found
*/
#[NoAdminRequired]
#[BruteForceProtection(action: 'text2image')]
public function deleteTask(int $id): DataResponse {
try {
$task = $this->textToImageManager->getUserTask($id, $this->userId);

$this->textToImageManager->deleteTask($task);

$json = $task->jsonSerialize();

return new DataResponse([
'task' => $json,
]);
} catch (TaskNotFoundException) {
$res = new DataResponse(['message' => $this->l->t('Task not found')], Http::STATUS_NOT_FOUND);
$res->throttle(['action' => 'text2image']);
return $res;
} catch (\RuntimeException) {
return new DataResponse(['message' => $this->l->t('Internal error')], Http::STATUS_INTERNAL_SERVER_ERROR);
}
}


/**
* This endpoint returns a list of tasks of a user that are related
* with a specific appId and optionally with an identifier
*
* @param string $appId ID of the app
* @param string|null $identifier An arbitrary identifier for the task
* @return DataResponse<Http::STATUS_OK, array{tasks: CoreTextToImageTask[]}, array{}>|DataResponse<Http::STATUS_INTERNAL_SERVER_ERROR, array{message: string}, array{}>
*
* 200: Task list returned
*/
#[NoAdminRequired]
#[AnonRateLimit(limit: 5, period: 120)]
public function listTasksByApp(string $appId, ?string $identifier = null): DataResponse {
try {
$tasks = $this->textToImageManager->getUserTasksByApp($this->userId, $appId, $identifier);
Fixed Show fixed Hide fixed
/** @var CoreTextToImageTask[] $json */
$json = array_map(static function (Task $task) {
return $task->jsonSerialize();
}, $tasks);

return new DataResponse([
'tasks' => $json,
]);
} catch (\RuntimeException) {
return new DataResponse(['message' => $this->l->t('Internal error')], Http::STATUS_INTERNAL_SERVER_ERROR);
}
}
}
92 changes: 92 additions & 0 deletions core/Migrations/Version28000Date20230906104802.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
<?php

declare(strict_types=1);

/**
* @copyright Copyright (c) 2023 Marcel Klehr <mklehr@gmx.net>
*
* @author Marcel Klehr <mklehr@gmx.net>
*
* @license GNU AGPL version 3 or any later version
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
*/

namespace OC\Core\Migrations;

use Closure;
use OCP\DB\ISchemaWrapper;
use OCP\DB\Types;
use OCP\Migration\IOutput;
use OCP\Migration\SimpleMigrationStep;

/**
* Introduce text2image_tasks table
*/
class Version28000Date20230906104802 extends SimpleMigrationStep {
/**
* @param IOutput $output
* @param Closure $schemaClosure The `\Closure` returns a `ISchemaWrapper`
* @param array $options
* @return null|ISchemaWrapper
*/
public function changeSchema(IOutput $output, Closure $schemaClosure, array $options): ?ISchemaWrapper {
/** @var ISchemaWrapper $schema */
$schema = $schemaClosure();
if (!$schema->hasTable('text2image_tasks')) {
$table = $schema->createTable('text2image_tasks');

$table->addColumn('id', Types::BIGINT, [
'notnull' => true,
'length' => 64,
'autoincrement' => true,
]);
$table->addColumn('input', Types::TEXT, [
'notnull' => true,
]);
$table->addColumn('status', Types::INTEGER, [
'notnull' => false,
'length' => 6,
'default' => 0,
]);
$table->addColumn('user_id', Types::STRING, [
'notnull' => false,
'length' => 64,
]);
$table->addColumn('app_id', Types::STRING, [
'notnull' => true,
'length' => 32,
'default' => '',
]);
$table->addColumn('identifier', Types::STRING, [
'notnull' => false,
'length' => 255,
'default' => '',
]);
$table->addColumn('last_updated', Types::DATETIME, [
'notnull' => false,
]);

$table->setPrimaryKey(['id'], 't2i_tasks_id_index');
$table->addIndex(['last_updated'], 't2i_tasks_updated');
$table->addIndex(['status'], 't2i_tasks_status');
$table->addIndex(['user_id', 'app_id', 'identifier'], 't2i_tasks_uid_appid_ident');

return $schema;
}

return null;
}
}
9 changes: 9 additions & 0 deletions core/ResponseDefinitions.php
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,15 @@
* output: ?string,
* identifier: string,
* }
*
* @psalm-type CoreTextToImageTask = array{
* id: ?int,
* status: 0|1|2|3|4,
* userId: ?string,
* appId: string,
* input: string,
* identifier: ?string,
* }
*/
class ResponseDefinitions {
}
Loading
Loading