Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement TextToImage OCP API #40326

Merged
merged 66 commits into from
Oct 26, 2023
Merged
Show file tree
Hide file tree
Changes from 60 commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
c8cab9d
Implement TextToImage OCP API
marcelklehr Sep 7, 2023
2d44c7c
Small fixes
marcelklehr Sep 7, 2023
666f7b3
Small fixes
marcelklehr Sep 7, 2023
dc8cba6
cs:Fix
marcelklehr Sep 8, 2023
1e36d74
Update core/Controller/TextToImageApiController.php
marcelklehr Oct 16, 2023
e199d1a
Update lib/public/TextToImage/IManager.php
marcelklehr Oct 16, 2023
53678a8
Update lib/public/TextToImage/Events/TaskSuccessfulEvent.php
marcelklehr Oct 16, 2023
41847c9
Update lib/private/TextToImage/TaskBackgroundJob.php
marcelklehr Oct 16, 2023
e5efbc8
enh(TextToImage): Address review comments
marcelklehr Oct 16, 2023
3e6a8b3
fix(TextToImage): Fix psalm errors
marcelklehr Oct 16, 2023
207c958
fix(TextToImage): Fix coding style
marcelklehr Oct 16, 2023
c59861a
enh(TextToImage): Implement removal of stale images and change Task#l…
marcelklehr Oct 17, 2023
f52d763
enh(TextToImage): Add routes
marcelklehr Oct 17, 2023
e8faaeb
enh(TextToImage): Allow anonymous access to IManager#getUserTasksByApp
marcelklehr Oct 17, 2023
1d07dcc
Update lib/private/TextToImage/Db/Task.php
marcelklehr Oct 18, 2023
4737238
Update lib/public/TextToImage/Exception/TaskNotFoundException.php
marcelklehr Oct 18, 2023
5ddf3c3
Update lib/public/TextToImage/Exception/TextToImageException.php
marcelklehr Oct 18, 2023
c5fbe5a
enh(TextToImage): Add bruteforce protection for anonymous API usage
marcelklehr Oct 18, 2023
d3da49d
fix(TextToImage): Fix docblock of getImage route
marcelklehr Oct 18, 2023
ca9a28a
fix(TextToImage): Fix notnull column to allow for empty strings on or…
marcelklehr Oct 18, 2023
9b7f639
fix(TextToImage): Fix psalm issues
marcelklehr Oct 18, 2023
ab856a5
fix(TextToImage): Fix psalm issues
marcelklehr Oct 18, 2023
e57e94e
fix(TextToImage): Add bruteforce protection to API
marcelklehr Oct 18, 2023
6238aca
fix(TextToImage): Fix bruteforce protection
marcelklehr Oct 18, 2023
3d11ab7
fix(TextToImage): Add autoloader changes and registerAlias
marcelklehr Oct 19, 2023
92cc171
fix(TextToImage): Fix OpenAPI definitions
marcelklehr Oct 19, 2023
8968573
enh(TextToImage): Add getExpectedRuntime to IProvider and run tasks d…
marcelklehr Oct 20, 2023
b7fd518
enh(TextToImage): Allow generating multiple images with one task
marcelklehr Oct 20, 2023
8ab47b6
enh(testing app): Add fake text2image provider
marcelklehr Oct 20, 2023
37c4ccc
fix(Text2Image): Fix Task#lastUpdated initialization to use DateTime …
marcelklehr Oct 20, 2023
7b7f552
fix(Text2Image): Fix OpenAPI types
marcelklehr Oct 20, 2023
bc85acf
fix(Text2Image): Fix psalm issues
marcelklehr Oct 20, 2023
73da7f2
fix(Text2Image): Add number_of_images to migration
marcelklehr Oct 20, 2023
4c58edc
fix(Text2Image): Fix psalm error
marcelklehr Oct 20, 2023
9ee7263
enh(Text2Image): Expose expected completion time
marcelklehr Oct 20, 2023
cee5aa8
fix(Text2Image): Fix psalm errors
marcelklehr Oct 20, 2023
9787f9d
enh(Text2Image): Add AI settings section to text2image
marcelklehr Oct 22, 2023
4055a90
fix(Text2Image): Fix $completionExpectedAt default value
marcelklehr Oct 22, 2023
b101859
fix(Text2Image): Fix appdata folder creation
marcelklehr Oct 22, 2023
6b6ac72
fix(Text2Image): Fix FakeText2ImageProvider in testing app
marcelklehr Oct 22, 2023
71a06b6
fix(Folder): Allow filename to be '0'
marcelklehr Oct 22, 2023
fa2fa47
fix(TextToImage/Manager): Appease psalm
marcelklehr Oct 22, 2023
bfc205f
fix(settings/AdminAI): Fix no-providers-installed view
marcelklehr Oct 22, 2023
6007b1b
fix(settings/AdminAI): Fix eslint
marcelklehr Oct 22, 2023
497559b
fix(RemoveOldTasksBackgroundJob): Update after change to support mult…
marcelklehr Oct 22, 2023
37419a9
fix: Update openapi.json
marcelklehr Oct 22, 2023
4624748
Update lib/public/TextToImage/Task.php
marcelklehr Oct 23, 2023
1f090a3
Update lib/public/TextToImage/Task.php
marcelklehr Oct 23, 2023
9144888
Update lib/private/TextToImage/RemoveOldTasksBackgroundJob.php
marcelklehr Oct 23, 2023
5d36c4c
Update lib/private/TextToImage/Manager.php
marcelklehr Oct 23, 2023
f9d209c
Update lib/private/TextToImage/Db/TaskMapper.php
marcelklehr Oct 23, 2023
ded7729
Update lib/private/TextToImage/Db/TaskMapper.php
marcelklehr Oct 23, 2023
5f815bd
fix(TextToImage\Db\Task): Inject ITimeFactory
marcelklehr Oct 23, 2023
68bb4a0
fix(TextToImage\Migration): Fix typo
marcelklehr Oct 23, 2023
4b8a58c
fix: appease linters and psalm
marcelklehr Oct 23, 2023
0c1bd84
fix: appease linters and psalm
marcelklehr Oct 23, 2023
8339b5b
fix: Minor copypasta
marcelklehr Oct 23, 2023
47e13cd
en(TextToImage): Use specific exception class instead of generic Runt…
marcelklehr Oct 26, 2023
14d1c18
en(TextToImage): Add getId method to IProvider
marcelklehr Oct 26, 2023
4e625f6
fix(TextToImage): Update testing provider to new interface
marcelklehr Oct 26, 2023
154bb53
Update core/Controller/TextToImageApiController.php
marcelklehr Oct 26, 2023
35bf7fc
fix(TextToImage): Fix copypasta
marcelklehr Oct 26, 2023
b9985bf
fix(TextToImage): Add missing task status update
marcelklehr Oct 26, 2023
9658d9c
Update lib/public/TextToImage/IManager.php
marcelklehr Oct 26, 2023
1823a8e
Update lib/public/TextToImage/IProvider.php
marcelklehr Oct 26, 2023
cef069e
fix(TextToImage): Update autoloaders
marcelklehr Oct 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/settings/lib/Controller/AISettingsController.php
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ public function __construct(
* @return DataResponse
*/
public function update($settings) {
$keys = ['ai.stt_provider', 'ai.textprocessing_provider_preferences', 'ai.translation_provider_preferences'];
$keys = ['ai.stt_provider', 'ai.textprocessing_provider_preferences', 'ai.translation_provider_preferences', 'ai.text2image_provider'];
foreach ($keys as $key) {
if (!isset($settings[$key])) {
continue;
Expand Down
11 changes: 11 additions & 0 deletions apps/settings/lib/Settings/Admin/ArtificialIntelligence.php
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@
private ISpeechToTextManager $sttManager,
private IManager $textProcessingManager,
private ContainerInterface $container,
private \OCP\TextToImage\IManager $text2imageManager,
) {
}

Expand Down Expand Up @@ -101,15 +102,25 @@
];
}

$text2imageProviders = [];
foreach ($this->text2imageManager->getProviders() as $provider) {
$text2imageProviders[] = [
'id' => $provider->getId(),
'name' => $provider->getName(),
];
}

$this->initialState->provideInitialState('ai-stt-providers', $sttProviders);
$this->initialState->provideInitialState('ai-translation-providers', $translationProviders);
$this->initialState->provideInitialState('ai-text-processing-providers', $textProcessingProviders);
$this->initialState->provideInitialState('ai-text-processing-task-types', $textProcessingTaskTypes);
$this->initialState->provideInitialState('ai-text2image-providers', $text2imageProviders);

$settings = [
'ai.stt_provider' => count($sttProviders) > 0 ? $sttProviders[0]['class'] : null,
'ai.textprocessing_provider_preferences' => $textProcessingSettings,
'ai.translation_provider_preferences' => $translationPreferences,
'ai.text2image_provider' => count($text2imageProviders) > 0 ? $text2imageProviders[0]['class'] : null,

Check failure on line 123 in apps/settings/lib/Settings/Admin/ArtificialIntelligence.php

View workflow job for this annotation

GitHub Actions / static-code-analysis

InvalidArrayOffset

apps/settings/lib/Settings/Admin/ArtificialIntelligence.php:123:66: InvalidArrayOffset: Cannot access value on variable $text2imageProviders[0] using offset value of 'class', expecting 'id' or 'name' (see https://psalm.dev/115)
Fixed Show fixed Hide fixed
];
foreach ($settings as $key => $defaultValue) {
$value = $defaultValue;
Expand Down
28 changes: 25 additions & 3 deletions apps/settings/src/components/AdminAI.vue
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,24 @@
</NcCheckboxRadioSwitch>
</template>
</NcSettingsSection>
<NcSettingsSection :name="t('settings', 'Image generation')"
:description="t('settings', 'Image generation can be implemented by different apps. Here you can set which app should be used.')">
<template v-for="provider in text2imageProviders">
<NcCheckboxRadioSwitch :key="provider.id"
:checked.sync="settings['ai.text2image_provider']"
:value="provider.id"
name="text2image_provider"
type="radio"
@update:checked="saveChanges">
{{ provider.name }}
</NcCheckboxRadioSwitch>
</template>
<template v-if="!hasText2ImageProviders">
<NcCheckboxRadioSwitch disabled type="radio">
{{ t('settings', 'None of your currently installed apps provide image generation functionality') }}
</NcCheckboxRadioSwitch>
</template>
</NcSettingsSection>
<NcSettingsSection :name="t('settings', 'Text processing')"
:description="t('settings', 'Text processing tasks can be implemented by different apps. Here you can set which app should be used for which task.')">
<template v-for="type in tpTaskTypes">
Expand Down Expand Up @@ -88,7 +106,7 @@ export default {
DragVerticalIcon,
ArrowDownIcon,
ArrowUpIcon,
NcButton
NcButton,
},
data() {
return {
Expand All @@ -100,6 +118,7 @@ export default {
translationProviders: loadState('settings', 'ai-translation-providers'),
textProcessingProviders: loadState('settings', 'ai-text-processing-providers'),
textProcessingTaskTypes: loadState('settings', 'ai-text-processing-task-types'),
text2imageProviders: loadState('settings', 'ai-text2image-providers'),
settings: loadState('settings', 'ai-settings'),
}
},
Expand All @@ -113,21 +132,24 @@ export default {
tpTaskTypes() {
return Object.keys(this.settings['ai.textprocessing_provider_preferences']).filter(type => !!this.getTaskType(type))
},
hasText2ImageProviders() {
return this.text2imageProviders.length > 0
},
},
methods: {
moveUp(i) {
this.settings['ai.translation_provider_preferences'].splice(
Math.min(i - 1, 0),
0,
...this.settings['ai.translation_provider_preferences'].splice(i, 1)
...this.settings['ai.translation_provider_preferences'].splice(i, 1),
)
this.saveChanges()
},
moveDown(i) {
this.settings['ai.translation_provider_preferences'].splice(
i + 1,
0,
...this.settings['ai.translation_provider_preferences'].splice(i, 1)
...this.settings['ai.translation_provider_preferences'].splice(i, 1),
)
this.saveChanges()
},
Expand Down
1 change: 1 addition & 0 deletions apps/testing/composer/composer/autoload_classmap.php
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
'OCA\\Testing\\Controller\\LockingController' => $baseDir . '/../lib/Controller/LockingController.php',
'OCA\\Testing\\Controller\\RateLimitTestController' => $baseDir . '/../lib/Controller/RateLimitTestController.php',
'OCA\\Testing\\Locking\\FakeDBLockingProvider' => $baseDir . '/../lib/Locking/FakeDBLockingProvider.php',
'OCA\\Testing\\Provider\\FakeText2ImageProvider' => $baseDir . '/../lib/Provider/FakeText2ImageProvider.php',
'OCA\\Testing\\Provider\\FakeTextProcessingProvider' => $baseDir . '/../lib/Provider/FakeTextProcessingProvider.php',
'OCA\\Testing\\Provider\\FakeTranslationProvider' => $baseDir . '/../lib/Provider/FakeTranslationProvider.php',
);
1 change: 1 addition & 0 deletions apps/testing/composer/composer/autoload_static.php
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ class ComposerStaticInitTesting
'OCA\\Testing\\Controller\\LockingController' => __DIR__ . '/..' . '/../lib/Controller/LockingController.php',
'OCA\\Testing\\Controller\\RateLimitTestController' => __DIR__ . '/..' . '/../lib/Controller/RateLimitTestController.php',
'OCA\\Testing\\Locking\\FakeDBLockingProvider' => __DIR__ . '/..' . '/../lib/Locking/FakeDBLockingProvider.php',
'OCA\\Testing\\Provider\\FakeText2ImageProvider' => __DIR__ . '/..' . '/../lib/Provider/FakeText2ImageProvider.php',
'OCA\\Testing\\Provider\\FakeTextProcessingProvider' => __DIR__ . '/..' . '/../lib/Provider/FakeTextProcessingProvider.php',
'OCA\\Testing\\Provider\\FakeTranslationProvider' => __DIR__ . '/..' . '/../lib/Provider/FakeTranslationProvider.php',
);
Expand Down
Binary file added apps/testing/img/logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions apps/testing/lib/AppInfo/Application.php
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
namespace OCA\Testing\AppInfo;

use OCA\Testing\AlternativeHomeUserBackend;
use OCA\Testing\Provider\FakeText2ImageProvider;
use OCA\Testing\Provider\FakeTranslationProvider;
use OCA\Testing\Provider\FakeTextProcessingProvider;
use OCP\AppFramework\App;
Expand All @@ -40,6 +41,7 @@ public function __construct(array $urlParams = []) {
public function register(IRegistrationContext $context): void {
$context->registerTranslationProvider(FakeTranslationProvider::class);
$context->registerTextProcessingProvider(FakeTextProcessingProvider::class);
$context->registerTextToImageProvider(FakeText2ImageProvider::class);
Fixed Show fixed Hide fixed
}

public function boot(IBootContext $context): void {
Expand Down
48 changes: 48 additions & 0 deletions apps/testing/lib/Provider/FakeText2ImageProvider.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
<?php
/**
* @copyright Copyright (c) 2023 Marcel Klehr <mklehr@gmx.net>
*
* @author Marcel Klehr <mklehr@gmx.net>
*
* @license GNU AGPL version 3 or any later version
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/

namespace OCA\Testing\Provider;

use OCP\TextToImage\IProvider;

class FakeText2ImageProvider implements IProvider {

public function getName(): string {
return 'Fake Text2Image provider';
}

public function generate(string $prompt, array $resources): void {
foreach ($resources as $resource) {
$read = fopen(__DIR__ . '/../../img/logo.png', 'r');
stream_copy_to_stream($read, $resource);
fclose($read);
}
}

public function getExpectedRuntime(): int {
return 1;
}

public function getId(): string {
return 'testing-fake-text2image-provider';
}
}
Loading
Loading