Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Setting for tweaking merging strategy for JSON arrays in config… #622

Merged
merged 2 commits into from
Dec 20, 2024
Merged

feat: Setting for tweaking merging strategy for JSON arrays in config… #622

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e7d6a43
feat: Setting for tweaking merging strategy for JSON arrays in config…
PMitrafanau Dec 19, 2024
e4f55ef
feat: fix PR comments v1
PMitrafanau Dec 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 51 additions & 50 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,56 +46,57 @@ Priority order:
2. File specified in "AIDIAL_SETTINGS" environment variable.
3. Default resource file: src/main/resources/aidial.settings.json.

| Setting | Default | Required | Description
|-----------------------------------------------|:--------------------------:|:--------:|-------------------------------------------------|
| config.files | aidial.config.json | No |List of paths to dynamic settings. Refer to [example](sample/aidial.config.json) of the file with [dynamic settings](#dynamic-settings).|
| config.reload | 60000 | No |Config reload interval in milliseconds.
| identityProviders | - | Yes |Map of identity providers. **Note**: At least one identity provider must be provided. Refer to [examples](sample/aidial.settings.json) to view available providers. Refer to [IDP Configuration](https://github.com/epam/ai-dial/blob/main/docs/Auth/2.%20Web/1.overview.md) to view guidelines for configuring supported providers.
| identityProviders.*.jwksUrl | - | Optional |Url to jwks provider. **Required** if `disabledVerifyJwt` is set to `false`. **Note**: Either `jwksUrl` or `userInfoEndpoint` must be provided.
| identityProviders.*.userInfoEndpoint | - | Optional |Url to user info endpoint. **Note**: Either `jwksUrl` or `userInfoEndpoint` must be provided or `disableJwtVerification` is unset. Refer to [Google example](sample/aidial.settings.json).
| identityProviders.*.rolePath | - | Yes |Path(s) to the claim user roles in JWT token or user info response, e.g. `resource_access.chatbot-ui.roles` or just `roles`. Can be single String or Array of Strings. Refer to [IDP Configuration](https://github.com/epam/ai-dial/blob/main/docs/Auth/2.%20Web/1.overview.md) to view guidelines for configuring supported providers.
| identityProviders.*.projectPath | - | No |Path(s) to the claim in JWT token or user info response, e.g. `azp`, `aud` or `some.path.client` from which project name can be taken. Can be single String. Refer to [IDP Configuration](https://github.com/epam/ai-dial/blob/main/docs/Auth/2.%20Web/1.overview.md) to view guidelines for configuring supported providers.
| identityProviders.*.rolesDelimiter | - | No |Delimiter to split roles into array in case when list of roles presented as single String. e.g. `"rolesDelimiter": " "`
| identityProviders.*.loggingKey | - | No |User information to search in claims of JWT token. `email` or `sub` should be sufficient in most cases. **Note**: `email` might be unavailable for some IDPs. Please check your IDP documentation in this case.
| identityProviders.*.loggingSalt | - | No |Salt to hash user information for logging.
| identityProviders.*.positiveCacheExpirationMs | 600000 | No | How long to retain JWKS response in the cache in case of successfull response.
| identityProviders.*.negativeCacheExpirationMs | 10000 | No |How long to retain JWKS response in the cache in case of failed response.
| identityProviders.*.issuerPattern | - | No |Regexp to match the claim "iss" to identity provider.
| identityProviders.*.disableJwtVerification | false | No |The flag disables JWT verification. *Note*. `userInfoEndpoint` must be unset if the flag is set to `true`.
| vertx.* | - | No |Vertx settings. Refer to [vertx.io](https://vertx.io/docs/apidocs/io/vertx/core/VertxOptions.html) to learn more.
| server.* | - | No |Vertx HTTP server settings for incoming requests.
| client.* | - | No |Vertx HTTP client settings for outbound requests.
| storage.provider | filesystem | Yes |Specifies blob storage provider. Supported providers: s3, aws-s3, azureblob, google-cloud-storage, filesystem. See examples in the sections below.
| storage.endpoint | - | Optional |Specifies endpoint url for s3 compatible storages. **Note**: The setting might be required. That depends on a concrete provider.
| storage.identity | - | Optional |Blob storage access key. Can be optional for filesystem, aws-s3, google-cloud-storage providers. Refer to [sections in this document](#aws-s3-blob-store) dedicated to specific storage providers.
| storage.credential | - | Optional |Blob storage secret key. Can be optional for filesystem, aws-s3, google-cloud-storage providers.
| storage.bucket | - | No |Blob storage bucket.
| storage.overrides.* | - | No |Key-value pairs to override storage settings. `*` might be any specific blob storage setting to be overridden. Refer to [examples](#temporary-credentials-1) in the sections below.
| storage.createBucket | false | No |Indicates whether bucket should be created on start-up.
| storage.prefix | - | No |Base prefix for all stored resources. The purpose to use the same bucket for different environments, e.g. dev, prod, pre-prod. Must not contain path separators or any invalid chars.
| storage.maxUploadedFileSize | 536870912 | No |Maximum size in bytes of uploaded file. If a size of uploaded file exceeds the limit the server returns HTTP code 413
| encryption.secret | - | No |Secret is used for AES encryption of a prefix to the bucket blob storage. The value should be random generated string.
| encryption.key | - | No |Key is used for AES encryption of a prefix to the bucket blob storage. The value should be random generated string.
| resources.maxSize | 1048576 | No |Max allowed size in bytes for a resource.
| resources.syncPeriod | 60000 | No |Period in milliseconds, how frequently check for resources to sync.
| resources.syncDelay | 120000 | No |Delay in milliseconds for a resource to be written back in object storage after last modification.
| resources.syncBatch | 4096 | No |How many resources to sync in one go.
| resources.cacheExpiration | 300000 | No |Expiration in milliseconds for synced resources in Redis.
| resources.compressionMinSize | 256 | No |Compress a resource with gzip if its size in bytes more or equal to this value.
| redis.singleServerConfig.address | - | Yes |Redis single server addresses, e.g. "redis://host:port". Either `singleServerConfig` or `clusterServersConfig` must be provided.
| redis.clusterServersConfig.nodeAddresses | - | Yes |Json array with Redis cluster server addresses, e.g. ["redis://host1:port1","redis://host2:port2"]. Either `singleServerConfig` or `clusterServersConfig` must be provided.
| redis.provider.* | - | No |Provider specific settings
| redis.provider.name | - | Yes |Provider name. The valid values are `aws-elasti-cache`(see [instructions](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/auth-iam.html)).
| redis.provider.userId | - | Yes | IAM-enabled user ID. **Note**. It's applied to `aws-elasti-cache`
| redis.provider.region | - | Yes | Geo region where the cache is located. **Note**. It's applied to `aws-elasti-cache`
| redis.provider.clusterName | - | Yes | Redis cluster name. **Note**. It's applied to `aws-elasti-cache`
| redis.provider.serverless | - | Yes | The flag indicates if the cache is serverless. **Note**. It's applied to `aws-elasti-cache`
| invitations.ttlInSeconds | 259200 | No |Invitation time to live in seconds.
| access.admin.rules | - | No |Matches claims from identity providers with the rules to figure out whether a user is allowed to perform admin actions, like deleting any resource or approving a publication. Example: [{"source": "roles", "function": "EQUAL", "targets": ["admin"]}]. If roles contain "admin, the actions are allowed.
| applications.includeCustomApps | false | No |The flag indicates whether custom applications should be included into openai listing
| applications.controllerEndpoint | - | No |The endpoint to Application Controller Web Service that manages deployments for applications with functions
| applications.controllerTimeout | 240000 | No |The timeout of operations to Application Controller Web Service
| applications.checkPeriod | 300000 | No |The interval at which to check the pending operations for applications with functions
| Setting | Default | Required | Description
|-----------------------------------------------|:------------------:|:--------:|-------------------------------------------------|
| config.files | aidial.config.json | No |List of paths to dynamic settings. Refer to [example](sample/aidial.config.json) of the file with [dynamic settings](#dynamic-settings).|
| config.reload | 60000 | No |Config reload interval in milliseconds.
| config.jsonMergeStrategy.overwriteArrays | false | No |Specifies a merging strategy for JSON arrays. If it's set to `true`, arrays will be overwritten. Otherwise, they will be concatenated.
| identityProviders | - | Yes |Map of identity providers. **Note**: At least one identity provider must be provided. Refer to [examples](sample/aidial.settings.json) to view available providers. Refer to [IDP Configuration](https://github.com/epam/ai-dial/blob/main/docs/Auth/2.%20Web/1.overview.md) to view guidelines for configuring supported providers.
| identityProviders.*.jwksUrl | - | Optional |Url to jwks provider. **Required** if `disabledVerifyJwt` is set to `false`. **Note**: Either `jwksUrl` or `userInfoEndpoint` must be provided.
| identityProviders.*.userInfoEndpoint | - | Optional |Url to user info endpoint. **Note**: Either `jwksUrl` or `userInfoEndpoint` must be provided or `disableJwtVerification` is unset. Refer to [Google example](sample/aidial.settings.json).
| identityProviders.*.rolePath | - | Yes |Path(s) to the claim user roles in JWT token or user info response, e.g. `resource_access.chatbot-ui.roles` or just `roles`. Can be single String or Array of Strings. Refer to [IDP Configuration](https://github.com/epam/ai-dial/blob/main/docs/Auth/2.%20Web/1.overview.md) to view guidelines for configuring supported providers.
| identityProviders.*.projectPath | - | No |Path(s) to the claim in JWT token or user info response, e.g. `azp`, `aud` or `some.path.client` from which project name can be taken. Can be single String. Refer to [IDP Configuration](https://github.com/epam/ai-dial/blob/main/docs/Auth/2.%20Web/1.overview.md) to view guidelines for configuring supported providers.
| identityProviders.*.rolesDelimiter | - | No |Delimiter to split roles into array in case when list of roles presented as single String. e.g. `"rolesDelimiter": " "`
| identityProviders.*.loggingKey | - | No |User information to search in claims of JWT token. `email` or `sub` should be sufficient in most cases. **Note**: `email` might be unavailable for some IDPs. Please check your IDP documentation in this case.
| identityProviders.*.loggingSalt | - | No |Salt to hash user information for logging.
| identityProviders.*.positiveCacheExpirationMs | 600000 | No | How long to retain JWKS response in the cache in case of successfull response.
| identityProviders.*.negativeCacheExpirationMs | 10000 | No |How long to retain JWKS response in the cache in case of failed response.
| identityProviders.*.issuerPattern | - | No |Regexp to match the claim "iss" to identity provider.
| identityProviders.*.disableJwtVerification | false | No |The flag disables JWT verification. *Note*. `userInfoEndpoint` must be unset if the flag is set to `true`.
| vertx.* | - | No |Vertx settings. Refer to [vertx.io](https://vertx.io/docs/apidocs/io/vertx/core/VertxOptions.html) to learn more.
| server.* | - | No |Vertx HTTP server settings for incoming requests.
| client.* | - | No |Vertx HTTP client settings for outbound requests.
| storage.provider | filesystem | Yes |Specifies blob storage provider. Supported providers: s3, aws-s3, azureblob, google-cloud-storage, filesystem. See examples in the sections below.
| storage.endpoint | - | Optional |Specifies endpoint url for s3 compatible storages. **Note**: The setting might be required. That depends on a concrete provider.
| storage.identity | - | Optional |Blob storage access key. Can be optional for filesystem, aws-s3, google-cloud-storage providers. Refer to [sections in this document](#aws-s3-blob-store) dedicated to specific storage providers.
| storage.credential | - | Optional |Blob storage secret key. Can be optional for filesystem, aws-s3, google-cloud-storage providers.
| storage.bucket | - | No |Blob storage bucket.
| storage.overrides.* | - | No |Key-value pairs to override storage settings. `*` might be any specific blob storage setting to be overridden. Refer to [examples](#temporary-credentials-1) in the sections below.
| storage.createBucket | false | No |Indicates whether bucket should be created on start-up.
| storage.prefix | - | No |Base prefix for all stored resources. The purpose to use the same bucket for different environments, e.g. dev, prod, pre-prod. Must not contain path separators or any invalid chars.
| storage.maxUploadedFileSize | 536870912 | No |Maximum size in bytes of uploaded file. If a size of uploaded file exceeds the limit the server returns HTTP code 413
| encryption.secret | - | No |Secret is used for AES encryption of a prefix to the bucket blob storage. The value should be random generated string.
| encryption.key | - | No |Key is used for AES encryption of a prefix to the bucket blob storage. The value should be random generated string.
| resources.maxSize | 1048576 | No |Max allowed size in bytes for a resource.
| resources.syncPeriod | 60000 | No |Period in milliseconds, how frequently check for resources to sync.
| resources.syncDelay | 120000 | No |Delay in milliseconds for a resource to be written back in object storage after last modification.
| resources.syncBatch | 4096 | No |How many resources to sync in one go.
| resources.cacheExpiration | 300000 | No |Expiration in milliseconds for synced resources in Redis.
| resources.compressionMinSize | 256 | No |Compress a resource with gzip if its size in bytes more or equal to this value.
| redis.singleServerConfig.address | - | Yes |Redis single server addresses, e.g. "redis://host:port". Either `singleServerConfig` or `clusterServersConfig` must be provided.
| redis.clusterServersConfig.nodeAddresses | - | Yes |Json array with Redis cluster server addresses, e.g. ["redis://host1:port1","redis://host2:port2"]. Either `singleServerConfig` or `clusterServersConfig` must be provided.
| redis.provider.* | - | No |Provider specific settings
| redis.provider.name | - | Yes |Provider name. The valid values are `aws-elasti-cache`(see [instructions](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/auth-iam.html)).
| redis.provider.userId | - | Yes | IAM-enabled user ID. **Note**. It's applied to `aws-elasti-cache`
| redis.provider.region | - | Yes | Geo region where the cache is located. **Note**. It's applied to `aws-elasti-cache`
| redis.provider.clusterName | - | Yes | Redis cluster name. **Note**. It's applied to `aws-elasti-cache`
| redis.provider.serverless | - | Yes | The flag indicates if the cache is serverless. **Note**. It's applied to `aws-elasti-cache`
| invitations.ttlInSeconds | 259200 | No |Invitation time to live in seconds.
| access.admin.rules | - | No |Matches claims from identity providers with the rules to figure out whether a user is allowed to perform admin actions, like deleting any resource or approving a publication. Example: [{"source": "roles", "function": "EQUAL", "targets": ["admin"]}]. If roles contain "admin, the actions are allowed.
| applications.includeCustomApps | false | No |The flag indicates whether custom applications should be included into openai listing
| applications.controllerEndpoint | - | No |The endpoint to Application Controller Web Service that manages deployments for applications with functions
| applications.controllerTimeout | 240000 | No |The timeout of operations to Application Controller Web Service
| applications.checkPeriod | 300000 | No |The interval at which to check the pending operations for applications with functions

### Storage requirements

Expand Down
Loading
Loading