[Question]:

[rajabhupati]

What is your question?

Is there a document that can help to understand how to integrate or create custom providers like databricks into this ###

More Details

Databricks recently started enabling LLMS ,similarly there could be new providers which might start supporting . How to contribute by creating new providers,what should be changed to enable them

What is the main subject of your question?

Documentation

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

There are two comprehensive guides on this:

https://docs.librechat.ai/install/configuration/custom_config.html#intro

https://docs.librechat.ai/install/configuration/ai_endpoints.html

[rajabhupati]

I tried the same guides and Removed env file from yaml and added - ./librechat.yaml:/app/librechat.yaml to volumes

# For more information, see the Configuration Guide:
# https://docs.librechat.ai/install/configuration/custom_config.html

# Configuration version (required)
version: 1.0.3

# Cache settings: Set to true to enable caching
# cache: true

# Example Registration Object Structure (optional)
# registration:
  # socialLogins: ["github", "google", "discord", "openid", "facebook"]
  # allowedDomains:
    # - "gmail.com"

# fileConfig:
#   endpoints:
#     assistants:
#       fileLimit: 5
#       fileSizeLimit: 10  # Maximum size for an individual file in MB
#       totalSizeLimit: 50  # Maximum total size for all files in a single request in MB
#       supportedMimeTypes:
#         - "image/.*"
#         - "application/pdf"
#     openAI:
#       disabled: true  # Disables file uploading to the OpenAI endpoint
#     default:
#       totalSizeLimit: 20
#     YourCustomEndpointName:
#       fileLimit: 2
#       fileSizeLimit: 5
#   serverFileSizeLimit: 100  # Global server file size limit in MB
#   avatarSizeLimit: 2  # Limit for user avatar image size in MB
# rateLimits:
#   fileUploads:
#     ipMax: 100
#     ipWindowInMinutes: 60  # Rate limit window for file uploads per IP
#     userMax: 50
#     userWindowInMinutes: 60  # Rate limit window for file uploads per user

# Definition of custom endpoints
endpoints:
  # assistants:
  #   disableBuilder: false # Disable Assistants Builder Interface by setting to `true`
  #   pollIntervalMs: 750  # Polling interval for checking assistant updates
  #   timeoutMs: 180000  # Timeout for assistant operations
  #   # Should only be one or the other, either `supportedIds` or `excludedIds`
  #   supportedIds: ["asst_supportedAssistantId1", "asst_supportedAssistantId2"]
  #   # excludedIds: ["asst_excludedAssistantId"]
  custom:
    # Mistral AI API
    - name: "LLM Gateway"  # Unique name for the endpoint
      # For `apiKey` and `baseURL`, you can use environment variables that you define.
      # recommended environment variables:
      apiKey: "mykey"
      baseURL: "https://databricks-host/serving-endpoints/gpt-35-turbo-us-east-2/invocations"

      # Models configuration
      #models: 
        # List of default models to use. At least one value is required.
        default: ["Databricks_GPT35_Turbo"]
        # Fetch option: Set to true to fetch models from API.
        fetch: true  # Defaults to false.

      # Optional configurations
      
      # Title Conversation setting
      titleConvo: true  # Set to true to enable title conversation

      # Title Method: Choose between "completion" or "functions".
      # titleMethod: "completion"  # Defaults to "completion" if omitted.

      # Title Model: Specify the model to use for titles.
      titleModel: "Databricks_GPT35_Turbo"  # Defaults to "gpt-3.5-turbo" if omitted.

      # Summarize setting: Set to true to enable summarization.
      # summarize: false

      # Summary Model: Specify the model to use if summarization is enabled.
      # summaryModel: "mistral-tiny"  # Defaults to "gpt-3.5-turbo" if omitted.

      # Force Prompt setting: If true, sends a `prompt` parameter instead of `messages`.
      # forcePrompt: false

      # The label displayed for the AI model in messages.
      modelDisplayLabel: "LlmGateway"  # Default is "AI" when not set.`

[rajabhupati]

Getting this error LibreChat | node:buffer:325
LibreChat | throw new ERR_INVALID_ARG_TYPE(
LibreChat | ^
LibreChat |
LibreChat | TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received undefined
LibreChat | at new NodeError (node:internal/errors:405:5)
LibreChat | at Function.from (node:buffer:325:9)
LibreChat | at Object. (/app/api/server/utils/crypto.js:4:20)
LibreChat | at Module._compile (node:internal/modules/cjs/loader:1356:14)
LibreChat | at Module._extensions..js (node:internal/modules/cjs/loader:1414:10)
LibreChat | at Module.load (node:internal/modules/cjs/loader:1197:32)
LibreChat | at Module._load (node:internal/modules/cjs/loader:1013:12)
LibreChat | at Module.require (node:internal/modules/cjs/loader:1225:19)
LibreChat | at require (node:internal/modules/helpers:177:18)
LibreChat | at Object. (/app/api/server/utils/index.js:5:21) {
LibreChat | code: 'ERR_INVALID_ARG_TYPE'
LibreChat | }
LibreChat |
LibreChat | Node.js v18.19.1
LibreChat exited with code 1

[fuegovic]

Did you used the local install method (npm) instead of docker?

Looks like you're missing some requireded .env variables.

• Start with the .env.example (make a copy and rename it .env),
• Leave everything in the new .env file as default, except the MONGO_URI, which you will need to provide yourself if you are not using docker. see: https://docs.librechat.ai/install/configuration/mongodb.html (If you are using docker, ignore this step, leave everything as default and try again)

[rajabhupati]

Thanks for insights . App is working fine now . I am not able to see my custom provider in the drop down in Librechat

I have the following entry in docker-compose - ./librechat.yaml:/app/librechat.yaml and in env file I enabled CONFIG_PATH="./librechat.yaml" and tried with CONFIG_PATH="./librechat.yaml"

This is my new error
LibreChat | 2024-03-04 07:00:28 info: Custom config file missing or YAML format invalid.
LibreChat |
LibreChat | Check out the latest config file guide for configurable options and features.
LibreChat | https://docs.librechat.ai/install/configuration/custom_config.html

[fuegovic]

• It is recommended to leave the docker-compose.yml file as is and use the override file instead. The docker-compose.override.yml file should contain this to enable the custom config file:

version: '3.4'

services:
  api:
    volumes:
      - ./librechat.yaml:/app/librechat.yaml

• You don't need to set the CONFIG_PATH if you left it in the default location, you should comment it out.
• Also you can check the validity of the yaml file here: https://yamlchecker.com/

[rajabhupati]

Despite trying all the steps . This is not working :-(. Yaml validation is fine

LibreChat | 2024-03-04 08:35:07 info: Custom config file missing or YAML format invalid.
LibreChat |
LibreChat | Check out the latest config file guide for configurable options and features.
LibreChat | https://docs.librechat.ai/install/configuration/custom_config.html
LibreChat |
LibreChat |
LibreChat | 2024-03-04 08:35:07 info: Server listening on all interfaces at port 3080. Use http://localhost:3080 to access it

[danny-avila]

I think this step is crucial, make sure you have it done: https://docs.librechat.ai/install/configuration/custom_config.html#docker-setup

That would involve using a separate file from the compose file, docker-compose.override.yml

Also do not use CONFIG_PATH for the default path (root) as already suggested.

You can try also docker compose build --no-cache

[rajabhupati]

Thanks this worked . We are good to close this question

…

On Mon, 4 Mar 2024 at 6:59 PM, Danny Avila ***@***.***> wrote: I think this step is crucial, make sure you have it done: https://docs.librechat.ai/install/configuration/custom_config.html#docker-setup That would involve using a separate file from the compose file, docker-compose.override.yml Also do not use CONFIG_PATH for the default path (root) as already suggestion. You can try also docker compose build --no-cache — Reply to this email directly, view it on GitHub <#1940 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AELPSQ5ZGJVLQEYXQNGLL23YWRZJZAVCNFSM6AAAAABEES2F5SVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DMNRWG43DO> . You are receiving this because you authored the thread.Message ID: ***@***.***>