Idea: A simple model selection setting

[brainlid]

I'd like a simple way of indicating what LLM service and model a conversation should use. This also allows for changing it when testing and working with multiples.

For instance, when a system works with multiple model types like ChatGPT, Anthropic and Bumblebee hosted models, I would like to convey the selected service, model, and possibly some config all with a simple setting.

In an application, I may have a "conversation". The conversation could have a "model" field that is a string. I imagine the UI might be a select input where the possible values could be:

openai:gpt4-turbo
anthropic:claude-3-opus-20240229
bumblebee:llama3

We could potentially allow additional settings like this:

openai:gpt4-turbo:"{\"temperature\": 1}"

Using simple string splitting or binary pattern matching, getting the data out is easy. The 3rd position could be a JSON string of model config options that may be specific to that model.

Thoughts? Opinions?

[brainlid]

Added differently. Will be on main shortly.

[brainlid]

This is published in v0.3.0-rc.0.

Each chat model implements a serialize_config function. This turns the config into a simple, string keyed Elixir map. This can easily be stored on an Ecto schema using a field :model, :map definition.

The restore_from_map function does the reverse. A version value is included to allow for restoring an older model's config with a newer version of the chat model where settings may have changed.

The purpose is to all a simple, portable config to be used to define a chat model and be stored and recovered later.