Enhanced Credential and Model Management

[emilkje]

Feature Proposal: Enhanced Credential and Model Management

Summary

The proposed feature request aims to enhance the cwc tool by overhauling the existing credential and model management system. This reimagined structure introduces a robust set of credential and model commands, allowing users to add, list, remove, and seamlessly switch between different sets of credentials and models. The redesign not only simplifies the user experience for Azure OpenAI services but also establishes a foundation to support additional OpenAI-compliant providers. By embracing this more adaptable and extensible approach, the cwc tool will be better positioned to serve the evolving needs of users working with a diverse range of models, AI platforms and environments.

Background

The cwc tool is designed to streamline interactions with the LLM services by providing a simple command-line interface. As the scale of operations grows and users begin to work with multiple models, the need for efficient credential and model management becomes increasingly evident.

Credential Management Overhaul: The current system implements a single pair of credentials tied to a specific model deployment. This approach limits flexibility and efficiency, especially for users who work with multiple models and potentially across different Azure OpenAI or different providers entirely. By enhancing the credential commands, users will be able to add, list, and remove credentials seamlessly. This improvement will make managing multiple sets of credentials more intuitive and less error-prone.

Redefining Model Association: We propose decoupling the model deployment identifiers from the credentials, thus allowing credentials to be associated with multiple models. This shift will permit users to define model configurations independently from their credentials and switch between models as needed without redundant authentication steps.

Enhanced User Experience: By introducing these changes, we aim to provide a user experience that is more aligned with the end-users workflows where multiple models and credentials are often in use. The proposed command structure is designed to be self-explanatory, which will help both new and existing users adapt quickly to the changes.

Credential Management Commands

credential add

• Description: Add new API credentials and assign them a reference name.
• Usage: cwc credential add "credential-name"
• Flags:
  • --api-key [-k]: The API key for Azure OpenAI.
  • --endpoint [-e]: The Azure OpenAI API Endpoint.
• Flow:
  1. User invokes credential add with the required flags.
  2. Input is collected and securely stored, associated with the "credential-name".

credential list

• Description: List all stored credentials by their reference names.
• Usage: cwc credential list
• Flow:
  1. Command displays all credentials with their names and endpoints in a tabular format.

credential remove

• Description: Remove specified credentials using their reference name.
• Usage: cwc credential remove "credential-name"
• Flow:
  1. User specifies the "credential-name" to remove.
  2. System deletes the credential after obtaining user confirmation.

Model Management Commands

model add

• Description: Add a new model and associate it with existing credentials.
• Usage: cwc model add "modelname" --with-credential "credential-name"
• Flags:
  • --with-credential: Name of the credential to associate with the model.
• Flow:
  1. User specifies a model name and its associated credential name.
  2. Model configuration is saved with the provided associations.

model list

• Description: List all models and their associated credentials.
• Usage: cwc model list
• Flow:
  1. Command lists models with their names and associated credential names.

model remove

• Description: Remove a specified model configuration.
• Usage: cwc model remove "modelname"
• Flow:
  1. User specifies the "modelname" to remove.
  2. System deletes the model configuration after user confirmation.

model use

• Description: Set a model as the active model to be used in ongoing sessions.
• Usage: cwc model use "modelname"
• Flow:
  1. User specifies the "modelname" they wish to activate.
  2. System updates the active model state to reflect the user’s choice.

---

Implementation Overview

1. Adjust Configuration Schema: Update the Config struct and related configuration management code to remove ModelDeployment and introduce a structure to associate credentials with model names, as well as manage the active model.

2. Command Updates:

   • Delete the createLoginCmd and createLogoutCmd.
   • Add new credential and model commands with their respective subcommands.

3. State Management: Implement state management to track and persist the currently active model:

   • Store active model information in the configuration or a dedicated state file.
   • Update related application logic to read the active model information as needed.

4. UI and Messaging: Update UI messages and prompts to align with the new flow for adding, listing, and removing both credentials and models, as well as when activating a model for use.

5. Security: Ensure sensitive data such as API keys continues to be securely stored and handled appropriately.

Note: This may be an opportunity to introduce the https://github.com/adrg/xdg package and remove the custom xdg code. This xdg package supports both ConfigHome and DataHome among others, making it easy to place state files and config in appropriate directories.

Migration Strategy

As part of the enhancement to the cwc tool, existing users with configurations created by the cwc login command will undergo a migration process to adapt to the new credential and model management system. This migration will ensure a smooth transition from the single credential model to one that supports multiple credentials and models. The strategy encompasses the following steps:

Automatic Conversion to New Structure

• Upon the first run of the updated cwc tool, the system will check for the presence of an existing configuration file.
• If an existing configuration is detected, the migration process will automatically extract the apiKey, endpoint, and modelDeployment values.
• A new 'default' credential will be created utilizing the extracted apiKey and endpoint. This will be the initial credential entry in the new system.
• Concurrently, a new model configuration will be established using the modelDeployment value from the existing configuration. This model will be associated with the newly created 'default' credential.
• The original Config struct will be deprecated and the cwc.yaml configuration file removed from the filesystem.

Notification and Verification

• Ideally the users will receive a notification informing them about the migration and providing a brief overview of the changes.

Finalization and Clean-up

• Once the migration is completed and verified, the user will be encouraged to utilize the new commands (credential and model) to manage their configurations henceforth.
• A review of the final configuration state will be conducted, and any deprecated fields or structures will be cleared from the system.
• Users will complete the migration with a more robust and flexible system that enables them to work efficiently with multiple models and credentials, including those from other OpenAI-compliant providers.

Documentation

Documentation in the README will be updated to guide users through the new command structure proposed in this feature request.