0.12.0

What's Changed

CLI

• add cli auto-prompt analyze - prompt analyzer in cli by @wenzhe-log10 in #288
  • Please check the CLI doc for detailed usage for input prompt/messages.
• cli benchmark_models - support OpenAI fine-tuned models by @wenzhe-log10 in #287

OpenAI Compatibility

• Generalize perplexity logic for streaming end detection across OpenAI compatible models by @kxtran in #286
  • Tested with Cerebras, Mistral, and Perplexity.

Maintenance

• update dependencies - OpenAI, Anthropic, and Magentic by @wenzhe-log10 in #289

Full Changelog: 0.11.0...0.12.0

0.11.0

What's Changed

New

• Add async streaming support for OpenAI compatible models (e.g Mistral and Perplexity) by @kxtran in #281

Full Changelog: 0.10.11...0.11.0

0.10.11

What's Changed

CLI

• fix - benchmark for responses does not have usage field by @wenzhe-log10 in #275

Full Changelog: 0.10.10...0.10.11

0.10.10

What's Changed

Fix

• get full completion in cli completion download by @wenzhe-log10 in #272

Full Changelog: 0.10.9...0.10.10

0.10.9

What's Changed

CLI updates

• add new models to benchmark_models by @wenzhe-log10 in #265
• add log10 tags of the completions into the output by @wenzhe-log10 in #267

Full Changelog: 0.10.8...0.10.9

0.10.8

What's Changed

CLI

• update feedback download cli with filter option by @wenzhe-log10 in #251
• update doc: log10 feedback download --help by @wenzhe-log10 in #262

Update dependencies

• Dependency update by @wenzhe-log10 in #263
  • openai -> 1.40.6
  • anthropic -> 0.33.1
  • magentic -> 0.31.0

Full Changelog: 0.10.7...0.10.8

0.10.7

What's Changed

Log19 CLI

• allow benchmark_models --id to take a json file of completion ids by @wenzhe-log10 in #250
  Example:

log10 completions benchmark_models --ids list_comp_ids.json --models=gpt-4o-mini -f benchmark_summ_test_gpt4o_mini.jsonl

JSON file examples:

[
    "545ab06c-07b0-4720-bce8-68df8201dbcc",
    "c01e663f-cd8a-4cd8-8aa8-6b6ef8966b4d",
    "c450eb50-5e0f-4bcc-9faf-3a1d4183f41a"
]

Full Changelog: 0.10.6...0.10.7

0.10.6

What's Changed

Maintenance

• update console message with status update instead of console print by @wenzhe-log10 in #242
• Update cli_docs.md pip install 'log10-io[cli]' by @wenzhe-log10 in #243
• Bump anthropic to 0.32.0 by @wenzhe-log10 in #244

Full Changelog: 0.10.5...0.10.6

0.10.5

What's Changed

Bug fixes

• Fix listing and download completions cli bug by @kxtran in #240

Chores

• Bump openai to 1.37.1 by @wenzhe-log10 in #236

Full Changelog: 0.10.4...0.10.5

0.10.4

What's Changed

CLI Features

• Implement feedback filtering in CLI & SDK by @nullfox @wenzhe-log10 in #206
• update cli doc feedback filter by @wenzhe-log10 in #237
• add --save_all_results_to_dataframe to benchmark_models cli by @wenzhe-log10 in #233
• update benchmark_models cli using --file to save to CSV, JSONL, or Markdown files by @wenzhe-log10 in #238

Full Changelog: 0.10.3...0.10.4