A framework for few-shot evaluation of language models.
-
Updated
Dec 20, 2024 - Python
#
evaluation-framework
Here are 152 public repositories matching this topic...
Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.
testing ci evaluation ci-cd pentesting cicd vulnerability-scanners prompts evaluation-framework red-teaming rag llm prompt-engineering llmops prompt-testing llm-eval llm-evaluation llm-evaluation-framework
-
Updated
Dec 20, 2024 - TypeScript
🐢 Open-Source Evaluation & Testing for AI & LLM systems
ai-safety evaluation-framework ethical-artificial-intelligence ai-security mlops fairness-ai responsible-ai ml-safety ml-validation red-team-tools trustworthy-ai ml-testing llm ai-red-team ai-testing llmops llm-security llm-eval llm-evaluation rag-evaluation
-
Updated
Dec 19, 2024 - Python
The LLM Evaluation Framework
evaluation-metrics evaluation-framework llm-evaluation llm-evaluation-framework llm-evaluation-metrics
-
Updated
Dec 20, 2024 - Python
This is the repository of our article published in RecSys 2019 "Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches" and of several follow-up studies.
deep-learning neural-network reproducible-research collaborative-filtering matrix-factorization hyperparameters bpr recommendation-system recommender-system reproducibility recommendation-algorithms knn matrix-completion evaluation-framework content-based-recommendation hybrid-recommender-system funksvd bprmf bprslim slimelasticnet
-
Updated
May 25, 2023 - Python
Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
-
Updated
Dec 19, 2024 - Python
Data-Driven Evaluation for LLM-Powered Applications
information-retrieval evaluation-metrics evaluation-framework rag llmops retrieval-augmented-generation llm-evaluation
-
Updated
Dec 19, 2024 - Python
Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications.
evaluation-metrics evaluation-framework rag large-language-models llm llms llmops retrieval-augmented-generation
-
Updated
Nov 14, 2024 - Python
Python SDK for running evaluations on LLM generated responses
evaluation evaluation-metrics evaluation-framework llmops llm-eval llm-ops llm-evaluation llm-evaluation-toolkit
-
Updated
Dec 20, 2024 - Python
The official evaluation suite and dynamic data release for MixEval.
benchmark evaluation benchmarking-suite evaluation-framework benchmarking-framework foundation-models large-language-models large-language-model llm-inference llm-evaluation large-multimodal-models llm-evaluation-framework benchmark-mixture mixeval
-
Updated
Nov 10, 2024 - Python
A research library for automating experiments on Deep Graph Networks
-
Updated
Sep 9, 2024 - Python
AI Data Management & Evaluation Platform
-
Updated
Oct 5, 2023 - Svelte
Moonshot - A simple and modular tool to evaluate and red-team any LLM application.
-
Updated
Dec 20, 2024 - Python
PySODEvalToolkit: A Python-based Evaluation Toolbox for Salient Object Detection and Camouflaged Object Detection
latex metrics evaluation python3 evaluator evaluation-metrics evaluation-framework metrics-visualization mae saliency salient-object-detection saliency-detection co-saliency f-measure pr-curve e-measure s-measure camouflaged-object-detection fm-curve co-salient-object-detection
-
Updated
Sep 27, 2024 - Python
Expressive is a cross-platform expression parsing and evaluation framework. The cross-platform nature is achieved through compiling for .NET Standard so it will run on practically any platform.
xamarin parsing cross-platform evaluation netstandard expression-parser expression-evaluator hacktoberfest evaluation-framework
-
Updated
Oct 1, 2024 - C#
AgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.
-
Updated
Dec 19, 2024 - Python
Test and evaluate LLMs and model configurations, across all the scenarios that matter for your application
-
Updated
Aug 20, 2024 - TypeScript
DevQualityEval: An evaluation benchmark 📈 and framework to compare and evolve the quality of code generation of LLMs.
-
Updated
Dec 19, 2024 - Go
Evaluation suite for large-scale language models.
-
Updated
Aug 15, 2021 - Python
Multilingual Large Language Models Evaluation Benchmark
multilingual nlp natural-language-processing evaluation evaluation-datasets datasets language-model evaluation-framework large-language-models
-
Updated
Aug 21, 2024 - Python
Improve this page
Add a description, image, and links to the evaluation-framework topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the evaluation-framework topic, visit your repo's landing page and select "manage topics."