Skip to content

Commit

Permalink
adding context recall
Browse files Browse the repository at this point in the history
Signed-off-by: aasavari <aasavari.dhananjay.kakne@intel.com>
  • Loading branch information
adkakne committed Oct 25, 2024
1 parent 05dd8d9 commit c6ce75d
Showing 1 changed file with 13 additions and 0 deletions.
13 changes: 13 additions & 0 deletions evals/metrics/ragaaf/prompt_templates/context_recall.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0


class ContextRecall:
name = "context_recall"
required_columns = ["context", "ground_truth"]
template = """- Context Recall: context recall measures how well ground truth is supported by the context.
- Score 1: None of facts/steps/aspects in the ground truth are supported by the context.
- Score 2: Only a small part of facts/steps/aspects in the ground truth are supported by the context.
- Score 3: About half of facts/steps/aspects in the ground truth are supported by the context.
- Score 4: All main/important facts/steps/aspects in the ground truth are supported by the context but some of the details are not supported by the context.
- Score 5: All of facts/steps/aspects in the ground truth are supported by the context. """

0 comments on commit c6ce75d

Please sign in to comment.