-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create explainable user model classes #223
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
"""Abstract class for creating explainable user models.""" | ||
|
||
from abc import ABC, abstractmethod | ||
from typing import Dict | ||
|
||
from dialoguekit.core import AnnotatedUtterance | ||
|
||
UserPreferences = Dict[str, Dict[str, float]] | ||
|
||
|
||
class ExplainableUserModel(ABC): | ||
@abstractmethod | ||
def generate_explanation( | ||
self, user_preferences: UserPreferences | ||
) -> AnnotatedUtterance: | ||
"""Generate an explanation based on the provided input data. | ||
|
||
Args: | ||
input_data: The input data for which an explanation is to be | ||
generated. | ||
|
||
Returns: | ||
The generated explanation. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggestion: Maybe this could be more precise, like utterance containing explanation. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. +1 I was wondering why the return type was AnnotatedUtterance, so it would be worth mentioning here that the explanation is to be returned as a system utterance. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||
|
||
Raises: | ||
NotImplementedError: This method must be implemented by a subclass. | ||
""" | ||
raise NotImplementedError( | ||
"This method must be implemented by a subclass." | ||
) |
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,81 @@ | ||||||
"""This module contains the ExplainableUserModelTagBased class. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Most modules contain a single class, so this docstring is not that informative. Suggestion instead: "Tag-based user model explanation." There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||||||
|
||||||
The class generates explanations for user preferences in the movie domain based | ||||||
on templates loaded from a YAML file. | ||||||
""" | ||||||
|
||||||
import random | ||||||
import re | ||||||
|
||||||
import yaml | ||||||
from dialoguekit.core import AnnotatedUtterance | ||||||
from dialoguekit.participant import DialogueParticipant | ||||||
|
||||||
from moviebot.explainability.explainable_user_model import ( | ||||||
ExplainableUserModel, | ||||||
UserPreferences, | ||||||
) | ||||||
|
||||||
_DEFAULT_TEMPLATE_FILE = "moviebot/explainability/explanation_templates.yaml" | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would argue to put this file in the folder There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||||||
|
||||||
|
||||||
class ExplainableUserModelTagBased(ExplainableUserModel): | ||||||
def __init__(self, template_file: str = _DEFAULT_TEMPLATE_FILE): | ||||||
"""Initialize the ExplainableUserModelTagBased class. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: conjugate verb in docstring.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||||||
|
||||||
Args: | ||||||
template_file: Path to the YAML file containing explanation | ||||||
templates. Defaults to _DEFAULT_TEMPLATE_FILE. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: second line should be indented. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||||||
""" | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggestion: check if file exists raise exception otherwise. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
||||||
with open(template_file, "r") as f: | ||||||
self.templates = yaml.safe_load(f) | ||||||
|
||||||
def generate_explanation( | ||||||
self, user_preferences: UserPreferences | ||||||
) -> AnnotatedUtterance: | ||||||
"""Generate an explanation based on the provided user preferences. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: conjugate verb in docstring.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||||||
|
||||||
Args: | ||||||
user_preferences: Nested dictionary of user preferences. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: simply user preferences? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||||||
|
||||||
Returns: | ||||||
The generated explanation. | ||||||
""" | ||||||
explanation = "" | ||||||
for category, prefs in user_preferences.items(): | ||||||
positive_tags = [tag for tag, value in prefs.items() if value == 1] | ||||||
negative_tags = [tag for tag, value in prefs.items() if value == -1] | ||||||
|
||||||
for i, tags in enumerate([positive_tags, negative_tags]): | ||||||
if len(tags) == 0: | ||||||
continue | ||||||
|
||||||
concatenated_tags = ", ".join(tags) | ||||||
template = random.choice(self.templates[category]).format( | ||||||
concatenated_tags | ||||||
) | ||||||
|
||||||
explanation += self._clean_negative_keyword( | ||||||
template, remove=i == 0 | ||||||
) | ||||||
|
||||||
return AnnotatedUtterance(explanation, DialogueParticipant.AGENT) | ||||||
|
||||||
def _clean_negative_keyword( | ||||||
self, template: str, remove: bool = True | ||||||
) -> str: | ||||||
"""Removes or keeps negation in template. | ||||||
|
||||||
Args: | ||||||
template: Template containing negative keyword. | ||||||
remove: If True, remove the negative keyword. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: add defaults value.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||||||
|
||||||
Returns: | ||||||
Template with negative keyword removed or replaced. | ||||||
""" | ||||||
if remove: | ||||||
return re.sub(r"\[.*?\]", "", template) | ||||||
|
||||||
chars_to_remove = "[]" | ||||||
trans = str.maketrans("", "", chars_to_remove) | ||||||
return template.translate(trans) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
genres: | ||
- "You [don't ]like {}." | ||
- "You [don't ]watch {} movies." | ||
- "You [don't ]enjoy {} films." | ||
|
||
actors: | ||
- "You [don't ]like movies with {}." | ||
- "You [don't ]watch films if {} are in them." | ||
- "You [don't ]care for {} in movies." | ||
|
||
directors: | ||
- "You [don't ]like {}' movies." | ||
- "You [don't ]watch films by {}." | ||
- "You [don't ]enjoy movies made by {}." | ||
|
||
keywords: | ||
- "You [don't ]like movies about {}." | ||
- "You [don't ]watch films with {}." | ||
- "You [don't ]care for {} in movies." | ||
|
||
year: | ||
- "You [don't ]like movies from {}." | ||
- "You [don't ]watch films made in {}." | ||
- "You [don't ]enjoy movies from {}." |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
import pytest | ||
|
||
from moviebot.explainability.explainable_user_model_tag_based import ( | ||
ExplainableUserModelTagBased, | ||
) | ||
|
||
|
||
@pytest.fixture | ||
def explainable_model(): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: missing return type. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||
return ExplainableUserModelTagBased( | ||
"moviebot/explainability/explanation_templates.yaml" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am wondering if it you be best to make There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We dont even need to make it public. It defaults to the default path :) |
||
) | ||
|
||
|
||
def test_generate_explanation_positive(explainable_model): | ||
user_prefs = {"genres": {"action": 1, "comedy": 1}} | ||
explanation = explainable_model.generate_explanation(user_prefs) | ||
assert "action" in explanation.text | ||
assert "comedy" in explanation.text | ||
|
||
|
||
def test_generate_explanation_negative(explainable_model): | ||
user_prefs = {"actors": {"Tom Hanks": -1, "Adam": -1}} | ||
explanation = explainable_model.generate_explanation(user_prefs) | ||
assert "Tom Hanks" in explanation.text | ||
assert "Adam" in explanation.text | ||
|
||
|
||
def test_generate_explanation_mixed(explainable_model): | ||
user_prefs = {"keywords": {"war theme": 1, "comic": -1}} | ||
explanation = explainable_model.generate_explanation(user_prefs) | ||
assert "war theme" in explanation.text | ||
assert "comic" in explanation.text | ||
|
||
|
||
def test_clean_negative_keyword_remove(explainable_model): | ||
template = "You [don't ]like action." | ||
cleaned = explainable_model._clean_negative_keyword(template) | ||
assert cleaned == "You like action." | ||
|
||
|
||
def test_clean_negative_keyword_keep(explainable_model): | ||
template = "You [don't ]like action." | ||
cleaned = explainable_model._clean_negative_keyword(template, remove=False) | ||
assert cleaned == "You don't like action." |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done