Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create explainable user model classes #223

Merged
merged 4 commits into from
Oct 27, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file.
30 changes: 30 additions & 0 deletions moviebot/explainability/explainable_user_model.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
"""Abstract class for creating explainable user models."""

from abc import ABC, abstractmethod
from typing import Dict

from dialoguekit.core import AnnotatedUtterance

UserPreferences = Dict[str, Dict[str, float]]


class ExplainableUserModel(ABC):
@abstractmethod
def generate_explanation(
self, user_preferences: UserPreferences
) -> AnnotatedUtterance:
"""Generate an explanation based on the provided input data.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"""Generate an explanation based on the provided input data.
"""Generates an explanation based on the provided input data.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Args:
input_data: The input data for which an explanation is to be
generated.

Returns:
The generated explanation.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Maybe this could be more precise, like utterance containing explanation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 I was wondering why the return type was AnnotatedUtterance, so it would be worth mentioning here that the explanation is to be returned as a system utterance.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Raises:
NotImplementedError: This method must be implemented by a subclass.
"""
raise NotImplementedError(
"This method must be implemented by a subclass."
)
81 changes: 81 additions & 0 deletions moviebot/explainability/explainable_user_model_tag_based.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
"""This module contains the ExplainableUserModelTagBased class.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most modules contain a single class, so this docstring is not that informative. Suggestion instead: "Tag-based user model explanation."

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


The class generates explanations for user preferences in the movie domain based
on templates loaded from a YAML file.
"""

import random
import re

import yaml
from dialoguekit.core import AnnotatedUtterance
from dialoguekit.participant import DialogueParticipant

from moviebot.explainability.explainable_user_model import (
ExplainableUserModel,
UserPreferences,
)

_DEFAULT_TEMPLATE_FILE = "moviebot/explainability/explanation_templates.yaml"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would argue to put this file in the folder data.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done



class ExplainableUserModelTagBased(ExplainableUserModel):
def __init__(self, template_file: str = _DEFAULT_TEMPLATE_FILE):
"""Initialize the ExplainableUserModelTagBased class.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: conjugate verb in docstring.

Suggested change
"""Initialize the ExplainableUserModelTagBased class.
"""Initializes the ExplainableUserModelTagBased class.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Args:
template_file: Path to the YAML file containing explanation
templates. Defaults to _DEFAULT_TEMPLATE_FILE.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: second line should be indented.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: check if file exists raise exception otherwise.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

with open(template_file, "r") as f:
self.templates = yaml.safe_load(f)

def generate_explanation(
self, user_preferences: UserPreferences
) -> AnnotatedUtterance:
"""Generate an explanation based on the provided user preferences.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: conjugate verb in docstring.

Suggested change
"""Generate an explanation based on the provided user preferences.
"""Generates an explanation based on the provided user preferences.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Args:
user_preferences: Nested dictionary of user preferences.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: simply user preferences?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Returns:
The generated explanation.
"""
explanation = ""
for category, prefs in user_preferences.items():
positive_tags = [tag for tag, value in prefs.items() if value == 1]
negative_tags = [tag for tag, value in prefs.items() if value == -1]

for i, tags in enumerate([positive_tags, negative_tags]):
if len(tags) == 0:
continue

concatenated_tags = ", ".join(tags)
template = random.choice(self.templates[category]).format(
concatenated_tags
)

explanation += self._clean_negative_keyword(
template, remove=i == 0
)

return AnnotatedUtterance(explanation, DialogueParticipant.AGENT)

def _clean_negative_keyword(
self, template: str, remove: bool = True
) -> str:
"""Removes or keeps negation in template.

Args:
template: Template containing negative keyword.
remove: If True, remove the negative keyword.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: add defaults value.

Suggested change
remove: If True, remove the negative keyword.
remove: If True, remove the negative keyword. Defaults to True.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Returns:
Template with negative keyword removed or replaced.
"""
if remove:
return re.sub(r"\[.*?\]", "", template)

chars_to_remove = "[]"
trans = str.maketrans("", "", chars_to_remove)
return template.translate(trans)
24 changes: 24 additions & 0 deletions moviebot/explainability/explanation_templates.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
genres:
- "You [don't ]like {}."
- "You [don't ]watch {} movies."
- "You [don't ]enjoy {} films."

actors:
- "You [don't ]like movies with {}."
- "You [don't ]watch films if {} are in them."
- "You [don't ]care for {} in movies."

directors:
- "You [don't ]like {}' movies."
- "You [don't ]watch films by {}."
- "You [don't ]enjoy movies made by {}."

keywords:
- "You [don't ]like movies about {}."
- "You [don't ]watch films with {}."
- "You [don't ]care for {} in movies."

year:
- "You [don't ]like movies from {}."
- "You [don't ]watch films made in {}."
- "You [don't ]enjoy movies from {}."
45 changes: 45 additions & 0 deletions tests/explainability/test_explainable_user_model_tag_based.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import pytest

from moviebot.explainability.explainable_user_model_tag_based import (
ExplainableUserModelTagBased,
)


@pytest.fixture
def explainable_model():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: missing return type.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

return ExplainableUserModelTagBased(
"moviebot/explainability/explanation_templates.yaml"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering if it you be best to make _DEFAULT_TEMPLATE_FILE public and reuse it here?

Copy link
Collaborator Author

@IKostric IKostric Oct 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We dont even need to make it public. It defaults to the default path :)

)


def test_generate_explanation_positive(explainable_model):
user_prefs = {"genres": {"action": 1, "comedy": 1}}
explanation = explainable_model.generate_explanation(user_prefs)
assert "action" in explanation.text
assert "comedy" in explanation.text


def test_generate_explanation_negative(explainable_model):
user_prefs = {"actors": {"Tom Hanks": -1, "Adam": -1}}
explanation = explainable_model.generate_explanation(user_prefs)
assert "Tom Hanks" in explanation.text
assert "Adam" in explanation.text


def test_generate_explanation_mixed(explainable_model):
user_prefs = {"keywords": {"war theme": 1, "comic": -1}}
explanation = explainable_model.generate_explanation(user_prefs)
assert "war theme" in explanation.text
assert "comic" in explanation.text


def test_clean_negative_keyword_remove(explainable_model):
template = "You [don't ]like action."
cleaned = explainable_model._clean_negative_keyword(template)
assert cleaned == "You like action."


def test_clean_negative_keyword_keep(explainable_model):
template = "You [don't ]like action."
cleaned = explainable_model._clean_negative_keyword(template, remove=False)
assert cleaned == "You don't like action."
Loading