SEC: Context-Aware Metric Learning for Efficient Emotion Recognition in Conversation

This repository presents a PyTorch implementation of meta-learning models applied on the DailyDialog dataset, performing emotions predictions. It consists in a Master thesis work for the Master of Data Science at the University of Luxembourg.

This work has been accepted at two venues:

• JEP-TALN 2024 "SEC : Contexte Émotionnel Phrastique pour la Reconnaissance Émotionnelle Efficiente dans la Conversation" (in French)
• WASSA @ ACL 2024 "SEC: Context-Aware Metric Learning for Efficient Emotion Recognition in Conversation"

Citations of both papers will soon be available in the Cite section.

Table of Contents

• Data
• Models
• Usage
  • Preprocessing
  • Training and evaluation
  • Qualitative insights
• Cite

Data

DailyDialog dataset was introduced by Li et al. in 2017 and consists in more than 12,000 generated dialogues intended to be representative of daily concerns. These dialogues are labelled with the 6 Ekman's emotions. More details about DailyDialog can be found here.

Models

In this work, we perform Emotion Recognition in Conversation (ERC) on DailyDialog conversation using meta-learning approaches. More precisely, this aim is to represent utterances from each dialog in its conversational context in order to predict accurate emotions. The model architecture used in the experiments in the Siamese Networks. Another architecture called MAML (Model-Agnostic Meta-Learning) is yet to be explored.

In the provided code for Siamese Networks, we implemented to models. The first one, referred to as isolated utterance model, is bound to perform emotion recognition on isolated utterances, without any contextual information at dialog level. The second one, referred to as contextual utterance model, uses a BERT-based encoding that takes the conversational context into account, as BERT tokenization is performed at dialog level. Below are illustrated the two different training pipeline of the aforementioned models:

Isolated utterance model

Contextual utterance model

Usage

The provided code allows to run data preprocessing, training and inference for both model. In order to achieve full reproducibility, please refer to the file requirements.txt by using:

pip install -r requirements.txt

To start with, clone the repository and go inside SiameseNetworks directory:

git clone https://github.com/B-Gendron/meta_dyda.git
cd meta_dyda
cd SiameseNetworks

It is possible to reproduce the following experiments:

• static utterance model training and evaluation
• contextual utterance model training and evaluation
• qualitative insights on static utterance model predictions
• qualitative insights on contextual utterance model predictions

Preprocessing

Because the processed datasets are not available in the repository, it is necessary to perform preprocessing before training on data. Three different preprocessing pipelines are available:

• Conversation-aware dialog representations
  This experiment is the main contribution of this software. It is required to perform preprocessing on both isolated and contextual data, because isolated utterance data will be use te train the auxiliary emotion classifier, while the contextual data is used to perform the main experiment, meaning emotion recognition in conversational context.

python3 preprocessing.py -l s -t u
python3 preprocessing.py -l s -t c

• Isolated utterance representations
  In this case that accounts for a baseline, of course only isolated data preprocessing is needed.

python3 preprocessing.py -l u

Training and evaluation

Please note that, due to their sizes, the models are not provided in this repository. Therefore, one needs to train the model in order to perform inference on it. Furthermore, the qualitative experiments can be performed only if the models are already stored.

To run the experiments, one calls main.py script with some arguments:

• -m for specifying the model (isolated or contextual)
• -t to run training and inference
• -q to run qualitative analysis The experiments triggered when specifying -t and -q depends on the model value in -m. This is therefore a required argument.

Below are given all commands for each experiment so they can be directly copied and pasted. If you encounter any issue running the following code snippets, don't hesitate to display help first:

python3 main.py -h

Use the following command to train the model you want. The isolated utterance model is based on static utterance representations, therefore it does not take the conversational context into account. The selected model in this case is a 5-layer LSTM. On the other hand, the contextual utterance model takes the conversational context into account. The selected model is based on Sentence BERT. Here is the command for the isolated utterance model:

python3 main.py -m i -t

And for contextual utterance model:

python3 main.py -m s -t

Qualitative insights

The following commands will proceed some qualitative analysis on parts of DailyDialog test set. Concretely, it will generate an output .txt file that contains dialogs along with the considered utterance and prediction compared to ground truth labels. For isolated utterance model:

python3 main.py -m i -q

And for contextual utterance model:

python3 main.py -m s -q

Cite

BibTeX citations to be added

JEP-TALN 2024

@inproceedings{Gendron-Guibon:JEP-TALN:2024,
    author = {Gendron, Barbara and Guibon, Ga\"el},
    title = "SEC : contexte \'emotionnel phrastique int\'egr\'e pour la reconnaissance \'emotionnelle efficiente dans la conversation",
    booktitle = "Actes de JEP-TALN-RECITAL 2024. 31\`eme Conf\'erence sur le Traitement Automatique des Langues Naturelles,  volume 1 : articles longs et prises de position",
    month = "7",
    year = "2024",
    address = "Toulouse, France",
    publisher = "Association pour le Traitement Automatique des Langues",
    pages = "219-233",
    note = "",
    abstract = "L'essor des mod\`eles d'apprentissage profond a apport\'e une contribution significative \`a la reconnaissance des \'emotions dans les conversations (ERC). Cependant, cette t\^ache reste un d\'efi important en raison de la pluralit\'e et de la subjectivit\'e des \'emotions humaines. Les travaux ant\'erieurs sur l'ERC fournissent des mod\`eles pr\'edictifs utilisant principalement des repr\'esentations de la conversation bas\'ees sur des graphes. Dans ce travail, nous proposons une fa\c{c}on de mod\'eliser le contexte conversationnel que nous incorporons \`a une strat\'egie d'apprentissage de m\'etrique, avec un processus en deux \'etapes. Cela permet d'effectuer l'ERC dans un sc\'enario de classification flexible et d'obtenir un mod\`ele l\'eger et efficace. En utilisant l'apprentissage de m\'etrique \`a travers une architecture de r\'eseau siamois, nous obtenons un score de macroF1 de 57,71\\% pour la classification des \'emotions dans les conversations sur le jeu de donn\'ees DailyDialog, ce qui surpasse les travaux connexes. Ce r\'esultat \'etat-de-l'art est prometteur en ce qui concerne l'utilisation de l'apprentissage de m\'etrique pour la reconnaissance des \'emotions, mais est perfectible au regard du microF1 obtenu.",
    keywords = "Apprentissage profond,Reconnaissance d'\'emotions en conversation,Apprentissage de m\'etrique",
    url = "https://talnarchives.atala.org/TALN/TALN-2024/7864.pdf"
}

WASSA 2024 (ACL workshop)