Automatic Harmful Memes Detection Resources

This repository contains resources (conference/journal publications, references to datasets) associated the work "Detecting and Understanding Harmful Memes: A Survey" accepted at IJCAI-ECAI (Survey), 2022.

The automatic identification of harmful content online is of major concern for social media platforms, policymakers, and society. Researchers have studied textual, visual, and audio content, but typically in isolation. Yet, harmful content often combines multiple modalities, as in the case of memes, which are of particular interest due to their viral nature. With this in mind, here we offer a comprehensive survey with a focus on harmful memes. Based on a systematic analysis of recent literature, we first propose a new typology of harmful memes, and then we highlight and summarize the relevant state of the art. One interesting finding is that many types of harmful memes are not really studied, e.g., such featuring self-harm and extremism, partly due to the lack of suitable datasets. We further find that existing datasets mostly capture multi-class scenarios, which are not inclusive of the affective spectrum that memes can represent. Another observation is that memes can propagate globally through repackaging in different languages and that they can also be multilingual, blending different cultures. We conclude by highlighting several challenges related to multimodal semiotics, technological constraints and non-trivial social engagement, and we present several open-ended aspects such as delineating online harm and empirically examining related frameworks and assistive interventions, which we believe will motivate and drive future research.

Overview

This repo contains relevant resources Automatic Harmful Memes Detection. We list a comprehensive and up-to-date information for harmful meme detection.

Table of Contents:

• Harmful Memes Detection
• Datasets
• Relevant Studies
• Relevant Surveys
• Current SOTA
• Cite

Harmful Memes Detection

Datasets

• Harmful Memes: Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md Shad Akhtar, Preslav Nakov, Tanmoy Chakraborty MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets, 2021. [Paper] [Dataset]

• Harmful Memes: Shraman Pramanik, Dimiter Dimitrov, Rituparna Mukherjee, Shivam Sharma, Md. Shad Akhtar, Preslav Nakov, Tanmoy Chakraborty, Detecting Harmful Memes and Their Targets, 2021. [Paper] [Dataset]

• Hateful Memes: Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, Davide Testuggine, The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes, 2020. [Paper] [Dataset]

• Fine-grained Propaganda Memes: Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov and Giovanni Da San Martino, Detecting Propaganda Techniques in Memes, 2021. [Paper] [Dataset]

• Racist or Sexist Meme: Haris Bin Zia, Ignacio Castro, Gareth TysonRacist or Sexist Meme? Classifying Memes beyond Hateful, 2021. [Paper] [Dataset]

• Antisemitism Detection: Mohit Chandra, Dheeraj Pailla, Himanshu Bhatia, Aadilmehdi Sanchawala, Manish Gupta, Manish Shrivastava, Ponnurangam Kumaraguru, Subverting the Jewtocracy: Online Antisemitism Detection Using Multimodal Deep Learning, 2021.

• Troll Memes: Shardul Suryawanshi, Bharathi Raja Chakravarthi, Pranav Varma, Mihael Arcan, John P. McCrae and Paul Buitelaar, A Dataset for Troll Classification of TamilMemes, 2020. [Paper] [Dataset]

• Sexist meme: Elisabetta Fersini; Francesca Gasparini; Silvia Corchs, Detecting Sexist MEME On The Web: A Study on Textual and Visual Cues, 2019. [Paper] [Dataset]

• Hateful Memes: Hannah Kirk, Yennie Jun, Paulius Rauba, Gal Wachtel, Ruining Li, Xingjian Bai, Noah Broestl, Martin Doff-Sotta, Aleksandar Shtedritski, Yuki M Asano, Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset,

• Offensive Memes: Benet Oriol Sabat, Cristian Canton Ferrer, Xavier Giro-i-Nieto, Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation, 2019. [Paper] [Dataset]

• Hateful memes (Pinterest) Hannah Kirk, Yennie Jun, Paulius Rauba, Gal Wachtel, Ruining Li, Xingjian Bai, Noah Broestl, Martin Doff-Sotta, Aleksandar Shtedritski, Yuki M Asano, Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset, [Paper] [[Dataset](https://github.com/iamatulsingh/ pinterest-image-scrap)]

• MMHS150K: Raul Gomez, Jaume Gibert, Lluis Gomez, Dimosthenis Karatzas, Exploring Hate Speech Detection in Multimodal Publications, 2019. [Paper] [Dataset]

• MultiOFF: Shardul Suryawanshi, Bharathi Raja Chakravarthi, Mihael Arcan, Paul Buitelaar, Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text [Paper] [Dataset]

Relevant Studies

Hate

• Detecting Harmful Memes and Their Targets [paper]
• MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets [paper]
• Racist or Sexist Meme? Classifying Memes beyond Hateful [paper]
• “Subverting the Jewtocracy”: Online Antisemitism Detection Using Multimodal Deep Learning [paper]
• Detecting Sexist MEME On The Web: A Study on Textual and Visual Cues [paper]
• Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset [paper]
• Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation [paper]
• Disentangling Hate in Online Memes [paper]
• Exploring Hate Speech Detection in Multimodal Publications [paper]

Offensive

• AOMD: An Analogy-aware Approach to Offensive Meme Detection on Social Media [paper]
• KnowMeme: A Knowledge-enriched Graph Neural Network Solution to Offensive Meme Detection [paper]
• An approach to detect offence in Memes using Natural Language Processing(NLP) and Deep learning [paper]
• Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text [paper]

Propaganda

• Detecting Propaganda Techniques in Memes [paper]
• MinD at SemEval-2021 Task 6: Propaganda Detection using Transfer Learning and Multimodal Fusion [paper]
• Volta at SemEval-2021 Task 6: Towards Detecting Persuasive Texts and Images using Textual and Multimodal Ensemble [paper]
• Alpha at SemEval-2021 Task 6: Transformer Based Propaganda Classification [paper]
• Fine-grained analysis of propaganda in news article [paper]

Harassment/Cyberbullying

• Detection of Cyberbullying Incidents on the Instagram Social Network [paper]
• A Dataset for Troll Classification of TamilMemes [paper]

Violence

• Violence detection in hollywood movies by the fusion of visual and mid-level audio cues [paper]

Self-Inflicted Harm

• The self—harmed, visualized, and reblogged: Remaking of self-injury narratives on tumblr [paper]

Relevant Surveys

• A unified taxonomy of harmful content [paper]
• A survey on multimodal disinformation detection [paper]
• A survey on stance detection for mis- and disinformation identification [paper]
• Detecting abusive language on online platforms: A critical analysis [paper]
• A Survey on Computational Propaganda Detection [paper]
• A multimodal memes classification: A survey and open research issues. [paper]
• A survey on automatic detection of hate speech in text [paper]
• Automatic cyberbullying detection: A systematic review[paper]
• A survey on hate speech detection using natural language processing [paper]
• A survey of video violence detection [paper]
• A survey of fake news: Fundamental theories, detection methods, and opportunities [paper]

Current SOTA

Below we provide a list of recent work with current state-of-the-art results.

Title	Types	Task	Dataset	Task Type	Approach	AUC	Acc	F1	Details
Detecting Harmful Memes and Their Targets	Harm	Harmful vs Non-harmful	HarMeme (Covid-19)	Binary	VisualBERT (Pretrained using MS COCO)		0.81	0.8
		Very harmful vs Partially-harmful vs Non-harmful		Multiclass			0.74	0.54
		Target Identification of Harmful Memes		Multiclass			0.76	0.66
MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets	Harm	Harmful vs Non-harmful	Harm-C	Binary	MOMENTA: CLIP, VGG-19, DistilBERT, Cross-modal attention fusion (CMAF)		0.84	0.83
		Very harmful vs Partially-harmful vs Non-harmful	Harm-C	Multiclass			0.77	0.55
		Target Identification of Harmful Memes	Harm-C	Multiclass			0.78	0.7
		Harmful vs Non-harmful	Harm-P	Binary			0.9	0.88
		Very harmful vs Partially-harmful vs Non-harmful	Harm-P	Multiclass			0.87	0.67
		Target Identification of Harmful Memes	Harm-P	Multiclass			0.79	0.69
Racist or Sexist Meme? Classifying Memes beyond Hateful	Hate	Protected category (PC) identification	FBHM	ML	CIMG + CTXT + LASER + LaBSE	0.96
		Detecting attack type (AT)	FBHM	Multilabel		0.97
“Subverting the Jewtocracy”: Online Antisemitism Detection Using Multimodal Deep Learning	Hate	Antisemitic content detection	GAB	Binary	MFAS		0.91
			Twitter	Binary	MFAS		0.71
		Antisemitism category classification	GAB	Multiclass	MFAS		0.67
			Twitter	Multiclass	MFAS		0.68
Detecting Sexist MEME On The Web: A Study on Textual and Visual Cues	Hate	Automatic detection of sexist memes.	The MEME	Binary	Late fusion			0.76	Multi-modal Late-fusion Hand-crafted visual dec\scriptiors used: Low-level greyscale, colored, photographic and semantic features. Bag-of-words approached used as textual features. SVM
Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset	Hate	Hateful Meme Detection	FBHM	Binary	CLIP (Linear Probe)			0.56
			Pinterest	Binary	CLIP (Linear Probe)			0.57
Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation	Hate	Hateful Meme Detection	Google	Binary	BERT, VGG-16, MLP		0.83
Disentangling Hate in Online Memes	Hate	Hateful Meme Detection	FBHM	Binary	DisMultiHate (BERT, Faster-RCNN, Disentangled representations)	0.83	0.76
			MultiOFF	Binary				0.65
Exploring Hate Speech Detection in Multimodal Publications	Hate	Hatespech detection in multimodal publications	MMHS150K	Binary	FCM (Feature concatenation model), Inception-V3, LSTM	0.73	0.68	0.70
AOMD: An Analogy-aware Approach to Offensive Meme Detection on Social Media	Offensive	Offensive (analogy) meme detection	GAB	Binary	Analogy-aware Multi-modal Representation Learning (Faster R-CNN, ResNet50, Glove-based LSTM, BERT) MLP		0.69	0.56
			Reddit	Binary			0.72	0.49
KnowMeme: A Knowledge-enriched Graph Neural Network Solution to Offensive Meme Detection	Offensive	Offensive meme detection	Reddit	Binary	YOLO V4, ConceptNET, GNN		0.73	0.49	Knowledge-aware Multimodal Entity Extraction (KMEE): YOLO V4 Knowledge-enriched Information Network Construction: ConceptNET Supervised Offensive Meme Classification: GNN
			GAB	Binary			0.7	0.55
An approach to detect offence in Memes using Natural Language Processing(NLP) and Deep learning	Offensive	Offensive meme detection	Offensive + Intensity dataset	Binary	CNN, FastText, LSTM - Sigmoid		0.96
			Offense intensity prediction	Multiclass	CNN, FastText, LSTM - Softmax		0.99
Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text	Offensive	Offensive content detection	MultiOFF	Binary	Early fusion: Stacked LSTM/ BiLSTM/CNN-Text + VGG16			0.5	Early fusion technique (Stacked LSTM/ BiLSTM/CNN-Text + VGG16)
Detecting Propaganda Techniques in Memes	Propaganda	Detecting the type of propaganda techniques used in memes	Facebook	Multilabel	VisualBERT (Pretrained using MS COCO)			0.48	micro F1
MinD at SemEval-2021 Task 6: Propaganda Detection using Transfer Learning and Multimodal Fusion	Propaganda	Propaganda technique detection (Unimodal: Text)	Facebook	Multilabel	Ensemble: BERT, RoBERTa, XLNet, ALBERT, DistilBERT, DeBERTa, Embeddings, Char n-gram			0.59	micro F1, Team name: MinD
Volta at SemEval-2021 Task 6: Towards Detecting Persuasive Texts and Images using Textual and Multimodal Ensemble	Propaganda	Propaganda technique and span detection (Unimodal: Text)	Facebook	Multilabel	RoBERTa			0.48	micro F1, Team name: Volta
Alpha at SemEval-2021 Task 6: Transformer Based Propaganda Classification	Propaganda	Propaganda technique detection (Multimodal: Meme)	Facebook	Multilabel	RoBERTa, Embeddings			0.58	micro F1, Team name: Alpha
Detection of Cyberbullying Incidents on the Instagram Social Network	Cyber-bullying	Detecting incidents of cyber-bullying	Instagram	Binary	SVD +(Unigram, 3-gram), kernelPCA+(meta data, image categories) + lin. SVM		0.87
A Dataset for Troll Classification of TamilMemes	Cyber-bullying	Detecting Troll memes	TamilMemes	Binary	ResNET (Training: TamilMemes)			0.52	macro F1
					ResNET (Training: TamilMemes + ImageNet)			0.52
					MobileNet (Training: TamilMemes + ImageNet + Flickr1k)			0.47
					ResNET (Training: TamilMemes + ImageNet + Flickr30k)			0.52

Cite

Please cite the following paper.

@inproceedings{ijcai2022Shivam,
  title     = {Detecting and Understanding Harmful Memes: A Survey},
  author    = {Shivam Sharma and  Firoj Alam and Md. Shad Akhtar  and Dimitar Dimitrov  and Giovanni Da San Martino  and Hamed Firooz and Alon Halevy and Fabrizio Silvestri and Preslav Nakov and Tanmoy Chakraborty},
  booktitle = {Proceedings of the 31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence},
  series={IJCAI-ECAI~'22},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  year      = {2022},
  note      = {Survey Track},
  address={Messe Wien, Vienna, Austria},
}