Awesome-Attention-Heads

Welcome to Awesome-Attention-Heads! This is the platform to get the latest research on Attention Heads. We hope to provide complete and clear cutting-edge informations for researchers studying LLM interpretability and LLM hallucination.

Background

With the development of large language models, their underlying network structure, the Transformer, is being extensively studied. Researching the Transformer structure helps us enhance our understanding of this "black box" and improve model interpretability. Recently, there has been an increasing body of work suggesting that the model contains two distinct partitions: attention mechanisms used for behavior, inference, and analysis, and feed-forward networks (FFN) for knowledge storage. The former is crucial for revealing the functional capabilities of the model, leading to a series of studies exploring various functions within attention mechanisms, which we have termed Attention Head Mining.

Table of Contents

• Cite this repo
• Latest Papers
• Star Trends

Cite this repo

@misc{AwesomeAttnHead_24_github_IAAR,
  author = {Song, Shichao and Zheng, Zifan and Wang, Yezhaohui and others},
  title = {Awesome-Attention-Heads},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/IAAR-Shanghai/Awesome-Attention-Heads}}
}

Lastest Papers

Papers below are ordered by publication date:

Date	Paper	Tags	Links & Summary
2024-07-22	RazorAttention: Efficient KV Cache Compression Through Retrieval Heads
2024-07-21	Answer, Assemble, Ace: Understanding How Transformers Answer Multiple Choice Questions
2024-07-09	Induction Heads as an Essential Mechanism for Pattern Matching in In-context Learning
2024-07-01	Steering Large Language Models for Cross-lingual Information Retrieval
2024-06-21	MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression
2024-06-19	On the Difficulty of Faithful Chain-of-Thought Reasoning in Large Language Models
2024-06-16	Induction Heads as a Primary Mechanism for Pattern Matching in In-context Learning
2024-06-04	Iteration Head: A Mechanistic Study of Chain-of-Thought
2024-05-28	Knowledge Circuits in Pretrained Transformers
2024-05-23	Linking In-context Learning in Transformers to Human Episodic Memory
2024-05-02	What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation
2024-04-24	Retrieval Head Mechanistically Explains Long-Context Factuality
2024-03-27	Non-Linear Inference Time Intervention: Improving LLM Truthfulness
2024-02-28	Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models
2024-02-27	Information Flow Routes: Automatically Interpreting Language Models at Scale
2024-02-20	Identifying Semantic Induction Heads to Understand In-Context Learning
2024-02-16	The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains
2024-02-11	Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMs
2024-02-05	How do Large Language Models Learn In-Context? Query and Key Matrices of In-Context Heads are Two Towers for Metric Learning
2024-01-16	Circuit Component Reuse Across Tasks in Transformer Language Models
2024-01-16	Successor Heads: Recurring, Interpretable Attention Heads In The Wild
2024-01-16	Function Vectors in Large Language Models
2023-10-23	Linear Representations of Sentiment in Large Language Models
2023-10-06	Copy Suppression: Comprehensively Understanding an Attention Head
2023-09-22	Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
2023-09-22	Birth of a Transformer: A Memory Viewpoint
2023-07-18	Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla
2023-02-02	Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
2022-03-08	In-context Learning and Induction Heads
2021-12-22	A Mathematical Framework for Transformer Circuits
2021-05-18	The Heads Hypothesis: A Unifying Statistical Approach Towards Understanding Multi-Headed Attention in BERT
2021-04-01	Have Attention Heads in BERT Learned Constituency Grammar? This paper investigates whether the attention heads in BERT and RoBERTa language models have learned constituency grammar. The researchers use an unsupervised method to extract constituency parsing trees from the attention weights of these models.
2019-11-27	Do Attention Heads in BERT Track Syntactic Dependencies? The researchers investigate if the attention heads in pre-trained transformer language models like BERT and RoBERTa can capture syntactic dependency relations between words.
2019-11-01	Adaptively Sparse Transformers
2019-08-01	What does BERT look at? An Analysis of BERT’s Attention
2019-05-22	Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned
2016-03-21	Incorporating Copying Mechanism in Sequence-to-Sequence Learning

Star Trends