This repository contains the official code for ACL 2024 paper "Representation Learning with Conditional Information Flow Maximization".
This paper proposes an information-theoretic representation learning framework, named conditional information flow maximization, to extract noise-invariant sufficient representations for the input data and target task. It promotes the learned representations have good feature uniformity and sufficient predictive ability, which can enhance the generalization of pre-trained language models (PLMs) for the target task.
Experiments on 13 language understanding benchmarks demonstrate that our method effectively improves the performance of PLMs for classification and regression. Extensive experiments show that the learned representations are more sufficient, robust and transferable.
- The code will be released soon.
- [Jun 2024]: Paper is available on arXiv.
- [May 2024]: Paper is accepted by ACL 2024 (main conference).