Noisy-Correspondence Learning Summary (Updating)

A new research direction of label noise learning. Noisy correspondence learning was first formally proposed in NCR (NeurIPS 2021, Oral) by the XLearning group, which aims to eliminate the negative impact of the mismatched pairs (e.g., false positives/negatives) instead of annotation errors in several tasks.

We mark works contributed by ourselves with ⭐.

This repository now is maintained by Mouxing Yang, Yijie Lin, and Yang Qin. We hope more AI-workers join us and thank all contributors!

Tasks

Image-Text Matching/Retrieval	Vision-Language Pre-training
Re-identification	Video-Text Learning
Image Captioning	Image Contrastive Learning
Graph Matching	Visual-Audio Learning
Machine Reading Comprehension	Dense Retrieval
Multi-View Clustering

Image-Text Matching/Retrieval

2024

• [2024 ACMMM] $\text{PC}^2$: Pseudo-Classification Based Pseudo-Captioning for Noisy Correspondence Learning in Cross-Modal Retrieval
  Yue Duan, Zhangxuan Gu, Zhenzhe Ying, Lei Qi, Changhua Meng, Yinghuan Shi
  [paper] [code]

• [2024 SIGIR] UGNCL: Uncertainty-Guided Noisy Correspondence Learning for Efficient Cross-Modal Matching
  Quanxing Zha, Xin Liu, Yiu-ming Cheung, Xing Xu, Nannan Wang, Jianjia Cao
  [paper] [code]

• [2024 IJCV] ⭐Learning with Noisy Correspondence
  Zhenyu Huang, Peng Hu, Guocheng Niu, Xinyan Xiao, Jiancheng Lv, Xi Peng
  [paper]

• [2024 TOIS] Breaking Through the Noisy Correspondence: A Robust Model for Image-Text Matching
  Haitao Shi, Meng Liu, Xiaoxuan Mu, Xuemeng Song, Yupeng Hu, Liqiang Nie
  [paper]

• [2024 ICASSP] NAC: Mitigating Noisy Correspondence in Cross-Modal Matching Via Neighbor Auxiliary Corrector
  Yuqing Li, Haoming Huang, Jian Xu, Shao-Lun Huang
  [paper]

• [2024 CVPR] Robust Noisy Correspondence Learning with Equivariant Similarity Consistency
  Yuchen Yang, Likai Wang, Erkun Yang, Cheng Deng
  [paper]

• [2024 CVPR] Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning
  Zihua Zhao, Mengxi Chen, Tianjie Dai, Jiangchao Yao, Bo han, Ya Zhang, Yanfeng Wang
  [paper] [code]

• [2024 CVPR] Learning to Rematch Mismatched Pairs for Robust Cross-Modal Retrieval
  Haochen Han, Qinghua Zheng, Guang Dai, Minnan Luo, Jingdong Wang
  [paper] [code]

• [2024 CVPR] Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning
  Zihua Zhao, Mengxi Chen, Tianjie Dai, Jiangchao Yao, Bo Han, Ya Zhang, Yanfeng Wang
  [paper]

• [2024 CVPR] Robust Noisy Correspondence Learning with Equivariant Similarity Consistency
  Yuchen Yang, Erkun Yang, Likai Wang, Cheng Deng
  [paper]

• [2024 Arxiv] REPAIR: Rank Correlation and Noisy Pair Half-replacing with Memory for Noisy Correspondence
  Ruochen Zheng, Jiahao Hong, Changxin Gao, Nong Sang
  [paper]

• [2024 TIP] ⭐Cross-modal Retrieval with Noisy Correspondence via Consistency Refining and Mining
  Xinran Ma, Mouxing Yang, Yunfan Li, Peng Hu, Jiancheng Lv, Xi Peng
  [paper] [code]

• [2024 AAAI] Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation
  Zhuohang Dang, Minnan Luo, Chengyou Jia, Guang Dai, Xiaojun Chang, Jingdong Wang
  [paper]

• [2024 AAAI] Negative Pre-aware for Noisy Cross-modal Matching
  Xu Zhang, Hao Li, Mang Ye
  [paper] [code]

2023

• [2023 NeurIPS] ⭐Cross-modal Active Complementary Learning with Self-refining Correspondence
  Yang Qin and Yuan Sun and Dezhong Peng and Joey Tianyi Zhou and Xi Peng and Peng Hu
  [paper] [code]

• [2023 TPAMI] ⭐Cross-Modal Retrieval with Partially Mismatched Pairs
  Peng Hu, Zhenyu Huang, Dezhong Peng, Xu Wang, Xi Peng
  [paper] [code]

• [2023 TMM] Integrating Language Guidance Into Image-Text Matching for Correcting False Negatives
  Zheng Li, Caili Guo, IEEE, Zerun Feng, Jenq-Neng Hwang, Zhongtian Du
  [paper]

• [2023 TMM] Learning From Noisy Correspondence With Tri-Partition for Cross-Modal Matching
  Feng, Zerun and Zeng, Zhimin and Guo, Caili and Li, Zheng and Hu, Lin
  [paper]

• [2023 CVPR] BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistency
  Shuo Yang, Zhapan XU, Kai Wang, Yang You, Hongxun Yao, Tongliang Liu, Min Xu
  [paper] [code]

• [2023 CVPR] MSCN: Noisy Correspondence Learning with Meta Similarity Correction
  Han, Haochen and Miao, Kaiyao and Zheng, Qinghua and Luo, Minnan
  [paper] [code]

2022

• [2022 ACMMM] ⭐Deep Evidential Learning with Noisy Correspondence for Cross-Modal Retrieval
  Qin, Yang and Peng, Dezhong and Peng, Xi and Wang, Xu and Hu, Peng
  [paper] [code]

2021

• [2021 NeurIPS Oral] ⭐Learning with Noisy Correspondence for Cross-modal Matching
  Huang, Zhenyu and Niu, Guocheng and Liu, Xiao and Ding, Wenbiao and Xiao, Xinyan and Wu, Hua and Peng, Xi
  [paper] [code]

Vision-Language Pre-training

• [2023 AAAI] NLIP: Noise-Robust Language-Image Pre-training
  Runhui Huang, Yanxin Long, Jianhua Han, Hang Xu, Xiwen Liang, Chunjing Xu, Xiaodan Liang
  [paper]

• [2022 CVPR] Robust Cross-Modal Representation Learning with Progressive Self-Distillation Andonian, Alex and Chen, Shixing and Hamid, Raffay
  [paper]

• [2022 ICML] Blip: Bootstrapping Language-image Pre-training for Unified Vision-language Understanding and Generation
  Li, Junnan and Li, Dongxu and Xiong, Caiming and Hoi, Steven*
  [paper] [code]

• [2021 NeurIPS Spotlight] Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
  Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, Steven Chu Hong Hoi
  [paper] [code]

Re-identification

• [2024 CVPR] ⭐Noisy-Correspondence Learning for Text-to-Image Person Re-identification
  Qin, Yang and Chen, Yingke and Peng, Dezhong and Peng, Xi and Zhou, Joey Tianyi and Hu, Peng
  [paper] [code]

• [2024 IJCV] ⭐Robust Object Re-identification with Coupled Noisy Labels
  Mouxing Yang, Zhenyu Huang, Xi Peng
  [paper] [code]

• [2024 EAAI] Modality Blur and Batch Alignment Learning for Twin Noisy Labels-based Visible–infrared Person Re-identification
  Song Wu, Shihao Shan, Guoqiang Xiao, Michael S. Lew, Xinbo Gao
  [paper] [code]

• [2022 CVPR] ⭐Learning With Twin Noisy Labels for Visible-Infrared Person Re-Identification
  Mouxing Yang, Zhenyu Huang, Peng Hu, Taihao Li, Jiancheng Lv, Xi Peng
  [paper] [code]

Video-Text Learning

• [2024 ICLR Oral] ⭐Multi-granularity Correspondence Learning from Long-term Noisy Videos
  Yijie Lin, Jie Zhang, Zhenyu Huang, Jia Liu, Zujie Wen, Xi Peng
  [paper] [code]

• [2024 Arxiv] A Strong Baseline for Temporal Video-Text Alignment
  Li, Zeqian and Chen, Qirui and Han, Tengda and Zhang, Ya and Wang, Yanfeng and Xie, Weidi
  [paper]

• [2023 TMM] Robust Video-Text Retrieval Via Noisy Pair Calibration
  Zhang, Huaiwen and Yang, Yang and Qi, Fan and Qian, Shengsheng and Xu, Changsheng
  [paper]

• [2021 ICCV] Crossclr: Cross-modal Contrastive Learning for Multi-modal Video Representations
  Zolfaghari, Mohammadreza and Zhu, Yi and Gehler, Peter and Brox, Thomas
  [paper]

• [2022 CVPR Oral] Temporal Alignment Networks for Long-term Video
  Han, Tengda and Xie, Weidi and Zisserman, Andrew
  [paper] [code]

• [2021 EMNLP] Videoclip: Contrastive Pre-training for Zero-shot Video-text Understanding Xu, Hu and Ghosh, Gargi and Huang, Po-Yao and Okhonko, Dmytro and Aghajanyan, Armen and Metze, Florian and Zettlemoyer, Luke and Feichtenhofer, Christoph
  [paper] [code]

Image Captioning

• [2024 AAAI] Noise-Aware Image Captioning with Progressively Exploring Mismatched Words Zhongtian Fu, Kefei Song, Luping Zhou, Yang Yang
  [paper] [code]

• [2022 CVPR] Noise-aware Learning from Web-crawled Image-Text Data for Image Captioning
  Wooyoung Kang, Jonghwan Mun, Sungjun Lee, Byungseok Roh
  [paper] [code]

Image Contrastive Learning

• [2022 CVPR] Robust contrastive learning against noisy views
  Ching-Yao Chuang, R Devon Hjelm, Xin Wang, Vibhav Vineet, Neel Joshi, Antonio Torralba, Stefanie Jegelka, Yale Song
  [paper] [code]

Graph Matching

• [2024 TIP] ⭐Cross-modal Retrieval with Noisy Correspondence via Consistency Refining and Mining
  Xinran Ma, Mouxing Yang, Yunfan Li, Peng Hu, Jiancheng Lv, Xi Peng
  [paper] [code]
• [2023 ICCV] ⭐Graph Matching with Noisy Correspondence
  Lin, Yijie and Yang, Mouxing and Yu, Jun and Hu, Peng and Zhang, Changqing and Peng, Xi
  [paper] [code]

Visual-Audio Learning

• [2024 TMM] Noise-Tolerant Learning for Audio-Visual Action Recognition
  Haochen Han, Qinghua Zheng, Minnan Luo, Kaiyao Miao, Feng Tian and Yan Chen
  [paper]

Machine Reading Comprehension

• [2023 AAAI] ⭐Robust domain adaptation for machine reading comprehension
  Jiang, Liang and Huang, Zhenyu and Liu, Jia and Wen, Zujie and Peng, Xi
  [paper]

Dense Retrieval

• [2023 EMNLP Findings] Noisy Pair Corrector for Dense Retrieval
  Hang Zhang, Yeyun Gong, Xingwei He, Dayiheng Liu, Daya Guo, Jiancheng Lv, Jian Guo
  [paper]

Multi-View Clustering

• [2024 TKDE] ⭐Robust Multi-View Clustering with Noisy Correspondence
  Yuan Sun, Yang Qin, Yongxiang Li, Dezhong Peng, Xi Peng, Peng Hu

• [2024 AAAI] ⭐Decoupled Contrastive Multi-view Clustering with High-order Random Walks
  Yiding Lu, Yijie Lin, Mouxing Yang, Dezhong Peng, Peng Hu, Xi Peng
  [paper] [code]

• [2022 TPAMI] ⭐Robust Multi-View Clustering With Incomplete Information
  Mouxing Yang, Yunfan Li, Peng Hu, Jinfeng Bai, Jiancheng Lv, Xi Peng
  [paper] [code]

• [2021 CVPR] ⭐Partially View-aligned Representation Learning with Noise-robust Contrastive Loss
  Mouxing Yang, Yunfan Li, Zhenyu Huang, Zitao Liu, Peng Hu, Xi Peng
  [paper] [code]

• [2020 NeurIPS Oral] ⭐Partially View-aligned Clustering
  Zhenyu Huang, Peng Hu, Joey Tianyi Zhou, Jiancheng Lv, and Xi Peng
  [paper] [code]