This repository is used in our paper:
JointCL: A Joint Contrastive Learning Framework for Zero-Shot Stance Detection
Bin Liang, Qinlin Zhu, Xiang Li, Min Yang, Lin Gui, Yulan He, Ruifeng Xu*. Proceedings of ACL 2022
Please cite our paper and kindly give a star for this repository if you use this code.
- Python 3.6
- PyTorch 1.6.0
- faiss-gpu 1.7.1
- transformers 2.5.1
- Install faiss package.
-
Train with command, optional arguments could be found in run_semeval.py & run_vast.py & run_wtwt.py
-
Run Semeval dataset:
python ./run_semeval.py -
Run VAST dataset:
python ./run_vast.py -
Run WTWT dataset:
python ./run_wtwt.py
- Due to the small number of dataset samples (especially SEM16), the performance gap between different seeds will vary greatly, please tune the parameter of --seed for better performance.
- We have provided checkpoints that are superior or equal to the performance reported in the paper.
- Please Run python files in run_checkpoints, you can use the trained model for prediction, and the model can be downloaded from Google drives.
- We also use 5 random seeds to run the code directly without any other tuning parameters. The performance is as follows:
| Task | Dataset | Target | Reported | Checkpoint | seed1 | seed2 | seed3 | seed4 | seed5 | Mean | Max | Gap |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Zero-shot | VAST | - | 72.3 | 72.4 | 70.6 | 71.3 | 72.4 | 72.0 | 71.3 | 71.5 | 72.4 | +0.1 |
| SEM16 | DT | 50.5 | 50.9 | 46.0 | 40.6 | 45.6 | 48.4 | 50.2 | 46.2 | 50.2 | -0.3 | |
| HC | 54.8 | 56.4 | 50.7 | 56.4 | 45.7 | 55.9 | 51.3 | 52.0 | 56.4 | +1.6 | ||
| FM | 53.8 | 54.2 | 50.9 | 51.1 | 49.1 | 49.8 | 49.4 | 50.1 | 51.1 | -2.7 | ||
| LA | 49.5 | 55.5 | 54.8 | 54.3 | 55.5 | 51.3 | 47.1 | 52.6 | 55.5 | +6 | ||
| A | 54.5 | 54.6 | 55.1 | 48.0 | 60.0 | 55.4 | 55.2 | 54.7 | 60.0 | +5.5 | ||
| CC | 39.7 | 40.7 | 31.9 | 36.9 | 39.7 | 40.2 | 28.3 | 35.4 | 40.2 | +0.5 | ||
| WTWT | CA | 72.4 | 73.6 | 72.5 | 71.4 | 73.3 | 73.4 | 74.9 | 73.1 | 74.9 | +2.5 | |
| CE | 70.2 | 70.9 | 70.1 | 71.4 | 70.4 | 70.3 | 70.3 | 70.1 | 71.4 | +1.2 | ||
| AC | 76.0 | 76.5 | 75.0 | 74.3 | 77.3 | 73.3 | 75.6 | 75.1 | 77.3 | +1.3 | ||
| AH | 75.2 | 76.5 | 76.2 | 76.1 | 76.0 | 77.9 | 78.0 | 76.8 | 78.0 | +2.8 | ||
| Few-shot | VAST | - | 71.5 | 71.6 | 71.6 | 71.9 | 68.4 | 66.1 | 69.5 | 69.5 | 71.9 | +0.4 |
| Cross-target | SEM16 | HC->DT | 52.8 | 54.6 | 42.9 | 46.9 | 48.1 | 53.7 | 54.2 | 49.2 | 54.2 | +1.4 |
| DT->HC | 54.3 | 55.4 | 52.1 | 55.8 | 54.6 | 47.8 | 38.6 | 49.8 | 55.8 | +1.5 | ||
| FM->LA | 58.8 | 60.0 | 49.8 | 58.0 | 58.3 | 46.7 | 45.7 | 51.7 | 60.0 | -0.5 | ||
| LA->FM | 54.5 | 54.8 | 45.8 | 41.8 | 54.1 | 36.2 | 47.9 | 45.2 | 54.1 | -0.4 |
- The code of this repository partly relies on ASGCN & ABSA-PyTorch & PCL.
- Here, I would like to express my gratitude to the authors of the ASGCN & ABSA-PyTorch & PCL repositories.