Skip to content

Latest commit

 

History

History
58 lines (49 loc) · 2.13 KB

readme.md

File metadata and controls

58 lines (49 loc) · 2.13 KB
Raw

Entailment as Robust Self-learner

The repo of paper Entailment as Robust Self-learner by Jiaxin Ge*, Hongyin Luo*, Yoon Kim, Jim Glass at ACL 2023 main conference.

Dependencies

  • Transformers >= 4.16.2
  • Pytorch >= 1.6.0

Preparing data

bash prep.sh

Reproducing GLUE Experiments

bash prompt_script.sh

Parameters in prompt_script.sh

  • domain: qnli, qqp, rte, sst2
  • ft_mode: st (self-training)
  • exp_id: an int number flaging the experiment ID.
  • train_size: The number of a unit training set. The number of training case is train_size x exp_id.
  • model_type_str: Either roberta or deberta

The prompt_script.sh file describes the entire process, and it runs multiple independent experiments. We also added Slurm flags so it can be submitted to slurm as a single job or a job array.

Reproducing Multi-class Experiments

cd multi-class

Train:
python3 ag_news.py --algo MODEL_NAME --index 0 --type ST_METHOD
python3 amazon_news.py --algo MODEL_NAME --index 0 --type ST_METHOD
python3 emotion.py --algo MODEL_NAME --index 0 --type ST_METHOD
python3 copa.py --algo MODEL_NAME --index 0 --type ST_METHOD

Test:
python3 test_agnews.py --algo MODEL_NAME --index 0
python3 test_amazon.py --algo MODEL_NAME --index 0
python3 test_copa.py --algo MODEL_NAME --index 0
python3 test_emotion.py --algo MODEL_NAME --index 0

—algo: which entailment model backbone to use [“deberta”, “roberta”]
—index : appendix in the model path 
—type: finetune algorithm [
    "pseudo",       # Baseline self training
    "confidence",   # Removing low-confidence cases
    'dropout'       # Simple dropout-based voting
    'unconf',       # Full SimPLE algorithm
]

Files included

  • pretrain_enty.py: Entailment pretraining on MNLI
  • proc_data.py: Preprocessing the training / eval corpora and constructing the suppositions.
  • prompt_cst.py: Running one training pipeline.
  • train_glue.py: Contains the training functions.
  • eval_prompt.py: Evaluate the pretrained, fine-tuned, and self-trained models.
  • prompt_script.sh: Running three independent training-evaluation pipelines.