Skip to content

'Semi-Supervised Teacher-Student Architecture for Relation Extraction' adapt Mean Teacher, a denoising semi-supervised framework, to improve the performance of relation extraction by incorporating both labeled and unlabeled data in the training process.

Notifications You must be signed in to change notification settings

Fan-Luo/MT-RelationExtraction

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 

Repository files navigation

By Fan Luo, Ajay Nagesh, Rebecca Sharp, and Mihai Surdeanu (Clulab)

Poster

In this work, we adapt Mean Teacher (Tarvainen and Valpola, 2017), a denoising SSL framework to extract semantic relations between pairs of entities. We explore the sweet spot of amount of supervision required for good performance on this binary relation extraction task.

Within the Mean Teacher framework, we investigate input representations of four types of syntactic abstraction. For example (Robert Kingsley, perGraduatedInstitution, Uni- versity of Minnesota):

Mean Teacher model

We evaluate our approach on the Google-IISc Distant Supervision (GDS) dataset.(Jat et al., 2017). Our results show that the SSL Mean Teacher approach nears the performance of fully-supervised approaches even with only 10% of the labeled corpus. Further, the syntax-aware model out- performs other syntax-free approaches across all levels of supervision.

Usage

TRAIN:

python main.py --dataset=gids --labels=100.0 --arch=lstm_RE --train-subdir=train_fullyLex --eval-subdir=dev_fullyLex --epochs=200 --labeled-batch-size=64 --random-seed=366 --consistency=1 --consistency-rampup=5 --word-frequency=5 --print-freq=2 --pretrained-wordemb=True --wordemb-size=100 --random-initial-unkown=True --update-pretrained-wordemb=True --run-name log_gids_labels100.0_epochs200_labeled64_cons1_ramp5_pre_update_rand366_wf5_fullyLex > logs/log_gids_labels100.0_epochs200_labeled64_cons1_ramp5_pre_update_rand366_wf5_fully Lex.txt 2>&1

TEST:

python eval_saved_model.py --dataset=gids --labels=100.0 --arch=lstm_RE --train-subdir=train_fullyLex --eval-subdir=test_fullyLex --labeled-batch-size=64 --random-seed=366 --word-frequency=5 --print-freq=2 --pretrained-wordemb=True --wordemb-size=100 --random-initial-unkown=True --update-pretrained-wordemb=True --run-name gids_labels100.0_labeled64_pre_update_wf5_fullyLex_test --ckpt-file='best.ckpt' --ckpt-path='main_log_gids_labels100.0_labeled64_pre_update_wf5_fullyLex_test/2019-04-03_19:39:45/0/transient' > test_results/gids_labels100.0_labeled64_pre_update_wf5_fullyLex_test.txt

About

'Semi-Supervised Teacher-Student Architecture for Relation Extraction' adapt Mean Teacher, a denoising semi-supervised framework, to improve the performance of relation extraction by incorporating both labeled and unlabeled data in the training process.

Topics

Resources

Stars

Watchers

Forks