This repository contains code connected to the paper "How Conservative are Language Models? Adapting to the Introduction of Gender-Neutral Pronouns." accepted at NAACL 2022.
Please refer to the paper for further details.
So far, we provide code to compute perplexity scores and correlate them with pre-computed
flow values. To carry out this analysis for Swedish, you first need to download the file
"HEN_preprocessing_scripts/HENprepdata.txt" here
and include the local data path into config.yaml. For Danish and English you find the translated version of that dataset here in the repo under data/stimuli/{lan}_translated_{pron}.pkl.
In config.yaml you also need to set the language (currently: en, da, sv) and the corresponding pronoun (they/xe, de/høn, hen respectively) and then run run_perplexity_flow.py