Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

neg training code is different from the paper #7

Open
seominjoon opened this issue Oct 8, 2019 · 0 comments
Open

neg training code is different from the paper #7

seominjoon opened this issue Oct 8, 2019 · 0 comments

Comments

@seominjoon
Copy link
Owner

Currently the neg training routine (--train_neg in run_piqa.py) is different from what is described in the paper.

In the paper, we use 'no answer' logit to train on negative examples so we just don't have a separate neg training routine. In the code, we have a neg training routine that instead attaches the neg example to each positive example (whose question embeddings are similar) after normal training.

In the code, several noise injections are also used.

In practice, the strategy in the current code is better than that in the paper (no answer logit). The paper will be updated soon and this issue will be resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant