-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WC task bert accuracy is much lower than paper claimed #2
Comments
It's about one month and still no one reply ... The issue is relatively critical because I have tried different language models with current codebase and none of them has a reasonable result on this one. Is there any reason behind that? I am very grateful if someone can reply me. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have tried use the SentEval WC dataset to evaluate Bert performance. But the result is much lower than the paper claimed. (0.4 compares to 24.9 in paper for layer 0). I achieve similar performance on all other 9 tasks.
The text was updated successfully, but these errors were encountered: