-
Notifications
You must be signed in to change notification settings - Fork 278
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Key error 0 on evaluation set #90
Comments
I have the same issue. Have you resolved it? I think you can treat 0's index as PAD tag/or O tag. And then compute scores exclude PAD/O tag. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello,
I built a sample model with only 18 records.
The tokenization used was from spacy and the format is BILUO instead of BIO and I only have one class which is Org. I modified the get labels function and also verified the model config json file.
My labels looks like this,
["O", "B-ORG", "I-ORG", "L-ORG", "U-ORG", "[CLS]", "[SEP]"]
The training goes fine, but validation fails with the following error.
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
06/12/2020 14:07:59 - INFO - main - ***** Running evaluation *****
06/12/2020 14:07:59 - INFO - main - Num examples = 18
06/12/2020 14:07:59 - INFO - main - Batch size = 8
Evaluating: 0%| | 0/3 [00:00<?, ?it/s]
Traceback (most recent call last):
File "run_ner.py", line 594, in
main()
File "run_ner.py", line 582, in main
temp_2.append(label_map[logits[i][j]])
KeyError: 0
I have attached the train and valid files.
Please point me in the right direction.
train.txt
valid.txt
The text was updated successfully, but these errors were encountered: