f1 score of AraBERTv01 on NER task

[AishaHarbi]

Hi,
It is mentioned in the AraBERT paper that the performance after finetuning the AraBERT v01 on NER is 84.2. However, when I tried to run the notebook in arabert/examples/old/AraBERT_ANERCorp_CamelSplits.ipynb I barely reached 83.1
My question is: was grid search used to obtain the score of 84.2? or is it a problem of hardware limitation that I'm running into?

Thanks so much

[WissamAntoun]

That score was obtained after doing some cleaning on the original AnerCorp dataset. For the most updated results please refer to the AraELECTRA paper where we use the AnerCorp-CamelSplits dataset https://camel.abudhabi.nyu.edu/anercorp/ . Since the way we split the dataset greatly affects the performance.

[AishaHarbi]

• I was referring to the table in this AraBERT paper (which indicates using v0.1 with NER yielded an 84.2 score). https://arxiv.org/abs/2003.00104

• Also, the notebook in here arabert/examples/old/AraBERT_ANERCorp_CamelSplits.ipynb uses the same Camel lab split of AnerCorp, was the 84.2 on v0.1 obtained on the AnerCorp before applying camel lab split?
  Thanks so much,

[WissamAntoun]

Yes those were obtained before applying camel lab splits. Actually, this result was the motivation behind creating the CamelLab splits.

[AishaHarbi]

Great, what was the split? 5/6 and the rest are test data? or else?