Does the bert performance resullt of 24_bert_on_squad.py achieve the benchmark?

[xsu0960]

https://huggingface.co/docs/transformers/model_doc/bert

I have just got an F1 score of about 88. Is that right? I think the analog result should catch up with the digital ones.

[xsu0960]

[maljoras]

The analog noise sources will in general reduce the performance of any model (like weight quantization does as well). The reduction can be minimized when the model is (or is made) robust to this noise. See Rasch et al 2023 for a discussion on this (including evaluation on BERT).