add my own data to the pretrained model

[nada2017]

Hi, thank you for this great work.
I am a beginner in this and I would like to know how can I add my data to the pretrained model without training all data all over again?

[WissamAntoun]

For continuing Pre-training I suggest you use this code https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm.py. It will run masked language modeling without the next sentence prediction task. You just have to provide it with a text file with 1 sentence per line i think. It works directly with all AraBERT models (notice that for v1 and v2 you have to pre-segment the text data first)

The task is bit hard to setup, but good luck

[AmjadNas]

Hi Wissam

could you please provide another example?
the link you provided no longer exists.

Thanks

[WissamAntoun]

the file lives here now https://github.com/huggingface/transformers/tree/main/examples/pytorch/language-modeling