loss suddenly increasing after some training steps

[pashanitw]

Hello,
Thanks for the great codebase. i am working on training the model for hindi lang. i have 4k hours of training data. so i am training it from scratch. i changed the hubert model and using hindi bert models instead. then i am training the model from scratch s2_train. but loss suddenly increasing after some steps. i attached the screenshot of training losses below

[RVC-Boss]

when meeting loss suddenly increasing issue, maybe fp32 training can fix it