Any suggestion about gradient accumulation and BatchNorm?

[tangjiasheng]

So if I directly use deepspeed on regular CNN model with BN, is there any suggestion about the usage?
e.g. Using gradient accumulation with large batch size thus to avoid BN unstable, is it right? Or, there's no need to address.