about train details

[mouzhengdong]

i run the train classification.py with baseline resnet50,and only get the best train_miou 44.12%. which is lower than your result in paper 47.82%. i used four nvidia 1080ti, conld you tell me the experiments details, thanks!

[shjo-april]

Hello,
thanks for question.

I already put up prerequisite on my github page. You can refer to that.
When I make ResNet-50 train PASCAL VOC 2012 dataset, I don't fix the parameters of batch normalization. Specifically, the size of VRAM on 4 GTX 1080 Ti is approximately 42GB. However, the size of VRAM on 4 Titan RTX is approximately 96GB. As you know, batch normalization is very sensitive to batch size. So, your mIoU is reasonable. In conclusion, I recommend two ways which are to increase batch size or to replace normal batch normalization to fixed batch normalization.

Best regard,
Sanghyun

[mouzhengdong]

Thanks for your reply, but i dont have the 96GB of VRAM. and i try the fix BN,the loss turned to NAN soon.
and I have noted that the syncBN will compute BN parametres of all devices and will it be
effective?

[lucasdavid]

@mouzhengdong I'm also getting NaN when setting fixed batch norm. Did you manage to fix this somehow?