Problematic normalization

[Coco-hanqi]

Got a validation accuracy around 58%, lower than the one proposed in the paper. Is the lower accuracy caused by this problematic normalization error?

[matyasbohacek]

Thanks for opening this ticket, @Coco-hanqi.

Yes, this seems strange, probably related to the under-performing accuracy. Let me try to reproduce this behavior on a new machine and investigate the source of the problem -- will get back to you here shortly.

[caicairay]

I also got this problem. It seems because of the problematic key points data. The left and right shoulder key points are overlapped in some frames.

[matyasbohacek]

Dear @Coco-hanqi & @caicairay,

I finally managed to reproduce this on an older external machine. Upon further research, it seems that some PyTorch versions seem to have various low-level issues with model checkpointing via saving the state dictionary. Interestingly enough, the same code with no edits worked on my macOS machine. Hence, the script has been rewritten to keep the entire model objects, which managed to solve this on the Linux instance where I reproduced this.

Please, let me know if this issue persists, and we can look into this deeper. Feel free to reach out should any other problems or questions arise.

[andresherrera97]

I'm getting the same issue of the Problematic Normalization on a Ubuntu 22 machine, any update on this? Should I expect this to affect the performance?

[andresherrera97]

I realized that Problematic Normalization occurs during training when the augmentation happens to be rotation. Disabling the option of rotation prevents the Problematic Normalization and accuracy on test increases to 60%, which is still below the results in the paper

[saurus2]

Hi,
I had a same problem.
The Validation accuracy was not over 60%.

I used MacOS though I got a problem.
Then I used my campus HPC server.
Then I got same problems.

I used WLASL 100 25fps dataset.
Somebody have an idea?

[ShortCatislong]

I realized that Problematic Normalization occurs during training when the augmentation happens to be rotation. Disabling the option of rotation prevents the Problematic Normalization and accuracy on test increases to 60%, which is still below the results in the paper

could you please kindly remind me which rotation method you have disabled?
the __rotate() , augment_rotate() or augment_arm_joint_rotate()?