Train GoPro Datasets error

[MscWood1]

Hello, i tried to train gopro dataset with default setting in single GPU. It works well at the first but raised "CUDA out of memory" error when at iter 10000. I wonder how could this happen or how could I modify the setting to avoid this error.
Thx for ur answer.

[MscWood1]

btw I disabled the validation because it also raises "CUDA out of memory" when it finished a validation and steps into the next training.