I use bert large for pretrain on vcr and encountered the error ResourceExhaustedError: OOM when allocating tensor

[AeroXi]

I tried using bert large instead of bert in the original code, and modified three parameters (hidden size=1024, hidden layers=24, attention heads=16) in bert config.
Here's the error log:
https://gist.github.com/AeroXi/d4d273da9f443c0f2cf9f6d6872eeffe
My device is 4 1080Ti
Maybe I can skip domain adaption and just extract features? However, the generated filename starts with "bert" instead of "bert_da", I can't use it directly even changed the correct filename when training r2c. Should I make other modification?

[liucongg]

请问您的问题解决了吗，我也遇到了相同的问题。就是用bert自带的optimizer就会OOM，如果用Adam就不会。

[tuyunbin]

请问您的问题解决了吗，我也遇到了相同的问题。就是用bert自带的optimizer就会OOM，如果用Adam就不会。

请问vcr这个数据集你怎么下的，为什么我点了‘I agree'以后，下面那两个蓝底的Annotations和Images点了没有任何反应

[JunnYu]

@tuyunbin F12审查元素，查看对应下载链接。