Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

显存很快就满了 #48

Open
Julong-Zheng opened this issue Jun 30, 2022 · 8 comments
Open

显存很快就满了 #48

Julong-Zheng opened this issue Jun 30, 2022 · 8 comments

Comments

@Julong-Zheng
Copy link

机器4张卡,单卡显存12G,上限只能跑如下参数,再大报显存错误了。
python -u train.py --gpu_devices 0,1,2,3 --num_actor_devices 3 --num_actors 2 --training_device 3

@Starshowss
Copy link

Starshowss commented Jun 30, 2022 via email

@daochenzha
Copy link
Collaborator

你用的什么型号的显卡 操作系统是什么

@Julong-Zheng
Copy link
Author

你用的什么型号的显卡 操作系统是什么

操作系统是ubuntu,显卡是RTX2080Ti,cuda版本是10.1,torch版本是1.7

@daochenzha
Copy link
Collaborator

试下只用一个显卡看能跑几个,多个显卡可能哪里出问题了

@Julong-Zheng
Copy link
Author

试下只用一个显卡看能跑几个,多个显卡可能哪里出问题了

单卡上限如下——

python -u train.py --gpu_devices 0,1 --num_actor_devices 1 --num_actors 7 --training_device 1

@Julong-Zheng
Copy link
Author

机器重启了一下,可能之前有什么其他问题吧,目前4张卡能跑到的上限如下——
python -u train.py --gpu_devices 0,1,2,3 --num_actor_devices 3 --num_actors 9 --training_device 3

num_actors 还是到不了你们的15

@1978mountain
Copy link

这个启动一个act进程,就会占用大概2g显存。

@Julong-Zheng
Copy link
Author

这个启动一个act进程,就会占用大概2g显存。

无所谓啦,我能启动9*3-27个actor也行了,多跑一段时间即可。但是现在主要是效果复现不了,跑了一个月还是很糟糕,完全达不到他们放到网上的水平,虽然网上那个水平也并不很好(反正打不过我)。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants