Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replay Buffer isnt Loaded Enough Yet #38

Open
oyadeyia opened this issue Jun 18, 2020 · 5 comments
Open

Replay Buffer isnt Loaded Enough Yet #38

oyadeyia opened this issue Jun 18, 2020 · 5 comments

Comments

@oyadeyia
Copy link

While running the Actor-Crtic Experiment, "the Pre-Training Critic with fixed Actor", the program stops expectedly after saying the replay buffer isnt loaded enough yet.

The error code is actually this:
W0618 13:04:57.966608 140700510074624 replay_buffer.py:156] Bucket input queue is empty when calling next_batch. Bucket queue size: 0, Input queue size: 0
I0618 13:04:57.967000 140700510074624 run_summarization.py:481] replay buffer not loaded enough yet...
^C

I have had a look at the summarization file and online but I really dont know how to fix this. I noticed it also occurs during the Training with true Q estimates.
@yaserkl Can you please help me out?

@khoaiha12
Copy link

@Fatman003 i have had same error ^C ^C

@oyadeyia
Copy link
Author

@khoaiha12 I think it might have to do with the GPU allocation. try editing the command to use only one GPU(which is available) gpu_num=0..

@khoaiha12
Copy link

@Fatman003 I tried it, but that error still occured. I run on GG Colab, only 1 GPU per session.

@khoaiha12
Copy link

INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Saving checkpoint to path ./src/logs/actor-critic-ddqn2/train/model.ckpt
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:Starting standard services.
INFO:tensorflow:Starting queue runners.
INFO:tensorflow:Saving checkpoint to path ./src/logs/actor-critic-ddqn2/dqn/train/model.ckpt
INFO:tensorflow:current_relay_network/global_step/sec: 0
INFO:tensorflow:Preparing or waiting for session...
INFO:tensorflow:Created session.
INFO:tensorflow:Starting run_training
INFO:tensorflow:Starting DQN training thread...
WARNING:tensorflow:Bucket input queue is empty when calling next_batch. Bucket queue size: 0, Input queue size: 0
INFO:tensorflow:replay buffer not loaded enough yet...
INFO:tensorflow:Starting Seq2Seq training...
INFO:tensorflow:Saving checkpoint to path ./src/logs/actor-critic-ddqn2/train/model.ckpt
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:seconds for dqn collection: 19.2806758881
INFO:tensorflow:Q-values collection time: 43.3507909775
ReplayBatch size: 1546
ReplayBatch example queue size: 1
ReplayBatch batch queue size: 0
INFO:tensorflow:RUNNNING DQN PRETRAIN: Adding data to relplay buffer only...
INFO:tensorflow:Saving checkpoint to path ./src/logs/actor-critic-ddqn2/dqn/train/model.ckpt
WARNING:tensorflow:Bucket input queue is empty when calling next_batch. Bucket queue size: 0, Input queue size: 0
INFO:tensorflow:replay buffer not loaded enough yet...
INFO:tensorflow:seconds for dqn collection: 23.9194500446
INFO:tensorflow:Q-values collection time: 25.8006739616
^C

This is the log when i trained with true Q estimates.

@oyadeyia
Copy link
Author

Did you change your GPU allocation? I didn't have this issue when I changed the gpu_num to 0 and used my school cluster. It retrained but it might be an issue with Colab.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants