Encoutered a RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x9 and 54x64)

[liufkm]

Hello, I encoutered following error while i tried to run the code:
Traceback (most recent call last):
File "run_madqn.py", line 150, in
main()
File "run_madqn.py", line 45, in main
training_loop(env, rl, outputs_dir)
File "run_madqn.py", line 64, in training_loop
actions = rl.exploration_act(env.normalize_state(states), n_episodes=eps)
File "/home/km/Downloads/RL_Highway_Merge-dev/MARL/MADQN.py", line 118, in exploration_act
action = agent.exploration_action(state, epsilon=self.epsilon)
File "/home/km/Downloads/RL_Highway_Merge-dev/MARL/agent/DQN.py", line 160, in exploration_action
action = self.action(state)
File "/home/km/Downloads/RL_Highway_Merge-dev/MARL/agent/DQN.py", line 166, in action
state_action_value_var = self.actor(state_var)
File "/home/km/Downloads/RL_Highway_Merge-dev/MARL/common/Model.py", line 100, in call
return self.model(state)
File "/home/km/anaconda3/envs/py36sumo4/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/km/anaconda3/envs/py36sumo4/lib/python3.6/site-packages/torch/nn/modules/container.py", line 117, in forward
input = module(input)
File "/home/km/anaconda3/envs/py36sumo4/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/km/anaconda3/envs/py36sumo4/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 93, in forward
return F.linear(input, self.weight, self.bias)
File "/home/km/anaconda3/envs/py36sumo4/lib/python3.6/site-packages/torch/nn/functional.py", line 1690, in linear
ret = torch.addmm(bias, input, weight.t())
RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x9 and 54x64)
Step #0.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 6 ACT 6 BUF 0) Error: tcpip::Socket::recvAndCheck @ recv: peer shutdown
Quitting (on error).

[l-ZhangHao-l]

Hello, I have encountered the same problem as you. Have you resolved it? What should I do? Thank you very much!

[mahmoudtaouti]

Hellow everyone, the problem that you encounter is because the bug made in defining the DQN agents in MADQN.py line 57 by multiplying the state_dim with n_agents. This is totally not correct when dealing with independent agent.

So, I correct this one and remove the multiplication in:
self.agents = [DQN(state_dim=state_dim * n_agents, action_dim=action_dim, memory_capacity=memory_capacity, reward_gamma=reward_gamma, reward_scale=reward_scale, actor_hidden_size=actor_hidden_size, critic_loss=critic_loss, actor_lr=actor_lr, target_update_freq=target_update_freq, optimizer_type=optimizer_type, batch_size=batch_size, epsilon_start=epsilon_start, epsilon_end=epsilon_end, epsilon_decay=epsilon_decay, max_grad_norm=max_grad_norm, use_cuda=use_cuda)] * n_agents
Note that I was trying by doing this multiplication to do a shared learning so agents learn from the total state(all agents state). Well its not a good approach and i did not implemented.

To do shared learning I was trying to try another approach called QMIX network (a centralized approach). But currently I did not complete it.

I hope this Help, Thank you.