Multi-Agent Support

[eadlam]

I was browsing through SML-Lab docs and saw this note:

The agent spec is a list to accommodate for multi-agent setting in the future version of SLM Lab

Is multi-agent support something you've already begun working on and/or are you looking for contributors?

[kengz]

Hi @eadlam sorry for the response, we've been away.
An earlier iteration of the lab's design had support for multi-agent-multi-environment, but it was not performant and so we retired it. The current SLM Lab is only single-agent, but still has multi-environment (parallel as a list). Since then we have not worked on it.

We definitely welcome contributions; it's also important to observe what's the current convention (data-format) for doing multi-agent. Another learning from our multi-agent attempt was that for larger games requiring larger models (e.g. Atari games), agents need to be of separate processes, so either it needs a large machine or it needs to be distributed.