How to do distributed training with for example NCCL

[npuichigo]

candle backend seems to have suuport for distributed training. so how to use that in burn

[antimora]

We only take advantage of Tensor operations of Candle. @nathanielsimard and @louisfd have recently added some ground work for distributed computing but it's not fully complete I believe.

[sharifhsn]

Any updates on the distributed computing story in burn?

[nathanielsimard]

We recently merged the backend Router (#2353). This is the foundation to support distributed training with any backend, or even mixed backends. Essentially, with lazy execution we transform tensor function calls into data and send those to backend runners to be executed. Right now everything is in-memory, but we plan to also support HTTP connections, or any network protocol really.