Backend and autodiffbackend

[wangjiawen2013]

Hi,
It is said that "not all tensors support auto-differentiation; you need a backend that implements both the Backend and AutodiffBackend traits." Now both the Backend and AutodiffBackend are used in burn's examples. Can we use AutodiffBackend instead of Backend and why don't use AutodiffBackend all the time ?

[laggui]

In the examples we use Autodiff to wrap the backend in use (e.g., Autodiff<B>).

Backend and AutodiffBackend are both traits that are implemented for the backends to support tensor operations and auto differentiation respectively.

When you use Autodiff<B>, then AutodiffBackend is used all the time 😉 and having it this way makes it nice and easy to specify that you want to add auto differentiation to the backend when you need it.

[wangjiawen2013]

Hi, what I really mean is that why don't we discard Backend and use AutodiffBackend all the time. In my opinion, AutodiffBackend can do anything that Backend can do.
Or in other words, why do you design the Backend as a super trait of AutodiffBackend ?

[laggui]

Actually, neither Backend nor AutodiffBackend requires the other trait. And both serve different purposes. The Backend trait is the most important to define your tensor operations and data types. If you were to implement a backend that is strictly used like an inference runtime and you're not interested in autdodiff, then you only have to implement the Backend trait, not AutodiffBackend.

Autodiff is an extension, and you don't want to always track the operations graph and gradients when you compute something. That's why it is only required for training, and you can enable it via the Autodiff<B> decorator for any backend B that implements AutodiffBackend.