We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
In the DDPM Unet implementation, the residual blocks incorporate the time embedding by applying a linear layer only with no prior activation:
annotated_deep_learning_paper_implementations/labml_nn/diffusion/ddpm/unet.py
Line 130 in b1f5c8e
Line 80 in b1f5c8e
In the original tensorflow implementation by the author, the time embedding is first passed through a nonlinearity and only then through a linear layer: https://github.com/hojonathanho/diffusion/blob/1e0dceb3b3495bbe19116a5e1b3596cd0706c543/diffusion_tf/models/unet.py#L49
The text was updated successfully, but these errors were encountered:
Yes you are correct, we have missed activation layer
Sorry, something went wrong.
vpj
No branches or pull requests
In the DDPM Unet implementation, the residual blocks incorporate the time embedding by applying a linear layer only with no prior activation:
annotated_deep_learning_paper_implementations/labml_nn/diffusion/ddpm/unet.py
Line 130 in b1f5c8e
However, the positionally encoded time embedding is already the result of a linear layer:
annotated_deep_learning_paper_implementations/labml_nn/diffusion/ddpm/unet.py
Line 80 in b1f5c8e
Hence, both these layers collapse to a single linear layer with no non-linear mapping per residual block.
In the original tensorflow implementation by the author, the time embedding is first passed through a nonlinearity and only then through a linear layer:
https://github.com/hojonathanho/diffusion/blob/1e0dceb3b3495bbe19116a5e1b3596cd0706c543/diffusion_tf/models/unet.py#L49
The text was updated successfully, but these errors were encountered: