Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Implement Spatial Transformer used by LDM UNet (#438)
Summary: Pull Request resolved: #438 **TL;DR** Implements the Spatial Transformer and the Cross-Attention layers used within the LDM Unet. **Summary of Changes** 1. Implement `TransformerCrossAttentionLayer` similar to LDM;s implementation. This module uses `GEGLU` activation instead of `GLU` 1. Implements `SpatialTransformer` that operates on images. 1. Implement `GEGLU` activation 1. Implement `zero_module` util 1. Add tests for each component Differential Revision: D47709486 fbshipit-source-id: b4945c6401b302503212d014767a827f4ac90c0b
- Loading branch information