v0.2.3
Add FlashAttention2 support ⚡
- Add FlashAttention2 support as mentioned here
- Remove old wrapper for
half()precision since Lightning already deals with this - Fix
scaled_dot_product_attentionimplementation in PyTorch < 2.0 - Minor fixes
half() precision since Lightning already deals with thisscaled_dot_product_attention implementation in PyTorch < 2.0