📚 Tensor/CUDA Cores, 📖150+ CUDA Kernels, toy-hgemm library🔥(achieve the performance of cuBLAS 🎉🎉).

Root Mean Square Layer Normalization

Code for the paper "On the Expressivity Role of LayerNorm in Transformers' Attention" (Findings of ACL'2023)

Implement layer normalization GRU in pytorch

Some common CUDA kernel implementations (Not the fastest).

MNIST Digit Prediction using Batch Normalization, Group Normalization, Layer Normalization and L1-L2 Regularizations

WGAN with feedback from discriminator& LayerNorm instead of BatchNorm

Fundamentals of Artificial Intelligence and Deep Learning Frameworks