https://arxiv.org/abs/2102.11972
Do Transformer Modifications Transfer Across Implementations and Applications? (Sharan Narang, Hyung Won Chung, Yi Tay, William Fedus, Thibault Fevry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam Roberts, Colin Raffel)
trasformer variant들에 대한 실험. 이런 연구들이 대부분 그렇듯 결론은 variant들 사이에 별 차이가 없더라이긴 하죠.
#transformer