210418 Data-Efficient Language-Supervised Zero-Shot Learning with.md

https://arxiv.org/abs/2104.08945

Data-Efficient Language-Supervised Zero-Shot Learning with Self-Distillation (Ruizhe Cheng, Bichen Wu, Peizhao Zhang, Peter Vajda, Joseph E. Gonzalez)

clip 데이터 양 낮춰보기. 프리트레이닝된 인코더 + self distillation. clip이 파워풀한 imagenet에서는 밀리긴 합니다.

#multimodal #pretraining