Skip to content

Code for DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning

Notifications You must be signed in to change notification settings

Yuting-Gao/DisCo-pytorch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

29 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning(ECCV-2022 Oral)

This repository contains the Official Pytorch Implementation for DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning

@article{gao2021disco,
  title={DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning},
  author={Yuting Gao, Jia-Xin Zhuang, Shaohui Lin, Hao Cheng, Xing Sun, Ke Li, Chunhua Shen},
  journal={European Conference on Computer Vision(ECCV)},
  year={2022}
}

If the project is useful to you, please give us a star. ⭐️

Framework

image

Checkpoints

Teacher Models

Architecture Self-supervised Methods Model Checkpoints
ResNet152 MoCo-V2 ResNet152-checkpoint_0799.pth.tar
ResNet101 MoCo-V2 ResNet101-checkpoint_0199.pth.tar
ResNet50 MoCo-V2 ResNet50-checkpoint_0199.pth.tar

For teacher models such as ResNet-50*2 etc, we use their official implementation, which can be downloaded from their github pages.

Student Models by DisCo

Teacher/Students Efficient-B0 Efficient-B1 ResNet-18 ResNet-34 MobileNet-v3 Vit-Tiny XCiT-Tiny
ResNet-50 ResNet50-EfficientB0-checkpoint_0199.pth.tar DisCo-R50-Effb1.pth.tar ResNet50-ResNet18-checkpoint_0199.pth.tar DisCo-R50-R34.pth.tar DisCo-R50-Mob.pth.tar - -
ResNet-101 ResNet101-EfficientB0-checkpoint_0199.pth.tar DisCo-R101-Effb1.pth.tar ResNet101-ResNet18-checkpoint_0199.pth.tar DisCo-R101-R34.pth.tar DisCo-R101-Mob.pth.tar - -
ResNet-152 ResNet152-EfficientB0-checkpoint_0199.pth.tar DisCo-R152-Effb1.pth.tar ResNet152-ResNet18-checkpoint_0199.pth.tar DisCo-R152-R34.pth.tar DisCo-R152-Mob.pth.tar - -
ResNet-50*2 ResNet50w2-EfficientB0-checkpoint_0199.pth.tar DisCo-RN50x2-Effb1.pth.tar ResNet50w2-ResNet18-checkpoint_0199.pth.tar DisCo-RN50x2-R34.pth.tar DisCo-RN50x2-Mob.pth.tar - -
ViT-Small - - - - - DisCo-ViT-small-ViT-tiny.pth -
XCiT-Small - - - - - - XCiT-Small-XCiT-Tiny-checkpoint.pth

Requirements

  • Python3

  • Pytorch 1.6+

  • Detectron2

  • 8 GPUs are preferred

  • ImageNet, Cifar10/100, VOC, COCO

Reproduction

Commands can be found on Reproduction.

Thanks

Code heavily depends on MoCo-V2, Detectron2.

About

Code for DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages