Skip to content

ZJCV/KnowledgeReview

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Language: 🇺🇸 🇨🇳

«KnowledgeReview» re-implements the paper Distilling Knowledge via Knowledge Review

arch_s top1 top5 arch_t top1 top5 dataset lambda top1 top5
MobileNetv2 80.620 95.820 ResNet50 83.540 96.820 CIFAR100 7.0 83.370 96.810
MobileNetv2 80.620 95.820 ResNet152 85.490 97.590 CIFAR100 8.0 84.530 97.470
MobileNetv2 80.620 95.820 ResNeXt_32x8d 85.720 97.650 CIFAR100 6.0 84.520 97.470
ResNet18 80.540 96.040 ResNet50 83.540 96.820 CIFAR100 10.0 83.130 96.350
ResNet50 83.540 96.820 ResNet152 85.490 97.590 CIFAR100 6.0 86.240 97.610
ResNet50 83.540 96.820 ResNeXt_32x8d 85.720 97.650 CIFAR100 6.0 86.220 97.490

more see docs

Table of Contents

Background

Unlike overhaul and other knowledge transfer algorithms,knowledge review use cross stage's teacher features to train student features. Meanwhile, it designed a new residual learning framework to simplify student transfer operation, and use ABF (attention based fusion) and HCL (hierarchical context loss) to improve feature distillation.

Current project implementation is based on ZJCV/overhaul and dvlab-research/ReviewKD.

Installation

$ pip install -r requirements.txt

Usage

  • Train
$ CUDA_VISIBLE_DEVICES=0 python tools/train.py -cfg=configs/rfd/resnet/rfd_6_0_r152_pret_r50_c100_224_e100_sgd_mslr.yaml
  • Test
$ CUDA_VISIBLE_DEVICES=0 python tools/test.py -cfg=configs/rfd/resnet/rfd_6_0_r152_pret_r50_c100_224_e100_sgd_mslr.yaml

Maintainers

  • zhujian - Initial work - zjykzj

Thanks

@misc{chen2021distilling,
      title={Distilling Knowledge via Knowledge Review}, 
      author={Pengguang Chen and Shu Liu and Hengshuang Zhao and Jiaya Jia},
      year={2021},
      eprint={2104.09044},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Contributing

Anyone's participation is welcome! Open an issue or submit PRs.

Small note:

License

Apache License 2.0 © 2021 zjykzj

Releases

No releases published

Packages

No packages published

Languages