Image-Backbones-PyTorch

Implement image backbones with PyTorch.

This repo also serves as a template for my deep learning projects.

Installation

The code is tested with python 3.12, torch 2.4.1 and cuda 12.4.

Clone this repo:

git clone https://github.com/xyfJASON/image-backbones-pytorch.git
cd image-backbones-pytorch

Create and activate a conda environment:

conda create -n backbones python=3.12
conda activate backbones

Install dependencies:

pip install torch==2.4.1 torchvision==0.19.1 --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt

Training

# single gpu
python train.py -c CONFIG [-e EXP_DIR] [--xxx.yyy zzz ...]

# multiple gpus (e.g., 4 gpus)
torchrun --nproc_per_node 4 train_ddp.py -c CONFIG [-e EXP_DIR] [--xxx.yyy zzz ...]

• Results (logs, checkpoints, tensorboard, etc.) of each run will be saved to EXP_DIR. If EXP_DIR is not specified, they will be saved to runs/exp-{current time}/.
• To modify some configuration items without creating a new configuration file, you can pass --key value pairs to the script. For example, the default optimizer in ./configs/resnet18_cifar10.yaml is SGD, and if you want to change it to Adam, you can simply pass --train.optim.type Adam.

For example, to train resnet18 on CIFAR-10:

python train.py -c ./configs/resnet18_cifar10.yaml

Results

CIFAR-10 Benchmark

models	#params	MACs	acc@1(%)
VGG-11	9.20M	0.15G	90.97
VGG-19 (BN)	20.0M	0.40G	94.00
ResNet-18	11.2M	5.59G	95.64
PreActResNet-18	11.2M	5.59G	95.45
ResNeXt-29 (32x4d)	4.78M	6.90G	95.16
SE-ResNet-18	11.3M	5.59G	95.65
CBAM-ResNet-18	11.3M	5.59G	95.49
MobileNet	3.22M	0.48G	92.09
ShuffleNet 1x (g=8)	0.91M	0.50G	92.82
ViT-Tiny/4	5.36M	0.37G	85.66

Note: MACs are calculated by fvcore library.

All the ConvNets are trained with the following settings:

• training duration: 64k steps
• batch size: 256
• learning rate: start with 0.1, end with 0.001 using a cosine annealing scheduler, no warm-up
• optimizer: SGD, weight decay 5e-4, momentum 0.9

The ViTs are trained with the following settings:

• training duration: 64k steps
• batch size: 512
• learning rate: start with 0.001, end with 0.00001 using a cosine annealing scheduler, no warm-up
• optimizer: Adam, weight decay 5e-5, betas (0.9, 0.999)