The official Faster R-CNN code (written in MATLAB) is available here. If your goal is to reproduce the results in our NIPS 2015 paper, please use the official code.
This repository contains a Python reimplementation of the MATLAB code. This Python implementation is built on a fork of Fast R-CNN. There are slight differences between the two implementations. In particular, this Python port
- is ~10% slower at test-time, because some operations execute on the CPU in Python layers (e.g., 220ms / image vs. 200ms / image for VGG16)
- gives similar, but not exactly the same, mAP as the MATLAB version
- is not compatible with models trained using the MATLAB code due to the minor implementation differences
- includes approximate joint training that is 1.5x faster than alternating optimization (for VGG16) -- see these slides for more information
By Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun (Microsoft Research)
This Python implementation contains contributions from Sean Bell (Cornell) written during an MSR internship.
Please see the official README.md for more details.
Faster R-CNN was initially described in an arXiv tech report and was subsequently published in NIPS 2015.
Faster R-CNN is released under the MIT License (refer to the LICENSE file for details).
If you find Faster R-CNN useful in your research, please consider citing:
@inproceedings{renNIPS15fasterrcnn,
Author = {Shaoqing Ren and Kaiming He and Ross Girshick and Jian Sun},
Title = {Faster {R-CNN}: Towards Real-Time Object Detection
with Region Proposal Networks},
Booktitle = {Advances in Neural Information Processing Systems ({NIPS})},
Year = {2015}
}
- Requirements: software
- Requirements: hardware
- Basic installation
- Demo
- Beyond the demo: training and testing
- Usage
- Requirements for
Caffe
andpycaffe
(see: Caffe installation instructions)
Note: Caffe must be built with support for Python layers!
# In your Makefile.config, make sure to have this line uncommented
WITH_PYTHON_LAYER := 1
You can download my Makefile.config for reference.
2. Python packages you might not have: cython
, python-opencv
, easydict
3. [optional] MATLAB (required for PASCAL VOC evaluation only)
- For training smaller networks (ZF, VGG_CNN_M_1024) a good GPU (e.g., Titan, K20, K40, ...) with at least 3G of memory suffices
- For training with VGG16, you'll need a K40 (~11G of memory)
- Clone the Faster R-CNN repository
# Make sure to clone with --recursive
git clone --recursive https://github.com/rbgirshick/py-faster-rcnn.git
-
We'll call the directory that you cloned Faster R-CNN into
FRCN_ROOT
Ignore notes 1 and 2 if you followed step 1 above.
Note 1: If you didn't clone Faster R-CNN with the
--recursive
flag, then you'll need to manually clone thecaffe-fast-rcnn
submodule:git submodule update --init --recursive
Note 2: The
caffe-fast-rcnn
submodule needs to be on thefaster-rcnn
branch (or equivalent detached state). This will happen automatically if you followed step 1 instructions. -
Build the Cython modules
cd $FRCN_ROOT/lib make
-
Build Caffe and pycaffe
cd $FRCN_ROOT/caffe-fast-rcnn # Now follow the Caffe installation instructions here: # http://caffe.berkeleyvision.org/installation.html # If you're experienced with Caffe and have all of the requirements installed # and your Makefile.config in place, then simply do: make -j8 && make pycaffe
-
Download pre-computed Faster R-CNN detectors
cd $FRCN_ROOT ./data/scripts/fetch_faster_rcnn_models.sh
This will populate the
$FRCN_ROOT/data
folder withfaster_rcnn_models
. Seedata/README.md
for details. These models were trained on VOC 2007 trainval.
After successfully completing basic installation, you'll be ready to run the demo.
Python
To run the demo
cd $FRCN_ROOT
./tools/demo.py
The demo performs detection using a VGG16 network trained for detection on PASCAL VOC 2007.
-
Download the training, validation, test data and VOCdevkit
Download ethier ILSVRC2013 or ILSVRC2014 detecetion task data, including image, bbox label, devkit.
-
Put the right directory to
ILSVRC folder
cd $FRCN_ROOT/lib/datasets vim factory_imagenet.py change devkit_path = 'path to your ILSVRC folder'
-
Follow the next sections to download pre-trained ImageNet models
Pre-trained ImageNet models can be downloaded for the three networks described in the paper: ZF and VGG16.
cd $FRCN_ROOT
./data/scripts/fetch_imagenet_models.sh
VGG16 comes from the Caffe Model Zoo, but is provided here for your convenience. ZF was trained at MSRA.
I only make training, testing model architecture, experimental shell code on end2end version.
To train and test a Faster R-CNN detector using the approximate joint training method, use experiments/scripts/faster_rcnn_end2end_imagenet.sh
.
Output is written underneath $FRCN_ROOT/output
.
cd $FRCN_ROOT
./experiments/scripts/faster_rcnn_end2end_imagenet.sh [GPU_ID] [NET] [--set ...]
# GPU_ID is the GPU you want to train on
# NET in {ZF, VGG_CNN_M_1024, VGG16} is the network arch to use
# --set ... allows you to specify fast_rcnn.config options, e.g.
# --set EXP_DIR seed_rng1701 RNG_SEED 1701
This method trains the RPN module jointly with the Fast R-CNN network, rather than alternating between training the two. It results in faster (~ 1.5x speedup) training times and similar detection accuracy. See these slides for more details.
-
The main different is at $FRCN_ROOT/lib/datasets. I create additional two Python code called factory_imagenet.py and imagenet.py for matching ILSVRC detection task dataset.
- factory_imagenet.py: I change data split into 'train', 'val', 'val1', 'val2', 'test' from '2007', '2012', 'train', 'val', 'trainval', 'test'. Variable "devkit_path" must be set into "path to ILSVRC folder". The other changes are fited for imagenet.py usage.
- imagenet.py: I set categories name from ILSVRC meta_det.mat, get every image index, path, link, and ground truth bounding box. The most important, because the coordinates of bbox ground truth provided by ImageNet Det start from 0, but Pascal Voc dataset start from 1, I make change in "_load_imagenet_annotation" function on bbox coordinates setting. Besides, "_write_imagenet_results_file" function is also change to fit my evaluation code modified from Pascal Voc.
-
My evaluation code is modified from Pascal Voc directly which are get_imagenet_opts.m and imagenet_eval.m in $FRCN_ROOT/lib/datasets/VOCdevkit-matlab-wrapper. Alos, the change is for fitting ILSVRC dataset.
-
The last change I make is in $FRCN_ROOT/tools, including train_net_imagenet.py and test_net_imagenet.py modified from train_net and test_net directly. The only change in those two code is at import module. "from datasets.factory import get_imdb" is changed into "from datasets.factory_imagenet import get_imdb" due to my setting above.
-
It also provides Faset RCNN version training and testing. You just need to put the region proposal produced by one of them:
- Selective Search: original matlab code, python wrapper
- EdgeBoxes: matlab code
- GOP and LPO: python code
- MCG: matlab code
- RIGOR: matlab code
and put the region proposal result in a new folder named "selective_search_data" and put this folder in the "path to ILSVRC folder". The code for data preparing by factory_imagenet.py and imagenet.py will handle it. Then, you need to change model config setting to enable Fast RCNN.
-
Faster RCNN, MSR [Paper]
- Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun,
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
, arXiv:1506.01497.
- Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun,
-
Thanks for Ross Girshick making everthing publicly available!