Sketch-based Image Retrieval Algorithm

Collection of subarasii and reproducible SBIR works.

Criteria: works must have codes available, and the reproducible results demonstrate state-of-the-art.

This collection is inspired by the image denoising collection. It makes me have an idea, why don't I summarize the SBIR works just like he did.

Feature descriptor

Global

• GF-HoG [Web] [Web] [Code] [PDF]
  • Gradient field descriptor for sketch based retrieval and localization (ICIP, 2010), Rui Hu et al.
  • A Bag-of-Regions Approach to Sketch Based Image Retrieval (ICIP, 2011), Rui Hu et al.
  • A Performance Evaluation of Gradient Field HOG Descriptor for Sketch Based Image Retrieval (COMPUT VIS IMAGE UND, 2013), Rui Hu et al.
• Color GF-HoG [Web] [Code] [PDF]
  • Scalable Sketch-based Image Retrieval using Color Gradient Features (ICIP, 2015), Tu Bui et al.

Local

• SHOG [Code(msvs)] [Code(origin)]
  • Sketch-based image retrieval: Benchmark and bag-of-features descriptors (IEEE T VIS COMPUT GR, 2011), Eitz M et al.
  • A descriptor for large scale image retrieval based on sketched feature lines (SBM, 2009), Eitz M et al.
  • How do humans sketch objects? (TOG, 2012), Eitz M et al.

Deep Learning

Modelling

• Classification

  • Sketch-a-Net that Beats Humans (BMVC, 2015), Qian Y et al. [PDF] [trained model] & Sketch-a-Net: A Deep Neural Network that Beats Humans (IJCV, 2016), Qian Y et al. [Code(Tensorflow)] [Code(Tensorflow)(modified net)] [Code(Matlab)] [Code(Pytorch)] [PDF]

• Triplet loss CNN

  • The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies (TOG, 2016), Sangkloy P et al. [Web] [Code(Caffe)] [PDF]
  • Compact Descriptors for Sketch-based Image Retrieval using a Triplet loss Convolutional Neural Network (CVIU, 2017), Tu Bui et al. [Web] [Code(Caffe)] [Code(Pytorch)] [PDF]
  • Sketch Me That Shoe¹ (CVPR, 2016), Yu Q et al. [Web] [Code(Caffe)] [Code(Tensorflow)] [PDF] [Dataset]
  • Deep Spatial-Semantic Attention for Fine-grained Sketch-based Image Retrieval¹ (ICCV, 2017), Song J et al. [Web] [Code(Tensorflow)] [PDF] [Dataset]

• DPM

  • Fine-grained sketch-based image retrieval by matching DPM¹ (unpublished), Yi L et al. [Web] [Code(Matlab)] [PDF]

¹: Fine-grained retrieval

Hash Coding

• Multi-column weight sharing
  • Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrieval (CVPR, 2017), Li L et al. [Web] [Code(Caffe)] [PDF]

Benchmark and Dataset

• Flickr15K [Web] [Image(source and groundtruth)] [Image(resized)] [Sketch] [Image(source and groundtruth)] [Image(resized)] [Sketch] [groundtruth] [PDF]
  • A Performance Evaluation of Gradient Field HOG Descriptor for Sketch Based Image Retrieval (CVIU 2013), Rui Hu et al.
  • 14,660 images labelled into 33 categories based on shape only
  • 10 non-expert sketchers(5 males, 5 females)
  • 330 free-hand sketches
• Flickr25K [Web] [Image] [PDF]
  • Compact Descriptors for Sketch-based Image Retrieval using a Triplet loss Convolutional Neural Network (CVIU 2017), Tu Bui et al.
  • 25,000 images labelled into 250 categories
  • images only
  • can be used associated with TU-Berlin, shown as follows
• TU-Berlin [Web] [Sketch] [PDF]
  • How Do Humans Sketch Objects? (Siggraph 2012), Eitz M et al.
  • 20,000 unique sketches evenly distributed over 250 object categories
  • sketch only
• Image100k [Web] [Image] [Benchmark/Sketch] [PDF]
  • Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors (IEEE T VIS COMPUT GR 2012), Eitz M et al.
  • [Benchmark] has 1,240 images labelled into 31 categories and 31 corresponding query sketches for testing
  • [Image] has 101,240 images for training
  • 28 participants(23 males, 5 females)
• GOLD [Web] [Image(resized)] [PDF]
  • Sketch-Based Image Retrieval by Salient Contour Reinforcement (IEEE T MULTIMEDIA 2016), Zhang Y et al.
  • contains more than 22,000 Flickr Crawled images together with their Geotags
  • extend image set, image only
• Sketchy [Web] [Image&Sketch] [PDF]
  • The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies (TOG, 2016), Sangkloy P et al.
  • sampled from 125 categories and acquire 75,471 sketches of 12,500 objects
  • a benchmark contained
• SketchyScene [Web] [PDF]
  • SketchyScene: Richly-Annotated Scene Sketches (ECCV, 2018), Changqing Z et al.