Neural Best Buddies

Algorithm implemented based on Neural Best-Buddies: Sparse Cross-Domain Correspondence for feature matching between images.

Neural activations ("feature maps") in different layers are used to match semantic correspondences between images.

The end result is matched pixels in two images.

  

By adding a constraint where to initially look inside the images, you can find correspondences in the same image.

The algorithm relies heavily on geometry and is therefore easily fooled.

References

• Kfir Aberman, Jing Liao, Mingyi Shi, Dani Lischinski, Daniel Cohen-Or, Chen Baoquan. Neural Best-Buddies: Sparse Cross-Domain Correspondence, ACM Transactions on Graphics (TOG), 37(4), 2018.
• X. Huang and S. Belongie, "Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization," ICCV, 2017, pp. 1510-1519.