Coherency Sensitive Hashing (CSH) extends Locality Sensitivity Hashing (LSH) and PatchMatch to quickly find matching patches between two images. LSH relies on hashing, which maps similar patches to the same bin, in order to find matching patches. PatchMatch, on the other hand, relies on the observation that images are coherent, to propagate good matches to their neighbors in the image plane, using random patch assignment to seed the initial matching. CSH relies on hashing to seed the initial patch matching and on image coherence to propagate good matches. In addition, hashing lets it propagate information between patches with similar appearance (i.e., map to the same bin). This way, information is propagated much faster because it can use similarity in appearance space or neighborhood in the image plane. As a result, CSH is at least three to four times faster than PatchMatch and more accurate, especially in textured regions, where reconstruction artifacts are most noticeable to the human eye. We verified CSH on a new, large scale, data set of 133 image pairs and experimented on several extensions, including: k nearest neighbor search, the addition of rotation and matching three dimensional patches in videos.
|Number of pages||14|
|Journal||IEEE Transactions on Pattern Analysis and Machine Intelligence|
|State||Published - 1 Jun 2016|
Bibliographical noteFunding Information:
This work was partially supported by Israel Science Foundation grant 1,556/10 and European Community grant PIRG05-GA-2009-248527. The authors thank Yonatan Hyatt, Guy Shwartz and Efrat Glikstein for their assistance.
© 2015 IEEE.
- Image Matching
- Nearest Neighbor Fields
- Patch Matching
- Video Matching
ASJC Scopus subject areas
- Computer Vision and Pattern Recognition
- Computational Theory and Mathematics
- Artificial Intelligence
- Applied Mathematics