Best Buddies Similarity for Robust Template Matching

Tali Dekel, Shaul Oron, Michael Rubinstein, Shai Avidan, William T. Freeman


We propose a novel method for template matching in unconstrained environments. Its essence is the Best Buddies Similarity (BBS), a useful, robust, and parameter-free similarity measure between two sets of points. BBS is based on counting the number of Best Buddies Pairs (BBPs)—pairs of points in source and target sets (resp.) which each one is the nearest neighbor of the other. BBS has several key features that make it robust against complex geometric deformations and high levels of outliers, such as those arising from background clutter and occlusions. We study these properties, provide a statistical analysis that justifies them, and demonstrate the consistent success of BBS on a challenging real-world dataset.


  • Best Buddies Similarity for Robust Template Matching PDF BIB
    Tali Dekel*, Shaul Oron*, Michael Rubinstein, Shai Avidan, William T. Freeman
    Computer Vision and Pattern Recognition, 2015

    * The first two authors contributed equally to this work


Three toys examples are shown: (A) cluttered background, (B) occlusions, (C) nonrigid deformation. The template (first column) is detected in the target image (second column) using the BBS; the results using BBS are marked in a blue. The likelihood maps (third column) show welllocalized distinct modes. The BBPs are shown in last column. See paper for more details.

BBS results on Real Data: (a), the templates are marked in green over the input images. (b) the target images marked with the detection results of 6 different methods (see text for more details). BBS results are marked in blue. (c)-(e), the resulting likelihood maps using BBS, EMD and NCC , respectively; each map is marked with the detection result, i.e., its global maxima.

Code and Data

BBS code and data CODE

If using our code or data please cite our paper

Please note this is academic code, USE AT YOUR OWN RISK !

Code is distributed under the GNU GPL license for more information go to