Dense Semantic Correspondence where Every Pixel is a Classifier

TL;DR

This paper introduces a novel approach to dense semantic correspondence by treating each pixel as a classifier, leveraging exemplar LDA classifiers for improved accuracy and interpretability, and modeling the problem with a graphical model to ensure smoothness.

Contribution

It proposes using exemplar LDA classifiers for each pixel to improve semantic correspondence accuracy and interpretability, and formulates the problem as a graphical model with smoothness constraints.

Findings

Exemplar LDA classifiers outperform traditional similarity metrics.

The approach provides globally interpretable posterior probabilities.

The method is faster to train than exemplar SVMs.

Abstract

Determining dense semantic correspondences across objects and scenes is a difficult problem that underpins many higher-level computer vision algorithms. Unlike canonical dense correspondence problems which consider images that are spatially or temporally adjacent, semantic correspondence is characterized by images that share similar high-level structures whose exact appearance and geometry may differ. Motivated by object recognition literature and recent work on rapidly estimating linear classifiers, we treat semantic correspondence as a constrained detection problem, where an exemplar LDA classifier is learned for each pixel. LDA classifiers have two distinct benefits: (i) they exhibit higher average precision than similarity metrics typically used in correspondence problems, and (ii) unlike exemplar SVM, can output globally interpretable posterior probabilities without calibration, whilst also being significantly faster to train. We pose the correspondence problem as a graphical model, where the unary potentials are computed via convolution with the set of exemplar classifiers, and the joint potentials enforce smoothly varying correspondence assignment.