# Hyperpixel Flow: Semantic Correspondence with Multi-layer Neural Features

## TL;DR

Hyperpixel Flow introduces a multi-layer neural feature-based method for semantic correspondence, achieving state-of-the-art results on multiple benchmarks and a new large dataset, by effectively matching images despite large intra-class variations.

## Contribution

The paper proposes hyperpixels, a novel image representation combining features from multiple CNN layers, and a real-time matching algorithm using Hough voting, advancing semantic correspondence techniques.

## Key findings

- Sets new state-of-the-art on three benchmarks.
- Introduces SPair-71k, a large dataset with detailed annotations.
- Demonstrates real-time performance with effective matching.

## Abstract

Establishing visual correspondences under large intra-class variations requires analyzing images at different levels, from features linked to semantics and context to local patterns, while being invariant to instance-specific details. To tackle these challenges, we represent images by "hyperpixels" that leverage a small number of relevant features selected among early to late layers of a convolutional neural network. Taking advantage of the condensed features of hyperpixels, we develop an effective real-time matching algorithm based on Hough geometric voting. The proposed method, hyperpixel flow, sets a new state of the art on three standard benchmarks as well as a new dataset, SPair-71k, which contains a significantly larger number of image pairs than existing datasets, with more accurate and richer annotations for in-depth analysis.