GECO: Geometrically Consistent Embedding with Lightspeed Inference

TL;DR

GECO introduces a novel training framework that produces geometrically consistent features with fast inference, significantly improving semantic correspondence accuracy in vision models while addressing limitations of existing self-supervised methods.

Contribution

The paper presents a lightweight, optimal transport-based training method for geometrically coherent features, achieving state-of-the-art results and faster inference compared to prior approaches.

Findings

GECO runs at 30 fps, 98.2% faster than previous methods.

Achieves state-of-the-art PCK scores on PFPascal, APK, and CUB datasets.

Introduces new metrics for evaluating geometric quality of features.

Abstract

Recent advances in feature learning have shown that self-supervised vision foundation models can capture semantic correspondences but often lack awareness of underlying 3D geometry. GECO addresses this gap by producing geometrically coherent features that semantically distinguish parts based on geometry (e.g., left/right eyes, front/back legs). We propose a training framework based on optimal transport, enabling supervision beyond keypoints, even under occlusions and disocclusions. With a lightweight architecture, GECO runs at 30 fps, 98.2% faster than prior methods, while achieving state-of-the-art performance on PFPascal, APK, and CUB, improving PCK by 6.0%, 6.2%, and 4.1%, respectively. Finally, we show that PCK alone is insufficient to capture geometric quality and introduce new metrics and insights for more geometry-aware feature learning. Link to project page: https://reginehartwig.github.io/publications/geco/