Local Consensus Enhanced Siamese Network with Reciprocal Loss for Two-view Correspondence Learning

TL;DR

This paper introduces a Local Feature Consensus plugin and a reciprocal loss for Siamese networks, significantly improving two-view correspondence learning by enhancing feature discrimination and mutual supervision.

Contribution

The paper proposes a novel Local Feature Consensus plugin and reciprocal loss for Siamese networks, advancing two-view correspondence learning with better feature augmentation and mutual supervision.

Findings

Achieved state-of-the-art performance on benchmark datasets.

Enhanced inlier/outlier classification accuracy.

Improved feature discriminability through consensus and reciprocal supervision.

Abstract

Recent studies of two-view correspondence learning usually establish an end-to-end network to jointly predict correspondence reliability and relative pose. We improve such a framework from two aspects. First, we propose a Local Feature Consensus (LFC) plugin block to augment the features of existing models. Given a correspondence feature, the block augments its neighboring features with mutual neighborhood consensus and aggregates them to produce an enhanced feature. As inliers obey a uniform cross-view transformation and share more consistent learned features than outliers, feature consensus strengthens inlier correlation and suppresses outlier distraction, which makes output features more discriminative for classifying inliers/outliers. Second, existing approaches supervise network training with the ground truth correspondences and essential matrix projecting one image to the other for an input image pair, without considering the information from the reverse mapping. We extend existing models to a Siamese network with a reciprocal loss that exploits the supervision of mutual projection, which considerably promotes the matching performance without introducing additional model parameters. Building upon MSA-Net, we implement the two proposals and experimentally achieve state-of-the-art performance on benchmark datasets.