Consensus-Adaptive RANSAC

TL;DR

This paper introduces a novel RANSAC framework that uses an attention mechanism and transformer to adaptively explore the parameter space, significantly improving robust estimation performance across various datasets.

Contribution

It presents a new RANSAC approach that incorporates attention and transformer-based state updates to guide sampling and refinement, outperforming existing methods.

Findings

Outperforms state-of-the-art estimators on essential and fundamental matrix tasks.

Maintains good generalization across different datasets and tasks.

Adds minimal runtime overhead compared to traditional RANSAC.

Abstract

RANSAC and its variants are widely used for robust estimation, however, they commonly follow a greedy approach to finding the highest scoring model while ignoring other model hypotheses. In contrast, Iteratively Reweighted Least Squares (IRLS) techniques gradually approach the model by iteratively updating the weight of each correspondence based on the residuals from previous iterations. Inspired by these methods, we propose a new RANSAC framework that learns to explore the parameter space by considering the residuals seen so far via a novel attention layer. The attention mechanism operates on a batch of point-to-model residuals, and updates a per-point estimation state to take into account the consensus found through a lightweight one-step transformer. This rich state then guides the minimal sampling between iterations as well as the model refinement. We evaluate the proposed approach on essential and fundamental matrix estimation on a number of indoor and outdoor datasets. It outperforms state-of-the-art estimators by a significant margin adding only a small runtime overhead. Moreover, we demonstrate good generalization properties of our trained model, indicating its effectiveness across different datasets and tasks. The proposed attention mechanism and one-step transformer provide an adaptive behavior that enhances the performance of RANSAC, making it a more effective tool for robust estimation. Code is available at https://github.com/cavalli1234/CA-RANSAC.