ClusterGNN: Cluster-based Coarse-to-Fine Graph Neural Network for Efficient Feature Matching

TL;DR

ClusterGNN introduces a cluster-based, coarse-to-fine graph neural network architecture that significantly reduces computational complexity and memory usage in visual feature matching, while maintaining competitive accuracy.

Contribution

The paper proposes ClusterGNN, a novel GNN architecture that operates on clusters and employs a progressive coarse-to-fine approach to improve efficiency in feature matching.

Findings

Reduces runtime by 59.7% compared to state-of-the-art methods.

Lowers memory consumption by 58.4% during dense detection.

Achieves competitive performance on various vision tasks.

Abstract

Graph Neural Networks (GNNs) with attention have been successfully applied for learning visual feature matching. However, current methods learn with complete graphs, resulting in a quadratic complexity in the number of features. Motivated by a prior observation that self- and cross- attention matrices converge to a sparse representation, we propose ClusterGNN, an attentional GNN architecture which operates on clusters for learning the feature matching task. Using a progressive clustering module we adaptively divide keypoints into different subgraphs to reduce redundant connectivity, and employ a coarse-to-fine paradigm for mitigating miss-classification within images. Our approach yields a 59.7% reduction in runtime and 58.4% reduction in memory consumption for dense detection, compared to current state-of-the-art GNN-based matching, while achieving a competitive performance on various computer vision tasks.