Multi-Spectral Image Stitching via Spatial Graph Reasoning

TL;DR

This paper introduces a novel multi-spectral image stitching method using spatial graph reasoning with GCNs, effectively aligning and integrating infrared and visible images to produce wide FOV scenes, supported by a new challenging dataset.

Contribution

The paper proposes a spatial graph reasoning approach with GCNs for multi-spectral image stitching, incorporating multi-scale features and long-range coherence for improved alignment.

Findings

Outperforms existing methods on benchmark datasets.

Effectively handles large parallax and multi-view variations.

Provides a new dataset for multi-spectral image stitching evaluation.

Abstract

Multi-spectral image stitching leverages the complementarity between infrared and visible images to generate a robust and reliable wide field-of-view (FOV) scene. The primary challenge of this task is to explore the relations between multi-spectral images for aligning and integrating multi-view scenes. Capitalizing on the strengths of Graph Convolutional Networks (GCNs) in modeling feature relationships, we propose a spatial graph reasoning based multi-spectral image stitching method that effectively distills the deformation and integration of multi-spectral images across different viewpoints. To accomplish this, we embed multi-scale complementary features from the same view position into a set of nodes. The correspondence across different views is learned through powerful dense feature embeddings, where both inter- and intra-correlations are developed to exploit cross-view matching and enhance inner feature disparity. By introducing long-range coherence along spatial and channel dimensions, the complementarity of pixel relations and channel interdependencies aids in the reconstruction of aligned multi-view features, generating informative and reliable wide FOV scenes. Moreover, we release a challenging dataset named ChaMS, comprising both real-world and synthetic sets with significant parallax, providing a new option for comprehensive evaluation. Extensive experiments demonstrate that our method surpasses the state-of-the-arts.