Rational Polynomial Camera Model Warping for Deep Learning Based Satellite Multi-View Stereo Matching

TL;DR

This paper introduces a novel rational polynomial camera (RPC) warping module for deep learning-based satellite multi-view stereo, enabling accurate large-scale Earth surface reconstruction from satellite imagery.

Contribution

The paper presents the first rigorous RPC warping module and a satellite MVS framework tailored for the RPC model, along with a large-scale satellite image dataset.

Findings

RPC warping improves reconstruction accuracy over pin-hole methods

The SatMVS framework outperforms conventional MVS methods

The TLC SatMVS dataset enhances satellite image-based 3D reconstruction

Abstract

Satellite multi-view stereo (MVS) imagery is particularly suited for large-scale Earth surface reconstruction. Differing from the perspective camera model (pin-hole model) that is commonly used for close-range and aerial cameras, the cubic rational polynomial camera (RPC) model is the mainstream model for push-broom linear-array satellite cameras. However, the homography warping used in the prevailing learning based MVS methods is only applicable to pin-hole cameras. In order to apply the SOTA learning based MVS technology to the satellite MVS task for large-scale Earth surface reconstruction, RPC warping should be considered. In this work, we propose, for the first time, a rigorous RPC warping module. The rational polynomial coefficients are recorded as a tensor, and the RPC warping is formulated as a series of tensor transformations. Based on the RPC warping, we propose the deep learning based satellite MVS (SatMVS) framework for large-scale and wide depth range Earth surface reconstruction. We also introduce a large-scale satellite image dataset consisting of 519 5120${\times}$5120 images, which we call the TLC SatMVS dataset. The satellite images were acquired from a three-line camera (TLC) that catches triple-view images simultaneously, forming a valuable supplement to the existing open-source WorldView-3 datasets with single-scanline images. Experiments show that the proposed RPC warping module and the SatMVS framework can achieve a superior reconstruction accuracy compared to the pin-hole fitting method and conventional MVS methods. Code and data are available at https://github.com/WHU-GPCV/SatMVS.