A Graph-Matching Approach for Cross-view Registration of Over-view 2 and Street-view based Point Clouds

TL;DR

This paper presents a fully automated geo-registration method that aligns over-view and street-view point clouds using graph-matching of object boundaries, enabling accurate cross-view registration for geospatial data.

Contribution

The method introduces a novel graph-matching framework utilizing belief propagation for non-rigid registration of multi-view point clouds based on semantic object boundaries.

Findings

Achieves robust cross-view registration of satellite and street-view point clouds.

Utilizes belief propagation for effective graph-matching of building segments.

Provides precise alignment with constrained bundle adjustment.

Abstract

In this paper, based on the assumption that the object boundaries (e.g., buildings) from the over-view data should coincide with footprints of fa\c{c}ade 3D points generated from street-view photogrammetric images, we aim to address this problem by proposing a fully automated geo-registration method for cross-view data, which utilizes semantically segmented object boundaries as view-invariant features under a global optimization framework through graph-matching: taking the over-view point clouds generated from stereo/multi-stereo satellite images and the street-view point clouds generated from monocular video images as the inputs, the proposed method models segments of buildings as nodes of graphs, both detected from the satellite-based and street-view based point clouds, thus to form the registration as a graph-matching problem to allow non-rigid matches; to enable a robust solution and fully utilize the topological relations between these segments, we propose to address the graph-matching problem on its conjugate graph solved through a belief-propagation algorithm. The matched nodes will be subject to a further optimization to allow precise-registration, followed by a constrained bundle adjustment on the street-view image to keep 2D29 3D consistencies, which yields well-registered street-view images and point clouds to the satellite point clouds.