Aerial Images Meet Crowdsourced Trajectories: A New Approach to Robust Road Extraction

TL;DR

This paper introduces CMMPNet, a neural network that effectively combines aerial images and crowdsourced trajectories to improve the accuracy and robustness of automatic road extraction from remote sensing data.

Contribution

The paper presents a novel dual-encoder neural network with a cross-modal refinement module for integrating multimodal data in road extraction tasks.

Findings

Outperforms state-of-the-art methods by large margins

Effective in blending image and trajectory data for robust road detection

Demonstrates success on three real-world benchmarks

Abstract

Land remote sensing analysis is a crucial research in earth science. In this work, we focus on a challenging task of land analysis, i.e., automatic extraction of traffic roads from remote sensing data, which has widespread applications in urban development and expansion estimation. Nevertheless, conventional methods either only utilized the limited information of aerial images, or simply fused multimodal information (e.g., vehicle trajectories), thus cannot well recognize unconstrained roads. To facilitate this problem, we introduce a novel neural network framework termed Cross-Modal Message Propagation Network (CMMPNet), which fully benefits the complementary different modal data (i.e., aerial images and crowdsourced trajectories). Specifically, CMMPNet is composed of two deep Auto-Encoders for modality-specific representation learning and a tailor-designed Dual Enhancement Module for cross-modal representation refinement. In particular, the complementary information of each modality is comprehensively extracted and dynamically propagated to enhance the representation of another modality. Extensive experiments on three real-world benchmarks demonstrate the effectiveness of our CMMPNet for robust road extraction benefiting from blending different modal data, either using image and trajectory data or image and Lidar data. From the experimental results, we observe that the proposed approach outperforms current state-of-the-art methods by large margins.Our source code is resealed on the project page http://lingboliu.com/multimodal_road_extraction.html.