CROVIA: Seeing Drone Scenes from Car Perspective via Cross-View Adaptation

TL;DR

This paper introduces CROVIA, a novel cross-view adaptation method that leverages geometry constraints and multi-modal networks to improve UAV scene segmentation by transferring knowledge from vehicle view datasets.

Contribution

The work proposes a new geometry-based constraint and a GeiCo loss for cross-view adaptation, enabling effective knowledge transfer without paired data.

Findings

Achieves state-of-the-art results on SYNTHIA to UAVID and GTA5 to UAVID benchmarks.

Introduces new cross-view adaptation benchmarks for UAV scene segmentation.

Demonstrates effective transfer of knowledge from vehicle to UAV perspectives.

Abstract

Understanding semantic scene segmentation of urban scenes captured from the Unmanned Aerial Vehicles (UAV) perspective plays a vital role in building a perception model for UAV. With the limitations of large-scale densely labeled data, semantic scene segmentation for UAV views requires a broad understanding of an object from both its top and side views. Adapting from well-annotated autonomous driving data to unlabeled UAV data is challenging due to the cross-view differences between the two data types. Our work proposes a novel Cross-View Adaptation (CROVIA) approach to effectively adapt the knowledge learned from on-road vehicle views to UAV views. First, a novel geometry-based constraint to cross-view adaptation is introduced based on the geometry correlation between views. Second, cross-view correlations from image space are effectively transferred to segmentation space without any requirement of paired on-road and UAV view data via a new Geometry-Constraint Cross-View (GeiCo) loss. Third, the multi-modal bijective networks are introduced to enforce the global structural modeling across views. Experimental results on new cross-view adaptation benchmarks introduced in this work, i.e., SYNTHIA to UAVID and GTA5 to UAVID, show the State-of-the-Art (SOTA) performance of our approach over prior adaptation methods