Multi-task learning from fixed-wing UAV images for 2D/3D city modeling

TL;DR

This paper presents a framework for evaluating multi-task learning methods applied to fixed-wing UAV images, aiming to improve 2D and 3D city modeling for urban management applications.

Contribution

It introduces a common framework for assessing multi-task learning techniques specifically for UAV-based urban scene understanding and city modeling.

Findings

Enhanced multi-task learning performance on UAV imagery

Improved accuracy in 2D/3D city models

Framework facilitates comparison of different methods

Abstract

Single-task learning in artificial neural networks will be able to learn the model very well, and the benefits brought by transferring knowledge thus become limited. In this regard, when the number of tasks increases (e.g., semantic segmentation, panoptic segmentation, monocular depth estimation, and 3D point cloud), duplicate information may exist across tasks, and the improvement becomes less significant. Multi-task learning has emerged as a solution to knowledge-transfer issues and is an approach to scene understanding which involves multiple related tasks each with potentially limited training data. Multi-task learning improves generalization by leveraging the domain-specific information contained in the training data of related tasks. In urban management applications such as infrastructure development, traffic monitoring, smart 3D cities, and change detection, automated multi-task data analysis for scene understanding based on the semantic, instance, and panoptic annotation, as well as monocular depth estimation, is required to generate precise urban models. In this study, a common framework for the performance assessment of multi-task learning methods from fixed-wing UAV images for 2D/3D city modeling is presented.