A Multi-Task Deep Learning Framework for Building Footprint Segmentation

TL;DR

This paper introduces a multi-task deep learning framework for building footprint segmentation in remote sensing images, improving accuracy by jointly optimizing segmentation, boundary detection, and image reconstruction tasks.

Contribution

It presents a novel multi-task learning model with auxiliary tasks and learnable loss weights, enhancing segmentation performance over single-task approaches.

Findings

Significant accuracy improvement over single-task models

Effective joint optimization of multiple related tasks

Robust performance on SpaceNet6 dataset

Abstract

The task of building footprint segmentation has been well-studied in the context of remote sensing (RS) as it provides valuable information in many aspects, however, difficulties brought by the nature of RS images such as variations in the spatial arrangements and in-consistent constructional patterns require studying further, since it often causes poorly classified segmentation maps. We address this need by designing a joint optimization scheme for the task of building footprint delineation and introducing two auxiliary tasks; image reconstruction and building footprint boundary segmentation with the intent to reveal the common underlying structure to advance the classification accuracy of a single task model under the favor of auxiliary tasks. In particular, we propose a deep multi-task learning (MTL) based unified fully convolutional framework which operates in an end-to-end manner by making use of joint loss function with learnable loss weights considering the homoscedastic uncertainty of each task loss. Experimental results conducted on the SpaceNet6 dataset demonstrate the potential of the proposed MTL framework as it improves the classification accuracy considerably compared to single-task and lesser compounded tasks.