Data Fusion for Multi-Task Learning of Building Extraction and Height Estimation

TL;DR

This paper explores a multi-task learning approach using optical and radar satellite imagery for building extraction and height estimation, but implements the tasks separately under constraints, achieving improved baseline results.

Contribution

It introduces a constrained multi-task learning framework for building extraction and height estimation from satellite imagery, with experimental validation showing significant improvements.

Findings

Baseline results for building extraction improved

Baseline results for height estimation improved

Experimental results demonstrate effectiveness of the approach

Abstract

In accordance with the urban reconstruction problem proposed by the DFC23 Track 2 Contest, this paper attempts a multitask-learning method of building extraction and height estimation using both optical and radar satellite imagery. Contrary to the initial goal of multitask learning which could potentially give a superior solution by reusing features and forming implicit constraints between multiple tasks, this paper reports the individual implementation of the building extraction and height estimation under constraints. The baseline results for the building extraction and the height estimation significantly increased after designed experiments.