# Late or Earlier Information Fusion from Depth and Spectral Data? Large-Scale Digital Surface Model Refinement by Hybrid-cGAN

## TL;DR

This paper introduces a Hybrid-cGAN approach for refining digital surface models by fusing spectral and height data early in the process, leading to improved detail, accuracy, and boundary rectilinearity in 3D building representations.

## Contribution

The paper proposes a novel Hybrid-cGAN architecture that fuses spectral and height information early, enhancing DSM refinement and 3D building detail reconstruction.

## Key findings

- Early data fusion improves fine detail propagation.
- Enhanced boundary rectilinearity in 3D models.
- Effective refinement of DSMs with spectral and height data.

## Abstract

We present the workflow of a DSM refinement methodology using a Hybrid-cGAN where the generative part consists of two encoders and a common decoder which blends the spectral and height information within one network. The inputs to the Hybrid-cGAN are single-channel photogrammetric DSMs with continuous values and single-channel pan-chromatic (PAN) half-meter resolution satellite images. Experimental results demonstrate that the earlier information fusion from data with different physical meanings helps to propagate fine details and complete an inaccurate or missing 3D information about building forms. Moreover, it improves the building boundaries making them more rectilinear.