Depth-SIMS: Semi-Parametric Image and Depth Synthesis
Valentina Musat, Daniele De Martini, Matthew Gadd, Paul Newman

TL;DR
Depth-SIMS introduces a semi-parametric synthesis approach that produces well-aligned RGB images, segmentation maps, and dense depth maps, improving data quality for training segmentation and depth tasks.
Contribution
The paper presents a novel compositing and in-painting method that enhances structural alignment and image quality in synthesized data for computer vision applications.
Findings
Achieves 3.7% higher mIoU than SOTA
Produces highly competitive FID scores
Generates data more effective for training segmentation and depth models
Abstract
In this paper we present a compositing image synthesis method that generates RGB canvases with well aligned segmentation maps and sparse depth maps, coupled with an in-painting network that transforms the RGB canvases into high quality RGB images and the sparse depth maps into pixel-wise dense depth maps. We benchmark our method in terms of structural alignment and image quality, showing an increase in mIoU over SOTA by 3.7 percentage points and a highly competitive FID. Furthermore, we analyse the quality of the generated data as training data for semantic segmentation and depth completion, and show that our approach is more suited for this purpose than other methods.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Computer Graphics and Visualization Techniques · Advanced Image and Video Retrieval Techniques
