# DepthSynth: Real-Time Realistic Synthetic Data Generation from CAD Models for 2.5D Recognition

## TL;DR

DepthSynth is a comprehensive framework that generates highly realistic synthetic depth data from 3D CAD models, improving training for recognition tasks by closely mimicking real sensor outputs.

## Contribution

It introduces an end-to-end simulation pipeline that models sensor noise and surface properties, producing more realistic depth data than previous methods.

## Key findings

- Enhanced realism of synthetic depth data verified through extensive evaluation.
- Improved neural network recognition performance when trained on DepthSynth data.
- Broader sensor coverage compared to prior approaches.

## Abstract

Recent progress in computer vision has been dominated by deep neural networks trained over large amounts of labeled data. Collecting such datasets is however a tedious, often impossible task; hence a surge in approaches relying solely on synthetic data for their training. For depth images however, discrepancies with real scans still noticeably affect the end performance. We thus propose an end-to-end framework which simulates the whole mechanism of these devices, generating realistic depth data from 3D models by comprehensively modeling vital factors e.g. sensor noise, material reflectance, surface geometry. Not only does our solution cover a wider range of sensors and achieve more realistic results than previous methods, assessed through extended evaluation, but we go further by measuring the impact on the training of neural networks for various recognition tasks; demonstrating how our pipeline seamlessly integrates such architectures and consistently enhances their performance.