DAViD: Data-efficient and Accurate Vision Models from Synthetic Data

TL;DR

This paper shows that high-accuracy human-centric vision models can be trained efficiently on synthetic data, reducing costs and addressing fairness, while maintaining performance across multiple dense prediction tasks.

Contribution

It introduces a method to train accurate vision models using smaller, high-fidelity synthetic datasets, improving efficiency and fairness compared to traditional large-scale approaches.

Findings

Models trained on synthetic data match real data accuracy

Significant reduction in training and inference costs

Enhanced control over data diversity and fairness

Abstract

The state of the art in human-centric computer vision achieves high accuracy and robustness across a diverse range of tasks. The most effective models in this domain have billions of parameters, thus requiring extremely large datasets, expensive training regimes, and compute-intensive inference. In this paper, we demonstrate that it is possible to train models on much smaller but high-fidelity synthetic datasets, with no loss in accuracy and higher efficiency. Using synthetic training data provides us with excellent levels of detail and perfect labels, while providing strong guarantees for data provenance, usage rights, and user consent. Procedural data synthesis also provides us with explicit control on data diversity, that we can use to address unfairness in the models we train. Extensive quantitative assessment on real input images demonstrates accuracy of our models on three dense prediction tasks: depth estimation, surface normal estimation, and soft foreground segmentation. Our models require only a fraction of the cost of training and inference when compared with foundational models of similar accuracy. Our human-centric synthetic dataset and trained models are available at https://aka.ms/DAViD.