A Dataset-Dispersion Perspective on Reconstruction Versus Recognition in Single-View 3D Reconstruction Networks

TL;DR

This paper investigates how the dispersion of training data influences neural networks' tendency to favor recognition over shape reconstruction in single-view 3D reconstruction, introducing a new dispersion score metric.

Contribution

It introduces the dispersion score, a novel data-driven metric, to analyze the impact of data dispersion on the recognition versus reconstruction bias in neural networks.

Findings

Higher data dispersion leads to increased recognition bias.

The dispersion score effectively predicts reconstruction quality.

The metric provides new insights beyond traditional reconstruction scores.

Abstract

Neural networks (NN) for single-view 3D reconstruction (SVR) have gained in popularity. Recent work points out that for SVR, most cutting-edge NNs have limited performance on reconstructing unseen objects because they rely primarily on recognition (i.e., classification-based methods) rather than shape reconstruction. To understand this issue in depth, we provide a systematic study on when and why NNs prefer recognition to reconstruction and vice versa. Our finding shows that a leading factor in determining recognition versus reconstruction is how dispersed the training data is. Thus, we introduce the dispersion score, a new data-driven metric, to quantify this leading factor and study its effect on NNs. We hypothesize that NNs are biased toward recognition when training images are more dispersed and training shapes are less dispersed. Our hypothesis is supported and the dispersion score is proved effective through our experiments on synthetic and benchmark datasets. We show that the proposed metric is a principal way to analyze reconstruction quality and provides novel information in addition to the conventional reconstruction score.