Learning to Generate Chairs, Tables and Cars with Convolutional Networks

TL;DR

This paper presents convolutional neural networks trained to generate images of chairs, tables, and cars from 3D models, capable of interpolation, extrapolation, and creating novel objects, with applications in object correspondence.

Contribution

The work introduces a generative neural network approach that learns meaningful 3D representations, enabling view interpolation, extrapolation, and object recombination, surpassing existing methods in object correspondence tasks.

Findings

Networks generate realistic object images from 3D models.

Models interpolate and extrapolate views effectively.

Outperforms existing methods in object correspondence detection.

Abstract

We train generative 'up-convolutional' neural networks which are able to generate images of objects given object style, viewpoint, and color. We train the networks on rendered 3D models of chairs, tables, and cars. Our experiments show that the networks do not merely learn all images by heart, but rather find a meaningful representation of 3D models allowing them to assess the similarity of different models, interpolate between given views to generate the missing ones, extrapolate views, and invent new objects not present in the training set by recombining training instances, or even two different object classes. Moreover, we show that such generative networks can be used to find correspondences between different objects from the dataset, outperforming existing approaches on this task.