# 3D Scene Reconstruction with Multi-layer Depth and Epipolar Transformers

## TL;DR

This paper presents a novel end-to-end convolutional approach for 3D scene reconstruction from a single RGB image, utilizing multi-layer depth and epipolar transformers to improve accuracy and handle complex indoor scenes.

## Contribution

It introduces the Epipolar Feature Transformer and multi-layer depth representations, advancing scene reconstruction without relying on object detection or voxel limitations.

## Key findings

- Improved 3D reconstruction accuracy on indoor scenes
- Effective transfer of features across virtual viewpoints
- Avoids voxel resolution and memory constraints

## Abstract

We tackle the problem of automatically reconstructing a complete 3D model of a scene from a single RGB image. This challenging task requires inferring the shape of both visible and occluded surfaces. Our approach utilizes viewer-centered, multi-layer representation of scene geometry adapted from recent methods for single object shape completion. To improve the accuracy of view-centered representations for complex scenes, we introduce a novel "Epipolar Feature Transformer" that transfers convolutional network features from an input view to other virtual camera viewpoints, and thus better covers the 3D scene geometry. Unlike existing approaches that first detect and localize objects in 3D, and then infer object shape using category-specific models, our approach is fully convolutional, end-to-end differentiable, and avoids the resolution and memory limitations of voxel representations. We demonstrate the advantages of multi-layer depth representations and epipolar feature transformers on the reconstruction of a large database of indoor scenes.