Novel 3D Scene Understanding Applications From Recurrence in a Single Image

TL;DR

This paper shows how discovering recurring patterns in a single image can improve 3D scene understanding tasks like vanishing point detection, symmetry hypothesis, and scene description, rivaling or surpassing existing methods.

Contribution

It introduces a novel approach leveraging recurrence in a single image for comprehensive 3D scene understanding and provides a new benchmark for evaluation.

Findings

Recurrence-based methods achieve comparable or better results than supervised approaches.

The approach enables detailed scene descriptions from a single image.

A new 1K+ Recurring Pattern benchmark is introduced for evaluation.

Abstract

We demonstrate the utility of recurring pattern discovery from a single image for spatial understanding of a 3D scene in terms of (1) vanishing point detection, (2) hypothesizing 3D translation symmetry and (3) counting the number of RP instances in the image. Furthermore, we illustrate the feasibility of leveraging RP discovery output to form a more precise, quantitative text description of the scene. Our quantitative evaluations on a new 1K+ Recurring Pattern (RP) benchmark with diverse variations show that visual perception of recurrence from one single view leads to scene understanding outcomes that are as good as or better than existing supervised methods and/or unsupervised methods that use millions of images.