Hole-robust Wireframe Detection

TL;DR

This paper introduces a novel wireframe detection method that is robust to occlusions and holes, utilizing conditional data generation, GANs, and pseudo labeling to better understand scene structure in man-made environments.

Contribution

It is the first to incorporate hole-aware training and GANs into wireframe detection, significantly improving robustness to occlusions and limited labeled data.

Findings

Outperforms previous hole-agnostic wireframe detection models.

Effectively handles occlusions and holes in scene images.

Enhances detection accuracy with pseudo labeling.

Abstract

"Wireframe" is a line segment based representation designed to well capture large-scale visual properties of regular, structural shaped man-made scenes surrounding us. Unlike the wireframes, conventional edges or line segments focus on all visible edges and lines without particularly distinguishing which of them are more salient to man-made structural information. Existing wireframe detection models rely on supervising the annotated data but do not explicitly pay attention to understand how to compose the structural shapes of the scene. In addition, we often face that many foreground objects occluding the background scene interfere with proper inference of the full scene structure behind them. To resolve these problems, we first time in the field, propose new conditional data generation and training that help the model understand how to ignore occlusion indicated by holes, such as foreground object regions masked out on the image. In addition, we first time combine GAN in the model to let the model better predict underlying scene structure even beyond large holes. We also introduce pseudo labeling to further enlarge the model capacity to overcome small-scale labeled data. We show qualitatively and quantitatively that our approach significantly outperforms previous works unable to handle holes, as well as improves ordinary detection without holes given.