Interactive Occlusion Boundary Estimation through Exploitation of Synthetic Data

TL;DR

This paper introduces a novel deep-learning framework for interactive occlusion boundary estimation that leverages synthetic data and multi-scribble interaction, significantly improving scene understanding and benchmark creation.

Contribution

The paper presents MS extsuperscript{3}PE, a new multi-scribble-guided network, and Mesh2OB, an automated synthetic data generation tool, advancing occlusion boundary estimation methods.

Findings

MS extsuperscript{3}PE outperforms state-of-the-art interactive segmentation methods.

Synthetic data training with Mesh2OB enables effective occlusion boundary estimation.

The OB-LIGM benchmark provides high-quality real-world annotations for evaluation.

Abstract

Occlusion boundaries (OBs) geometrically localize occlusion events in 2D images and provide critical cues for scene understanding. In this paper, we present the first systematic study of Interactive Occlusion Boundary Estimation (IOBE), introducing MS\textsuperscript{3}PE, a novel multi-scribble-guided deep-learning framework that advances IOBE through two key innovations: (1) an intuitive multi-scribble interaction mechanism, and (2) a 3-encoding-path network enhanced with multi-scale strip convolutions. Our MS\textsuperscript{3}PE surpasses adapted baselines from seven state-of-the-art interactive segmentation methods, and demonstrates strong potential for OB benchmark construction through our real-user experiment. Besides, to address the scarcity of well-annotated real-world data, we propose using synthetic data for training IOBE models, and developed Mesh2OB, the first automated tool for generating precise ground-truth OBs from 3D scenes with self-occlusions explicitly handled, enabling creation of the OB-FUTURE synthetic benchmark that facilitates generalizable training without domain adaptation. Finally, we introduce OB-LIGM, a high-quality real-world benchmark comprising 120 meticulously annotated high-resolution images advancing evaluation standards in OB research. Source code and resources are available at https://github.com/xul-ops/IOBE.