Object-Compositional Neural Implicit Surfaces

TL;DR

ObjectSDF introduces a novel object-compositional neural implicit representation that explicitly models individual objects using SDFs and semantic information, leading to high-fidelity 3D reconstruction and better object differentiation.

Contribution

The paper proposes ObjectSDF, a new framework that unifies scene and object representations by linking SDFs with semantic labels for improved 3D modeling.

Findings

Outperforms existing methods in 3D scene and object reconstruction

Effectively models individual object geometry and semantics

Demonstrates high fidelity in complex scene reconstructions

Abstract

The neural implicit representation has shown its effectiveness in novel view synthesis and high-quality 3D reconstruction from multi-view images. However, most approaches focus on holistic scene representation yet ignore individual objects inside it, thus limiting potential downstream applications. In order to learn object-compositional representation, a few works incorporate the 2D semantic map as a cue in training to grasp the difference between objects. But they neglect the strong connections between object geometry and instance semantic information, which leads to inaccurate modeling of individual instance. This paper proposes a novel framework, ObjectSDF, to build an object-compositional neural implicit representation with high fidelity in 3D reconstruction and object representation. Observing the ambiguity of conventional volume rendering pipelines, we model the scene by combining the Signed Distance Functions (SDF) of individual object to exert explicit surface constraint. The key in distinguishing different instances is to revisit the strong association between an individual object's SDF and semantic label. Particularly, we convert the semantic information to a function of object SDF and develop a unified and compact representation for scene and objects. Experimental results show the superiority of ObjectSDF framework in representing both the holistic object-compositional scene and the individual instances. Code can be found at https://qianyiwu.github.io/objectsdf/