Inter3D: A Benchmark and Strong Baseline for Human-Interactive 3D Object Reconstruction

TL;DR

Inter3D introduces a new benchmark and a baseline method for reconstructing human-interactive 3D objects with multiple movable parts, addressing the challenge of unseen part combinations during training.

Contribution

The paper presents a novel benchmark dataset, a new evaluation pipeline, and a strong baseline approach leveraging Space Discrepancy Tensors and regularization for efficient multi-part object reconstruction.

Findings

Our approach outperforms existing methods on the benchmark.

The proposed regularization improves spatial density consistency.

Occupancy grid sampling enhances training efficiency.

Abstract

Recent advancements in implicit 3D reconstruction methods, e.g., neural rendering fields and Gaussian splatting, have primarily focused on novel view synthesis of static or dynamic objects with continuous motion states. However, these approaches struggle to efficiently model a human-interactive object with n movable parts, requiring 2^n separate models to represent all discrete states. To overcome this limitation, we propose Inter3D, a new benchmark and approach for novel state synthesis of human-interactive objects. We introduce a self-collected dataset featuring commonly encountered interactive objects and a new evaluation pipeline, where only individual part states are observed during training, while part combination states remain unseen. We also propose a strong baseline approach that leverages Space Discrepancy Tensors to efficiently modelling all states of an object. To alleviate the impractical constraints on camera trajectories across training states, we propose a Mutual State Regularization mechanism to enhance the spatial density consistency of movable parts. In addition, we explore two occupancy grid sampling strategies to facilitate training efficiency. We conduct extensive experiments on the proposed benchmark, showcasing the challenges of the task and the superiority of our approach.