XDen-1K: A Density Field Dataset of Real-World Objects

TL;DR

XDen-1K is a large-scale, multi-modal dataset of 1,000 real-world objects with geometric and X-ray data, enabling improved physical property estimation and robotic manipulation through novel density recovery methods.

Contribution

The paper introduces XDen-1K, the first large-scale dataset combining geometric and X-ray data for physical property estimation, along with a new density recovery framework and applications in segmentation and robotics.

Findings

Density recovery improves volumetric density estimation accuracy.

X-ray conditioned segmentation enhances segmentation performance.

Leveraging the dataset boosts robotic manipulation success rates.

Abstract

A deep understanding of the physical world is a central goal for embodied AI and realistic simulation. While current models excel at capturing an object's surface geometry and appearance, they largely neglect its internal physical properties. This omission is critical, as properties like volumetric density are fundamental for predicting an object's center of mass, stability, and interaction dynamics in applications ranging from robotic manipulation to physical simulation. The primary bottleneck has been the absence of large-scale, real-world data. To bridge this gap, we introduce XDen-1K, the first large-scale, multi-modal dataset designed for real-world physical property estimation, with a particular focus on volumetric density. The core of this dataset consists of 1,000 real-world objects across 148 categories, for which we provide comprehensive multi-modal data, including a high-resolution 3D geometric model with part-level annotations and a corresponding set of real-world biplanar X-ray scans. Building upon this data, we introduce a novel optimization framework that recovers a high-fidelity volumetric density field of each object from its sparse X-ray views. To demonstrate its practical value, we add X-ray images as a conditioning signal to an existing segmentation network and perform volumetric segmentation. Furthermore, we conduct experiments on downstream robotics tasks. The results show that leveraging the dataset can effectively improve the accuracy of center-of-mass estimation and the success rate of robotic manipulation. We believe XDen-1K will serve as a foundational resource and a challenging new benchmark, catalyzing future research in physically grounded visual inference and embodied AI.