SLAT-Phys: Fast Material Property Field Prediction from Structured 3D Latents

TL;DR

SLAT-Phys is a fast, end-to-end method that predicts detailed material properties of 3D objects directly from a single RGB image, leveraging pretrained 3D models to significantly speed up the process.

Contribution

It introduces a novel approach that uses spatially organized latent features from pretrained 3D models to estimate material properties without explicit 3D reconstruction.

Findings

Achieves competitive accuracy in material property prediction.

Reduces computation time by 120x compared to prior methods.

Requires only 9.9 seconds per object on a high-end GPU.

Abstract

Estimating the material property field of 3D assets is critical for physics-based simulation, robotics, and digital twin generation. Existing vision-based approaches are either too expensive and slow or rely on 3D information. We present SLAT-Phys, an end-to-end method that predicts spatially varying material property fields of 3D assets directly from a single RGB image without explicit 3D reconstruction. Our approach leverages spatially organised latent features from a pretrained 3D asset generation model that encodes rich geometry and semantic prior, and trains a lightweight neural decoder to estimate Young's modulus, density, and Poisson's ratio. The coarse volumetric layout and semantic cues of the latent representation about object geometry and appearance enable accurate material estimation. Our experiments demonstrate that our method provides competitive accuracy in predicting continuous material parameters when compared against prior approaches, while significantly reducing computation time. In particular, SLAT-Phys requires only 9.9 seconds per object on an NVIDIA RTXA5000 GPU and avoids reconstruction and voxelization preprocessing. This results in 120x speedup compared to prior methods and enables faster material property estimation from a single image.