TranSplat: Surface Embedding-guided 3D Gaussian Splatting for Transparent Object Manipulation

TL;DR

TranSplat introduces a novel surface embedding-guided 3D Gaussian Splatting technique that significantly improves depth reconstruction for transparent objects in robotics, overcoming limitations of traditional sensors and methods.

Contribution

The paper presents TranSplat, a new method combining surface embeddings from a latent diffusion model with 3D Gaussian splatting for transparent object manipulation.

Findings

Achieves accurate, dense depth completion on synthetic and real-world benchmarks.

Demonstrates robustness to viewpoint and lighting changes.

Improves robot grasping performance with transparent objects.

Abstract

Transparent object manipulation remains a significant challenge in robotics due to the difficulty of acquiring accurate and dense depth measurements. Conventional depth sensors often fail with transparent objects, resulting in incomplete or erroneous depth data. Existing depth completion methods struggle with interframe consistency and incorrectly model transparent objects as Lambertian surfaces, leading to poor depth reconstruction. To address these challenges, we propose TranSplat, a surface embedding-guided 3D Gaussian Splatting method tailored for transparent objects. TranSplat uses a latent diffusion model to generate surface embeddings that provide consistent and continuous representations, making it robust to changes in viewpoint and lighting. By integrating these surface embeddings with input RGB images, TranSplat effectively captures the complexities of transparent surfaces, enhancing the splatting of 3D Gaussians and improving depth completion. Evaluations on synthetic and real-world transparent object benchmarks, as well as robot grasping tasks, show that TranSplat achieves accurate and dense depth completion, demonstrating its effectiveness in practical applications. We open-source synthetic dataset and model: https://github. com/jeongyun0609/TranSplat