Fiducial Marker Splatting for High-Fidelity Robotics Simulations

TL;DR

This paper introduces a hybrid simulation framework that combines Gaussian Splatting with fiducial markers, improving visual realism and localization accuracy in complex, cluttered environments like greenhouses for robotic training.

Contribution

A novel algorithm for efficiently generating GS-based fiducial markers within cluttered scenes, enhancing simulation fidelity and pose estimation for robotics.

Findings

Outperforms traditional image-fitting in efficiency

Achieves higher pose-estimation accuracy

Demonstrates effectiveness in greenhouse simulations

Abstract

High-fidelity 3D simulation is critical for training mobile robots, but its traditional reliance on mesh-based representations often struggle in complex environments, such as densely packed greenhouses featuring occlusions and repetitive structures. Recent neural rendering methods, like Gaussian Splatting (GS), achieve remarkable visual realism but lack flexibility to incorporate fiducial markers, which are essential for robotic localization and control. We propose a hybrid framework that combines the photorealism of GS with structured marker representations. Our core contribution is a novel algorithm for efficiently generating GS-based fiducial markers (e.g., AprilTags) within cluttered scenes. Experiments show that our approach outperforms traditional image-fitting techniques in both efficiency and pose-estimation accuracy. We further demonstrate the framework's potential in a greenhouse simulation. This agricultural setting serves as a challenging testbed, as its combination of dense foliage, similar-looking elements, and occlusions pushes the limits of perception, thereby highlighting the framework's value for real-world applications.