ToosiCubix: Monocular 3D Cuboid Labeling via Vehicle Part Annotations

TL;DR

ToosiCubix is a practical monocular image-based method for annotating 3D vehicle cuboids using minimal user input, leveraging geometric constraints and priors to achieve accurate 3D annotations without expensive equipment.

Contribution

It introduces a novel approach for 3D vehicle annotation from monocular images with minimal user clicks, utilizing geometric optimization and probabilistic priors for high accuracy.

Findings

Achieves accurate 3D vehicle annotations with only 10 user clicks.

Validates effectiveness on KITTI and Cityscapes3D datasets.

Offers a cost-effective, scalable alternative to sensor-based methods.

Abstract

Many existing methods for 3D cuboid annotation of vehicles rely on expensive and carefully calibrated camera-LiDAR or stereo setups, limiting their accessibility for large-scale data collection. We introduce ToosiCubix, a simple yet powerful approach for annotating ground-truth cuboids using only monocular images and intrinsic camera parameters. Our method requires only about 10 user clicks per vehicle, making it highly practical for adding 3D annotations to existing datasets originally collected without specialized equipment. By annotating specific features (e.g., wheels, car badge, symmetries) across different vehicle parts, we accurately estimate each vehicle's position, orientation, and dimensions up to a scale ambiguity (8 DoF). The geometric constraints are formulated as an optimization problem, which we solve using a coordinate descent strategy, alternating between Perspective-n-Points (PnP) and least-squares subproblems. To handle common ambiguities such as scale and unobserved dimensions, we incorporate probabilistic size priors, enabling 9 DoF cuboid placements. We validate our annotations against the KITTI and Cityscapes3D datasets, demonstrating that our method offers a cost-effective and scalable solution for high-quality 3D cuboid annotation.