KRONC: Keypoint-based Robust Camera Optimization for 3D Car Reconstruction

TL;DR

KRONC is a new method for estimating camera poses in 3D car reconstruction that uses semantic keypoints and requires less computation than traditional Structure-from-Motion techniques.

Contribution

KRONC introduces a lightweight optimization approach leveraging semantic keypoints for robust camera pose estimation in vehicle scenes.

Findings

KRONC achieves accurate camera pose estimates from coarse initializations.

The method offers comparable results to traditional SfM with significantly reduced computational cost.

Experiments on real-world car datasets validate the effectiveness of KRONC.

Abstract

The three-dimensional representation of objects or scenes starting from a set of images has been a widely discussed topic for years and has gained additional attention after the diffusion of NeRF-based approaches. However, an underestimated prerequisite is the knowledge of camera poses or, more specifically, the estimation of the extrinsic calibration parameters. Although excellent general-purpose Structure-from-Motion methods are available as a pre-processing step, their computational load is high and they require a lot of frames to guarantee sufficient overlapping among the views. This paper introduces KRONC, a novel approach aimed at inferring view poses by leveraging prior knowledge about the object to reconstruct and its representation through semantic keypoints. With a focus on vehicle scenes, KRONC is able to estimate the position of the views as a solution to a light optimization problem targeting the convergence of keypoints' back-projections to a singular point. To validate the method, a specific dataset of real-world car scenes has been collected. Experiments confirm KRONC's ability to generate excellent estimates of camera poses starting from very coarse initialization. Results are comparable with Structure-from-Motion methods with huge savings in computation. Code and data will be made publicly available.