EgoCOL: Egocentric Camera pose estimation for Open-world 3D object Localization @Ego4D challenge 2023

TL;DR

EgoCOL is a novel egocentric camera pose estimation method that improves 3D object localization accuracy in open-world scenarios by leveraging sparse reconstructions, achieving significant performance gains on the Ego4D benchmark.

Contribution

The paper introduces EgoCOL, a new approach that independently estimates camera poses from video and scan data, enhancing 3D object localization in egocentric videos.

Findings

EgoCOL outperforms the Ego4D baseline by 62% in pose estimation recall.

EgoCOL achieves 59% higher accuracy on the test set.

The method demonstrates high recall and precision in open-world 3D localization.

Abstract

We present EgoCOL, an egocentric camera pose estimation method for open-world 3D object localization. Our method leverages sparse camera pose reconstructions in a two-fold manner, video and scan independently, to estimate the camera pose of egocentric frames in 3D renders with high recall and precision. We extensively evaluate our method on the Visual Query (VQ) 3D object localization Ego4D benchmark. EgoCOL can estimate 62% and 59% more camera poses than the Ego4D baseline in the Ego4D Visual Queries 3D Localization challenge at CVPR 2023 in the val and test sets, respectively. Our code is publicly available at https://github.com/BCV-Uniandes/EgoCOL