EgoXtreme: A Dataset for Robust Object Pose Estimation in Egocentric Views under Extreme Conditions

TL;DR

EgoXtreme is a new large-scale dataset capturing egocentric views under extreme conditions like motion blur and poor lighting, revealing the limitations of current pose estimation methods and guiding future robust model development.

Contribution

The paper introduces EgoXtreme, a challenging egocentric 6D pose dataset with extreme scenarios, highlighting the failure of existing methods and the need for more robust solutions.

Findings

State-of-the-art estimators perform poorly under extreme conditions.

Image restoration techniques do not significantly improve performance.

Temporal information can enhance tracking in fast-motion scenarios.

Abstract

Smart glass is emerging as an useful device since it provides plenty of insights under hands-busy, eyes-on-task situations. To understand the context of the wearer, 6D object pose estimation in egocentric view is becoming essential. However, existing 6D object pose estimation benchmarks fail to capture the challenges of real-world egocentric applications, which are often dominated by severe motion blur, dynamic illumination, and visual obstructions. This discrepancy creates a significant gap between controlled lab data and chaotic real-world application. To bridge this gap, we introduce EgoXtreme, a new large-scale 6D pose estimation dataset captured entirely from an egocentric perspective. EgoXtreme features three challenging scenarios - industrial maintenance, sports, and emergency rescue - designed to introduce severe perceptual ambiguities through extreme lighting, heavy motion blur, and smoke. Evaluations of state-of-the-art generalizable pose estimators on EgoXtreme indicate that their generalization fails to hold in extreme conditions, especially under low light. We further demonstrate that simply applying image restoration (e.g., deblurring) offers no positive improvement for extreme conditions. While performance gain has appeared in tracking-based approach, implying using temporal information in fast-motion scenarios is meaningful. We conclude that EgoXtreme is an essential resource for developing and evaluating the next generation of pose estimation models robust enough for real-world egocentric vision. The dataset and code are available at https://taegyoun88.github.io/EgoXtreme/