Adapting Depth Anything to Adverse Imaging Conditions with Events

TL;DR

This paper introduces ADAE, a novel event-guided fusion framework that enhances depth estimation in adverse lighting and motion conditions by adaptively merging frame and event data, improving robustness of foundation models.

Contribution

The paper proposes a new fusion framework that leverages event data to improve depth estimation under challenging conditions, extending foundation models' robustness without retraining from scratch.

Findings

ADAEs outperform existing methods in degraded scenes.

The entropy-aware fusion effectively handles illumination variations.

Motion cues improve depth accuracy in blurred regions.

Abstract

Robust depth estimation under dynamic and adverse lighting conditions is essential for robotic systems. Currently, depth foundation models, such as Depth Anything, achieve great success in ideal scenes but remain challenging under adverse imaging conditions such as extreme illumination and motion blur. These degradations corrupt the visual signals of frame cameras, weakening the discriminative features of frame-based depths across the spatial and temporal dimensions. Typically, existing approaches incorporate event cameras to leverage their high dynamic range and temporal resolution, aiming to compensate for corrupted frame features. However, such specialized fusion models are predominantly trained from scratch on domain-specific datasets, thereby failing to inherit the open-world knowledge and robust generalization inherent to foundation models. In this work, we propose ADAE, an event-guided spatiotemporal fusion framework for Depth Anything in degraded scenes. Our design is guided by two key insights: 1) Entropy-Aware Spatial Fusion. We adaptively merge frame-based and event-based features using an information entropy strategy to indicate illumination-induced degradation. 2) Motion-Guided Temporal Correction. We resort to the event-based motion cue to recalibrate ambiguous features in blurred regions. Under our unified framework, the two components are complementary to each other and jointly enhance Depth Anything under adverse imaging conditions. Extensive experiments have been performed to verify the superiority of the proposed method. Our code will be released upon acceptance.