SeFENet: Robust Deep Homography Estimation via Semantic-Driven Feature Enhancement

TL;DR

SeFENet is a novel deep learning model that enhances feature robustness for homography estimation in harsh environments by leveraging semantic information and multi-scale feature aggregation.

Contribution

The paper introduces SeFENet, a semantic-driven feature enhancement network with a hierarchical scale-aware module and a meta-learning training strategy for improved robustness in adverse conditions.

Findings

Reduces point match error by at least 41% on large-scale datasets.

Outperforms state-of-the-art methods in both normal and harsh environments.

Effectively extracts features under diverse challenging conditions.

Abstract

Images captured in harsh environments often exhibit blurred details, reduced contrast, and color distortion, which hinder feature detection and matching, thereby affecting the accuracy and robustness of homography estimation. While visual enhancement can improve contrast and clarity, it may introduce visual-tolerant artifacts that obscure the structural integrity of images. Considering the resilience of semantic information against environmental interference, we propose a semantic-driven feature enhancement network for robust homography estimation, dubbed SeFENet. Concretely, we first introduce an innovative hierarchical scale-aware module to expand the receptive field by aggregating multi-scale information, thereby effectively extracting image features under diverse harsh conditions. Subsequently, we propose a semantic-guided constraint module combined with a high-level perceptual framework to achieve degradation-tolerant with semantic feature. A meta-learning-based training strategy is introduced to mitigate the disparity between semantic and structural features. By internal-external alternating optimization, the proposed network achieves implicit semantic-wise feature enhancement, thereby improving the robustness of homography estimation in adverse environments by strengthening the local feature comprehension and context information extraction. Experimental results under both normal and harsh conditions demonstrate that SeFENet significantly outperforms SOTA methods, reducing point match error by at least 41% on the large-scale datasets.