Revisiting Near/Remote Sensing with Geospatial Attention

TL;DR

This paper introduces geospatial attention, a geometry-aware mechanism that enhances near/remote sensing image segmentation by explicitly modeling the spatial relationship between ground-level and overhead images, leading to improved accuracy.

Contribution

The paper proposes a novel geospatial attention mechanism and architecture for near/remote sensing that explicitly incorporates geometric features, advancing the state-of-the-art in segmentation tasks.

Findings

Significant accuracy improvements over previous methods.

Effective use of geometric features in attention mechanism.

Versatility demonstrated across five segmentation tasks.

Abstract

This work addresses the task of overhead image segmentation when auxiliary ground-level images are available. Recent work has shown that performing joint inference over these two modalities, often called near/remote sensing, can yield significant accuracy improvements. Extending this line of work, we introduce the concept of geospatial attention, a geometry-aware attention mechanism that explicitly considers the geospatial relationship between the pixels in a ground-level image and a geographic location. We propose an approach for computing geospatial attention that incorporates geometric features and the appearance of the overhead and ground-level imagery. We introduce a novel architecture for near/remote sensing that is based on geospatial attention and demonstrate its use for five segmentation tasks. The results demonstrate that our method significantly outperforms the previous state-of-the-art methods.