Classifying geospatial objects from multiview aerial imagery using semantic meshes

TL;DR

This paper introduces a multiview aerial imagery classification method using semantic meshes, improving accuracy in geospatial object classification by leveraging multiple viewpoints and raw images instead of traditional orthomosaics.

Contribution

The paper presents a novel multiview classification approach with semantic meshes, enabling better use of raw aerial images and vertical object information, outperforming traditional orthomosaic-based methods.

Findings

Classification accuracy improved from 53% to 75%.

Method demonstrated on a new forest site benchmark dataset.

Open-source toolkit released for broader use.

Abstract

Aerial imagery is increasingly used in Earth science and natural resource management as a complement to labor-intensive ground-based surveys. Aerial systems can collect overlapping images that provide multiple views of each location from different perspectives. However, most prediction approaches (e.g. for tree species classification) use a single, synthesized top-down "orthomosaic" image as input that contains little to no information about the vertical aspects of objects and may include processing artifacts. We propose an alternate approach that generates predictions directly on the raw images and accurately maps these predictions into geospatial coordinates using semantic meshes. This method$\unicode{x2013}$released as a user-friendly open-source toolkit$\unicode{x2013}$enables analysts to use the highest quality data for predictions, capture information about the sides of objects, and leverage multiple viewpoints of each location for added robustness. We demonstrate the value of this approach on a new benchmark dataset of four forest sites in the western U.S. that consists of drone images, photogrammetry results, predicted tree locations, and species classification data derived from manual surveys. We show that our proposed multiview method improves classification accuracy from 53% to 75% relative to an orthomosaic baseline on a challenging cross-site tree species classification task.