Large-Scale 3D Scene Classification With Multi-View Volumetric CNN

TL;DR

This paper presents a multi-view volumetric CNN approach for large-scale 3D scene classification that leverages multi-angle projections to improve pixel-level categorization of complex materials in aerial imagery.

Contribution

The authors introduce a novel multi-view projection CNN method that simplifies training and enhances classification accuracy for complex scene elements without boundary labels.

Findings

Outperforms state-of-the-art image classification models like Inception-V3.

Effectively classifies water and tree regions in large aerial datasets.

Enables pixel-level classification with minimal context and simple training data.

Abstract

We introduce a method to classify imagery using a convo- lutional neural network (CNN) on multi-view image pro- jections. The power of our method comes from using pro- jections of multiple images at multiple depth planes near the reconstructed surface. This enables classification of categories whose salient aspect is appearance change un- der different viewpoints, such as water, trees, and other materials with complex reflection/light response proper- ties. Our method does not require boundary labelling in images and works on pixel-level classification with a small (few pixels) context, which simplifies the cre- ation of a training set. We demonstrate this application on large-scale aerial imagery collections, and extend the per-pixel classification to robustly create a consistent 2D classification which can be used to fill the gaps in non- reconstructible water regions. We also apply our method to classify tree regions. In both cases, the training data can quickly be generated using a small number of manually- created polygons on a map. We show that even with a very simple and standard network our CNN outperforms the state-of-the-art image classification, the Inception-V3 model retrained from a large collection of aerial images.