On the usability of deep networks for object-based image analysis

TL;DR

This paper demonstrates the effectiveness of deep convolutional networks for object detection, segmentation, and classification of vehicles in aerial imagery, highlighting their suitability for object-oriented analysis in remote sensing.

Contribution

It shows how to apply Fully Convolutional Networks and CNNs to detect, segment, and classify vehicles in complex aerial datasets, advancing object-based remote sensing analysis.

Findings

Deep networks achieve precise vehicle segmentation in aerial images.

Semantic maps from FCNs facilitate analysis of vehicle distribution.

Knowledge transfer improves vehicle classification accuracy.

Abstract

As computer vision before, remote sensing has been radically changed by the introduction of Convolution Neural Networks. Land cover use, object detection and scene understanding in aerial images rely more and more on deep learning to achieve new state-of-the-art results. Recent architectures such as Fully Convolutional Networks (Long et al., 2015) can even produce pixel level annotations for semantic mapping. In this work, we show how to use such deep networks to detect, segment and classify different varieties of wheeled vehicles in aerial images from the ISPRS Potsdam dataset. This allows us to tackle object detection and classification on a complex dataset made up of visually similar classes, and to demonstrate the relevance of such a subclass modeling approach. Especially, we want to show that deep learning is also suitable for object-oriented analysis of Earth Observation data. First, we train a FCN variant on the ISPRS Potsdam dataset and show how the learnt semantic maps can be used to extract precise segmentation of vehicles, which allow us studying the repartition of vehicles in the city. Second, we train a CNN to perform vehicle classification on the VEDAI (Razakarivony and Jurie, 2016) dataset, and transfer its knowledge to classify candidate segmented vehicles on the Potsdam dataset.