Sky pixel detection in outdoor imagery using an adaptive algorithm and machine learning

TL;DR

This paper introduces an adaptive sky pixel detection system for outdoor images that combines segmentation, clustering, and neural network selection to improve accuracy across diverse conditions.

Contribution

The novel contribution is an adaptive algorithm that selects the best detection technique for each image using a trained neural network, enhancing accuracy over existing methods.

Findings

Achieved 82% accuracy with the neural network classifier.

Adaptive process significantly outperforms single-technique methods.

System performs better than existing published techniques.

Abstract

Computer vision techniques enable automated detection of sky pixels in outdoor imagery. In urban climate, sky detection is an important first step in gathering information about urban morphology and sky view factors. However, obtaining accurate results remains challenging and becomes even more complex using imagery captured under a variety of lighting and weather conditions. To address this problem, we present a new sky pixel detection system demonstrated to produce accurate results using a wide range of outdoor imagery types. Images are processed using a selection of mean-shift segmentation, K-means clustering, and Sobel filters to mark sky pixels in the scene. The algorithm for a specific image is chosen by a convolutional neural network, trained with 25,000 images from the Skyfinder data set, reaching 82% accuracy for the top three classes. This selection step allows the sky marking to follow an adaptive process and to use different techniques and parameters to best suit a particular image. An evaluation of fourteen different techniques and parameter sets shows that no single technique can perform with high accuracy across varied Skyfinder and Google Street View data sets. However, by using our adaptive process, large increases in accuracy are observed. The resulting system is shown to perform better than other published techniques.