Effectiveness of State-of-the-Art Super Resolution Algorithms in Surveillance Environment

TL;DR

This study evaluates and compares traditional and deep learning super-resolution algorithms in surveillance scenarios, identifying a CNN-based method with an external dictionary as the most effective for enhancing face detection accuracy.

Contribution

It provides a comprehensive performance comparison of seven SR algorithms in surveillance environments, highlighting the superiority of a CNN-based approach with external dictionary.

Findings

CNN-based SR with external dictionary outperforms others in face detection accuracy.

Deep learning SR algorithms generally yield better results than conventional methods.

The best algorithm achieved robust performance across diverse surveillance conditions.

Abstract

Image Super Resolution (SR) finds applications in areas where images need to be closely inspected by the observer to extract enhanced information. One such focused application is an offline forensic analysis of surveillance feeds. Due to the limitations of camera hardware, camera pose, limited bandwidth, varying illumination conditions, and occlusions, the quality of the surveillance feed is significantly degraded at times, thereby compromising monitoring of behavior, activities, and other sporadic information in the scene. For the proposed research work, we have inspected the effectiveness of four conventional yet effective SR algorithms and three deep learning-based SR algorithms to seek the finest method that executes well in a surveillance environment with limited training data op-tions. These algorithms generate an enhanced resolution output image from a sin-gle low-resolution (LR) input image. For performance analysis, a subset of 220 images from six surveillance datasets has been used, consisting of individuals with varying distances from the camera, changing illumination conditions, and complex backgrounds. The performance of these algorithms has been evaluated and compared using both qualitative and quantitative metrics. These SR algo-rithms have also been compared based on face detection accuracy. By analyzing and comparing the performance of all the algorithms, a Convolutional Neural Network (CNN) based SR technique using an external dictionary proved to be best by achieving robust face detection accuracy and scoring optimal quantitative metric results under different surveillance conditions. This is because the CNN layers progressively learn more complex features using an external dictionary.