APT: Adaptive Perceptual quality based camera Tuning using reinforcement learning

TL;DR

This paper introduces APT, a reinforcement learning system that dynamically tunes camera parameters over 5G to maintain high-quality video capture, thereby improving the accuracy of various video analytics in changing environments.

Contribution

The paper presents a novel reinforcement learning-based approach for real-time, remote camera parameter tuning to sustain optimal video quality and analytics accuracy amidst environmental changes.

Findings

Object detection accuracy improved by ~42% with APT.

APT maintains high video quality across different environmental conditions.

The reward function is task-independent, enabling versatile application.

Abstract

Cameras are increasingly being deployed in cities, enterprises and roads world-wide to enable many applications in public safety, intelligent transportation, retail, healthcare and manufacturing. Often, after initial deployment of the cameras, the environmental conditions and the scenes around these cameras change, and our experiments show that these changes can adversely impact the accuracy of insights from video analytics. This is because the camera parameter settings, though optimal at deployment time, are not the best settings for good-quality video capture as the environmental conditions and scenes around a camera change during operation. Capturing poor-quality video adversely affects the accuracy of analytics. To mitigate the loss in accuracy of insights, we propose a novel, reinforcement-learning based system APT that dynamically, and remotely (over 5G networks), tunes the camera parameters, to ensure a high-quality video capture, which mitigates any loss in accuracy of video analytics. As a result, such tuning restores the accuracy of insights when environmental conditions or scene content change. APT uses reinforcement learning, with no-reference perceptual quality estimation as the reward function. We conducted extensive real-world experiments, where we simultaneously deployed two cameras side-by-side overlooking an enterprise parking lot (one camera only has manufacturer-suggested default setting, while the other camera is dynamically tuned by APT during operation). Our experiments demonstrated that due to dynamic tuning by APT, the analytics insights are consistently better at all times of the day: the accuracy of object detection video analytics application was improved on average by ~ 42%. Since our reward function is independent of any analytics task, APT can be readily used for different video analytics tasks.