Agile wide-field imaging with selective high resolution

TL;DR

This paper introduces an agile wide-field imaging system with selective high resolution using only two detectors, leveraging scene sparsity and deep learning for real-time ROI detection and tracking, suitable for UAV applications.

Contribution

It presents a novel imaging framework combining low- and high-resolution cameras with deep learning-based ROI detection, reducing complexity and cost compared to traditional large detector arrays.

Findings

Achieved 120° FOV with 0.45 mrad instantaneous FOV.

Built a lightweight (1181g) UAV-mounted proof-of-concept system.

Demonstrated real-time ROI tracking and high-resolution imaging in aerial monitoring.

Abstract

Wide-field and high-resolution (HR) imaging is essential for various applications such as aviation reconnaissance, topographic mapping and safety monitoring. The existing techniques require a large-scale detector array to capture HR images of the whole field, resulting in high complexity and heavy cost. In this work, we report an agile wide-field imaging framework with selective high resolution that requires only two detectors. It builds on the statistical sparsity prior of natural scenes that the important targets locate only at small regions of interests (ROI), instead of the whole field. Under this assumption, we use a short-focal camera to image wide field with a certain low resolution, and use a long-focal camera to acquire the HR images of ROI. To automatically locate ROI in the wide field in real time, we propose an efficient deep-learning based multiscale registration method that is robust and blind to the large setting differences (focal, white balance, etc) between the two cameras. Using the registered location, the long-focal camera mounted on a gimbal enables real-time tracking of the ROI for continuous HR imaging. We demonstrated the novel imaging framework by building a proof-of-concept setup with only 1181 gram weight, and assembled it on an unmanned aerial vehicle for air-to-ground monitoring. Experiments show that the setup maintains 120$^{\circ}$ wide field-of-view (FOV) with selective 0.45$mrad$ instantaneous FOV.