Reconstruction-free action inference from compressive imagers

TL;DR

This paper introduces a novel approach for action recognition directly from compressive camera measurements, avoiding the need for full video reconstruction, and achieves high recognition accuracy at very high compression ratios.

Contribution

It proposes reconstruction-free action inference methods using spatio-temporal smashed filters, enabling effective recognition directly from compressive measurements at high compression ratios.

Findings

Recognition rates comparable to uncompressed methods at high compression ratios.

Effective use of spatio-temporal smashed filters for direct inference.

Validation on publicly available databases demonstrates practical viability.

Abstract

Persistent surveillance from camera networks, such as at parking lots, UAVs, etc., often results in large amounts of video data, resulting in significant challenges for inference in terms of storage, communication and computation. Compressive cameras have emerged as a potential solution to deal with the data deluge issues in such applications. However, inference tasks such as action recognition require high quality features which implies reconstructing the original video data. Much work in compressive sensing (CS) theory is geared towards solving the reconstruction problem, where state-of-the-art methods are computationally intensive and provide low-quality results at high compression rates. Thus, reconstruction-free methods for inference are much desired. In this paper, we propose reconstruction-free methods for action recognition from compressive cameras at high compression ratios of 100 and above. Recognizing actions directly from CS measurements requires features which are mostly nonlinear and thus not easily applicable. This leads us to search for such properties that are preserved in compressive measurements. To this end, we propose the use of spatio-temporal smashed filters, which are compressive domain versions of pixel-domain matched filters. We conduct experiments on publicly available databases and show that one can obtain recognition rates that are comparable to the oracle method in uncompressed setup, even for high compression ratios.