HMM-guided frame querying for bandwidth-constrained video search
Bhairav Chidambaram, Mason McGill, Pietro Perona

TL;DR
This paper presents a method combining CNNs and HMMs to efficiently identify key video frames under bandwidth constraints, enabling high accuracy with minimal data transfer.
Contribution
It introduces a novel approach that uses HMM-guided frame querying with CNN scores to reduce bandwidth in remote video search tasks.
Findings
Achieves 98% frame omission without losing classification accuracy.
Effective in identifying temporal regions of interest with sparse sampling.
Demonstrates scalability on ImageNet-VID dataset.
Abstract
We design an agent to search for frames of interest in video stored on a remote server, under bandwidth constraints. Using a convolutional neural network to score individual frames and a hidden Markov model to propagate predictions across frames, our agent accurately identifies temporal regions of interest based on sparse, strategically sampled frames. On a subset of the ImageNet-VID dataset, we demonstrate that using a hidden Markov model to interpolate between frame scores allows requests of 98% of frames to be omitted, without compromising frame-of-interest classification accuracy.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Image and Video Retrieval Techniques · Video Analysis and Summarization · Advanced Vision and Imaging
