Video in 10 Bits: Few-Bit VideoQA for Efficiency and Privacy

TL;DR

This paper introduces a Few-Bit VideoQA approach that compresses video information to as little as 10 bits for efficient and privacy-preserving question answering, with minimal accuracy loss.

Contribution

It proposes a task-specific feature compression method with a lightweight module, achieving significant storage efficiency and privacy benefits while maintaining high accuracy.

Findings

Over 100,000-fold storage efficiency compared to MPEG4

Only 2.0-6.6% accuracy loss with few-bit features

Features eliminate most non-task-specific information

Abstract

In Video Question Answering (VideoQA), answering general questions about a video requires its visual information. Yet, video often contains redundant information irrelevant to the VideoQA task. For example, if the task is only to answer questions similar to "Is someone laughing in the video?", then all other information can be discarded. This paper investigates how many bits are really needed from the video in order to do VideoQA by introducing a novel Few-Bit VideoQA problem, where the goal is to accomplish VideoQA with few bits of video information (e.g., 10 bits). We propose a simple yet effective task-specific feature compression approach to solve this problem. Specifically, we insert a lightweight Feature Compression Module (FeatComp) into a VideoQA model which learns to extract task-specific tiny features as little as 10 bits, which are optimal for answering certain types of questions. We demonstrate more than 100,000-fold storage efficiency over MPEG4-encoded videos and 1,000-fold over regular floating point features, with just 2.0-6.6% absolute loss in accuracy, which is a surprising and novel finding. Finally, we analyze what the learned tiny features capture and demonstrate that they have eliminated most of the non-task-specific information, and introduce a Bit Activation Map to visualize what information is being stored. This decreases the privacy risk of data by providing k-anonymity and robustness to feature-inversion techniques, which can influence the machine learning community, allowing us to store data with privacy guarantees while still performing the task effectively.