Busy-Quiet Video Disentangling for Video Classification

TL;DR

This paper introduces a novel video classification model that separates busy motion from quiet background using a trainable frequency-based module, improving efficiency and accuracy across multiple datasets.

Contribution

The paper proposes the Busy-Quiet Net with a Motion Band-Pass Module to efficiently disentangle motion information, reducing redundancy and computational cost in video processing.

Findings

Outperforms recent models on multiple datasets

Efficiently separates busy and quiet information

Reduces redundancy in feature processing

Abstract

In video data, busy motion details from moving regions are conveyed within a specific frequency bandwidth in the frequency domain. Meanwhile, the rest of the frequencies of video data are encoded with quiet information with substantial redundancy, which causes low processing efficiency in existing video models that take as input raw RGB frames. In this paper, we consider allocating intenser computation for the processing of the important busy information and less computation for that of the quiet information. We design a trainable Motion Band-Pass Module (MBPM) for separating busy information from quiet information in raw video data. By embedding the MBPM into a two-pathway CNN architecture, we define a Busy-Quiet Net (BQN). The efficiency of BQN is determined by avoiding redundancy in the feature space processed by the two pathways: one operating on Quiet features of low-resolution, while the other processes Busy features. The proposed BQN outperforms many recent video processing models on Something-Something V1, Kinetics400, UCF101 and HMDB51 datasets.