Frequency Decoupling for Motion Magnification via Multi-Level Isomorphic Architecture

TL;DR

The paper introduces FD4MM, a frequency decoupling approach with a multi-level isomorphic architecture for video motion magnification, effectively capturing subtle motions while reducing noise and computational costs.

Contribution

It proposes a novel frequency decoupling paradigm with multi-level isomorphic architecture, sparse filters, and contrastive regularization for improved motion magnification.

Findings

Outperforms state-of-the-art methods in accuracy.

Reduces FLOPs by 1.63 times and increases inference speed by 1.68 times.

Effectively captures multi-level high-frequency details and stable low-frequency motion structures.

Abstract

Video Motion Magnification (VMM) aims to reveal subtle and imperceptible motion information of objects in the macroscopic world. Prior methods directly model the motion field from the Eulerian perspective by Representation Learning that separates shape and texture or Multi-domain Learning from phase fluctuations. Inspired by the frequency spectrum, we observe that the low-frequency components with stable energy always possess spatial structure and less noise, making them suitable for modeling the subtle motion field. To this end, we present FD4MM, a new paradigm of Frequency Decoupling for Motion Magnification with a Multi-level Isomorphic Architecture to capture multi-level high-frequency details and a stable low-frequency structure (motion field) in video space. Since high-frequency details and subtle motions are susceptible to information degradation due to their inherent subtlety and unavoidable external interference from noise, we carefully design Sparse High/Low-pass Filters to enhance the integrity of details and motion structures, and a Sparse Frequency Mixer to promote seamless recoupling. Besides, we innovatively design a contrastive regularization for this task to strengthen the model's ability to discriminate irrelevant features, reducing undesired motion magnification. Extensive experiments on both Real-world and Synthetic Datasets show that our FD4MM outperforms SOTA methods. Meanwhile, FD4MM reduces FLOPs by 1.63$\times$ and boosts inference speed by 1.68$\times$ than the latest method. Our code is available at https://github.com/Jiafei127/FD4MM.