SpikeMM: Flexi-Magnification of High-Speed Micro-Motions

TL;DR

SpikeMM is a novel spike-based algorithm designed to magnify high-speed micro-motions by leveraging spike camera data, overcoming limitations of traditional methods in high-frequency scenarios.

Contribution

It introduces SpikeMM, a self-supervised, spike-based motion magnification method that integrates multi-level information extraction and spatial upsampling for high-speed motion analysis.

Findings

Effective magnification of high-speed micro-motions demonstrated

Seamless integration with super-resolution algorithms achieved

Validated on real-world spike camera scenes

Abstract

The amplification of high-speed micro-motions holds significant promise, with applications spanning fault detection in fast-paced industrial environments to refining precision in medical procedures. However, conventional motion magnification algorithms often encounter challenges in high-speed scenarios due to low sampling rates or motion blur. In recent years, spike cameras have emerged as a superior alternative for visual tasks in such environments, owing to their unique capability to capture temporal and spatial frequency domains with exceptional fidelity. Unlike conventional cameras, which operate at fixed, low frequencies, spike cameras emulate the functionality of the retina, asynchronously capturing photon changes at each pixel position using spike streams. This innovative approach comprehensively records temporal and spatial visual information, rendering it particularly suitable for magnifying high-speed micro-motions.This paper introduces SpikeMM, a pioneering spike-based algorithm tailored specifically for high-speed motion magnification. SpikeMM integrates multi-level information extraction, spatial upsampling, and motion magnification modules, offering a self-supervised approach adaptable to a wide range of scenarios. Notably, SpikeMM facilitates seamless integration with high-performance super-resolution and motion magnification algorithms. We substantiate the efficacy of SpikeMM through rigorous validation using scenes captured by spike cameras, showcasing its capacity to magnify motions in real-world high-frequency settings.