Artificial Microsaccade Compensation: Stable Vision for an Ornithopter

TL;DR

This paper introduces a real-time artificial microsaccade compensation method that stabilizes shaky video from an ornithopter, outperforming commercial stabilizers in quality without sacrificing speed.

Contribution

It develops a novel real-time stabilization technique inspired by biological microsaccades, optimized over 3D rotation in SO(3), effective for highly unstable camera motion.

Findings

Achieves higher stabilization quality than Adobe Premiere Pro.

Operates in real time suitable for live viewing.

Effectively reduces inter-frame motion in shaky videos.

Abstract

Animals with foveated vision, including humans, experience microsaccades, small, rapid eye movements that they are not aware of. Inspired by this phenomenon, we develop a method for "Artificial Microsaccade Compensation". It can stabilize video captured by a tailless ornithopter that has resisted attempts to use camera-based sensing because it shakes at 12-20 Hz. Our approach minimizes changes in image intensity by optimizing over 3D rotation represented in SO(3). This results in a stabilized video, computed in real time, suitable for human viewing, and free from distortion. When adapted to hold a fixed viewing orientation, up to occasional saccades, it can dramatically reduce inter-frame motion while also benefiting from an efficient recursive update. When compared to Adobe Premier Pro's warp stabilizer, which is widely regarded as the best commercial video stabilization software available, our method achieves higher quality results while also running in real time.