Event-based Visual Deformation Measurement

TL;DR

This paper introduces an event-frame fusion framework for visual deformation measurement that combines event data and frames to achieve dense, accurate deformation tracking with reduced resource usage, outperforming existing methods.

Contribution

The paper proposes a novel Affine Invariant Simplicial (AIS) framework and neighborhood-greedy optimization for efficient, robust deformation estimation from event and frame data.

Findings

Outperforms state-of-the-art by 1.6% in survival rate

Uses only 18.9% of resources compared to high-speed video methods

Establishes a new benchmark dataset with 120+ sequences

Abstract

Visual Deformation Measurement (VDM) aims to recover dense deformation fields by tracking surface motion from camera observations. Traditional image-based methods rely on minimal inter-frame motion to constrain the correspondence search space, which limits their applicability to highly dynamic scenes or necessitates high-speed cameras at the cost of prohibitive storage and computational overhead. We propose an event-frame fusion framework that exploits events for temporally dense motion cues and frames for spatially dense precise estimation. Revisiting the solid elastic modeling prior, we propose an Affine Invariant Simplicial (AIS) framework. It partitions the deformation field into linearized sub-regions with low-parametric representation, effectively mitigating motion ambiguities arising from sparse and noisy events. To speed up parameter searching and reduce error accumulation, a neighborhood-greedy optimization strategy is introduced, enabling well-converged sub-regions to guide their poorly-converged neighbors, effectively suppress local error accumulation in long-term dense tracking. To evaluate the proposed method, a benchmark dataset with temporally aligned event streams and frames is established, encompassing over 120 sequences spanning diverse deformation scenarios. Experimental results show that our method outperforms the state-of-the-art baseline by 1.6% in survival rate. Remarkably, it achieves this using only 18.9% of the data storage and processing resources of high-speed video methods.