Event-based Non-Rigid Reconstruction from Contours

TL;DR

This paper introduces a novel event-based camera method for reconstructing fast non-rigid object deformations over time, focusing on contours under static background assumptions, and demonstrates its effectiveness over existing approaches.

Contribution

It presents a new probabilistic optimization framework for non-rigid shape reconstruction from event-based data, specifically targeting contour-based deformation estimation.

Findings

Outperforms state-of-the-art optimization methods

Effective on synthetic and real data

Improves motion reconstruction of human hands

Abstract

Visual reconstruction of fast non-rigid object deformations over time is a challenge for conventional frame-based cameras. In this paper, we propose a novel approach for reconstructing such deformations using measurements from event-based cameras. Under the assumption of a static background, where all events are generated by the motion, our approach estimates the deformation of objects from events generated at the object contour in a probabilistic optimization framework. It associates events to mesh faces on the contour and maximizes the alignment of the line of sight through the event pixel with the associated face. In experiments on synthetic and real data, we demonstrate the advantages of our method over state-of-the-art optimization and learning-based approaches for reconstructing the motion of human hands. A video of the experiments is available at https://youtu.be/gzfw7i5OKjg