TTPOINT: A Tensorized Point Cloud Network for Lightweight Action Recognition with Event Cameras

TL;DR

TTPOINT is a lightweight, tensor-train compressed point cloud network that achieves state-of-the-art action recognition performance on event camera data with minimal computational resources.

Contribution

The paper introduces TTPOINT, a novel tensorized point cloud network that is highly efficient and maintains high accuracy, advancing lightweight action recognition with event cameras.

Findings

TTPOINT achieves SOTA results on three datasets.

TTPOINT maintains accuracy while reducing parameters by 55%.

TTPOINT uses tensor-train decomposition for model compression.

Abstract

Event cameras have gained popularity in computer vision due to their data sparsity, high dynamic range, and low latency. As a bio-inspired sensor, event cameras generate sparse and asynchronous data, which is inherently incompatible with the traditional frame-based method. Alternatively, the point-based method can avoid additional modality transformation and naturally adapt to the sparsity of events. Still, it typically cannot reach a comparable accuracy as the frame-based method. We propose a lightweight and generalized point cloud network called TTPOINT which achieves competitive results even compared to the state-of-the-art (SOTA) frame-based method in action recognition tasks while only using 1.5 % of the computational resources. The model is adept at abstracting local and global geometry by hierarchy structure. By leveraging tensor-train compressed feature extractors, TTPOINT can be designed with minimal parameters and computational complexity. Additionally, we developed a straightforward downsampling algorithm to maintain the spatio-temporal feature. In the experiment, TTPOINT emerged as the SOTA method on three datasets while also attaining SOTA among point cloud methods on all five datasets. Moreover, by using the tensor-train decomposition method, the accuracy of the proposed TTPOINT is almost unaffected while compressing the parameter size by 55 % in all five datasets.