Towards Streaming Egocentric Action Anticipation
Antonino Furnari, Giovanni Maria Farinella

TL;DR
This paper introduces a streaming evaluation protocol for egocentric action anticipation, emphasizing model runtime, and proposes a lightweight model that outperforms heavier models in this real-time scenario.
Contribution
It proposes a new streaming evaluation scheme considering runtime and introduces a lightweight, effective 3D CNN model optimized with knowledge distillation.
Findings
Smaller models outperform heavier ones in streaming scenarios.
The proposed lightweight model surpasses prior art in real-time settings.
Runtime-aware evaluation changes model rankings compared to offline assessments.
Abstract
Egocentric action anticipation is the task of predicting the future actions a camera wearer will likely perform based on past video observations. While in a real-world system it is fundamental to output such predictions before the action begins, past works have not generally paid attention to model runtime during evaluation. Indeed, current evaluation schemes assume that predictions can be made offline, and hence that computational resources are not limited. In contrast, in this paper, we propose a "streaming" egocentric action anticipation evaluation protocol which explicitly considers model runtime for performance assessment, assuming that predictions will be available only after the current video segment is processed, which depends on the processing time of a method. Following the proposed evaluation scheme, we benchmark different state-of-the-art approaches for egocentric action…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Pose and Action Recognition · Video Surveillance and Tracking Methods · Advanced Vision and Imaging
Methods3 Dimensional Convolutional Neural Network · Knowledge Distillation
