UnweaveNet: Unweaving Activity Stories
Will Price, Carl Vondrick, Dima Damen

TL;DR
UnweaveNet is a neural model designed to parse egocentric videos into distinct activity threads, effectively detecting goal changes and activity resumption, with demonstrated success on the EPIC-KITCHENS dataset.
Contribution
The paper introduces UnweaveNet, a novel neural architecture with a thread bank and controller for unweaving complex activity sequences in videos, including a self-supervised pretraining approach.
Findings
Effective activity thread detection demonstrated on EPIC-KITCHENS.
Self-supervised pretraining improves model performance.
UnweaveNet accurately identifies goal changes and activity resumption.
Abstract
Our lives can be seen as a complex weaving of activities; we switch from one activity to another, to maximise our achievements or in reaction to demands placed upon us. Observing a video of unscripted daily activities, we parse the video into its constituent activity threads through a process we call unweaving. To accomplish this, we introduce a video representation explicitly capturing activity threads called a thread bank, along with a neural controller capable of detecting goal changes and resuming of past activities, together forming UnweaveNet. We train and evaluate UnweaveNet on sequences from the unscripted egocentric dataset EPIC-KITCHENS. We propose and showcase the efficacy of pretraining UnweaveNet in a self-supervised manner.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Pose and Action Recognition · Context-Aware Activity Recognition Systems
