Learning reusable concepts across different egocentric video understanding tasks

TL;DR

This paper introduces Hier-EgoPack, a unified framework that learns reusable concepts across various egocentric video understanding tasks to enhance holistic perception and task transfer in autonomous systems.

Contribution

The paper presents Hier-EgoPack, a novel framework for abstracting and transferring knowledge across multiple egocentric video tasks.

Findings

Hier-EgoPack effectively captures shared concepts across tasks.

The framework improves performance on downstream tasks.

It demonstrates the benefits of multi-task concept learning.

Abstract

Our comprehension of video streams depicting human activities is naturally multifaceted: in just a few moments, we can grasp what is happening, identify the relevance and interactions of objects in the scene, and forecast what will happen soon, everything all at once. To endow autonomous systems with such holistic perception, learning how to correlate concepts, abstract knowledge across diverse tasks, and leverage tasks synergies when learning novel skills is essential. In this paper, we introduce Hier-EgoPack, a unified framework able to create a collection of task perspectives that can be carried across downstream tasks and used as a potential source of additional insights, as a backpack of skills that a robot can carry around and use when needed.