EASE: Embodied Active Event Perception via Self-Supervised Energy Minimization

TL;DR

EASE introduces a self-supervised, energy minimization framework for real-time, adaptive event perception in embodied systems, eliminating the need for annotations or external rewards.

Contribution

It unifies perception and control through intrinsic signals, enabling scalable, adaptable event understanding without predefined action spaces or labeled data.

Findings

Effective in simulation and real-world environments

Achieves emergent behaviors like implicit memory and target tracking

Operates without explicit annotations or external rewards

Abstract

Active event perception, the ability to dynamically detect, track, and summarize events in real time, is essential for embodied intelligence in tasks such as human-AI collaboration, assistive robotics, and autonomous navigation. However, existing approaches often depend on predefined action spaces, annotated datasets, and extrinsic rewards, limiting their adaptability and scalability in dynamic, real-world scenarios. Inspired by cognitive theories of event perception and predictive coding, we propose EASE, a self-supervised framework that unifies spatiotemporal representation learning and embodied control through free energy minimization. EASE leverages prediction errors and entropy as intrinsic signals to segment events, summarize observations, and actively track salient actors, operating without explicit annotations or external rewards. By coupling a generative perception model with an action-driven control policy, EASE dynamically aligns predictions with observations, enabling emergent behaviors such as implicit memory, target continuity, and adaptability to novel environments. Extensive evaluations in simulation and real-world settings demonstrate EASE's ability to achieve privacy-preserving and scalable event perception, providing a robust foundation for embodied systems in unscripted, dynamic tasks.