Event-Grounding Graph: Unified Spatio-Temporal Scene Graph from Robotic Observations
Phuoc Nguyen, Francesco Verdoja, Ville Kyrki

TL;DR
The paper introduces the Event-Grounding Graph (EGG), a unified spatio-temporal scene graph framework that links spatial features with dynamic events, enabling robots to better perceive, reason, and respond to complex queries about their environment.
Contribution
The novel EGG framework connects spatial features with event interactions, enhancing robotic scene understanding and reasoning capabilities.
Findings
EGG effectively retrieves relevant environmental information.
EGG accurately responds to human queries about scenes and events.
Open-source code and dataset available for further research.
Abstract
A fundamental aspect for building intelligent autonomous robots that can assist humans in their daily lives is the construction of rich environmental representations. While advances in semantic scene representations have enriched robotic scene understanding, current approaches lack a connection between spatial features and dynamic events; e.g., connecting the blue mug to the event washing a mug. In this work, we introduce the event-grounding graph (EGG), a framework grounding event interactions to spatial features of a scene. This representation allows robots to perceive, reason, and respond to complex spatio-temporal queries. Experiments using real robotic data demonstrate EGG's capability to retrieve relevant information and respond accurately to human inquiries concerning the environment and events within. Furthermore, the EGG framework's source code and evaluation dataset are…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Robotics and Sensor-Based Localization · Autonomous Vehicle Technology and Safety
