Variational Temporal Abstraction
Taesup Kim, Sungjin Ahn, Yoshua Bengio

TL;DR
This paper presents a variational hierarchical model for learning interpretable temporal structures in sequential data, enhancing agent learning efficiency through jumpy imagination in navigation tasks.
Contribution
It introduces the Variational Temporal Abstraction (VTA), a hierarchical recurrent state space model that infers latent temporal structures and improves imagination-based agent learning.
Findings
Successfully models 2D and 3D visual sequences with interpretable temporal hierarchies.
Enables more efficient agent learning in 3D navigation through jumpy imagination.
Demonstrates the effectiveness of hierarchical temporal inference in sequential data modeling.
Abstract
We introduce a variational approach to learning and inference of temporally hierarchical structure and representation for sequential data. We propose the Variational Temporal Abstraction (VTA), a hierarchical recurrent state space model that can infer the latent temporal structure and thus perform the stochastic state transition hierarchically. We also propose to apply this model to implement the jumpy-imagination ability in imagination-augmented agent-learning in order to improve the efficiency of the imagination. In experiments, we demonstrate that our proposed method can model 2D and 3D visual sequence datasets with interpretable temporal structure discovery and that its application to jumpy imagination enables more efficient agent-learning in a 3D navigation task.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Image and Video Retrieval Techniques · Image Retrieval and Classification Techniques · Time Series Analysis and Forecasting
