Learning State-Space Models of Dynamic Systems from Arbitrary Data using Joint Embedding Predictive Architectures

TL;DR

This paper introduces a novel method combining joint embedding architectures and neural ODEs to learn structured state-space models of dynamic systems directly from arbitrary observation data, demonstrated on a pendulum system.

Contribution

It presents a new approach that integrates sequence embeddings with neural ODEs and contractive loss functions to create well-organized latent state spaces from arbitrary data.

Findings

Successfully modeled a simple pendulum from image data.

Generated structured latent state-space models.

Potential applications in robotics control and estimation.

Abstract

With the advent of Joint Embedding Predictive Architectures (JEPAs), which appear to be more capable than reconstruction-based methods, this paper introduces a novel technique for creating world models using continuous-time dynamic systems from arbitrary observation data. The proposed method integrates sequence embeddings with neural ordinary differential equations (neural ODEs). It employs loss functions that enforce contractive embeddings and Lipschitz constants in state transitions to construct a well-organized latent state space. The approach's effectiveness is demonstrated through the generation of structured latent state-space models for a simple pendulum system using only image data. This opens up a new technique for developing more general control algorithms and estimation techniques with broad applications in robotics.