A Hierarchical Bayesian Linear Regression Model with Local Features for Stochastic Dynamics Approximation

TL;DR

This paper introduces a hierarchical Bayesian linear regression model with local features for efficiently learning stochastic dynamics in robotic systems, enabling accurate predictions and potential real-time control applications.

Contribution

It proposes a novel hierarchical Bayesian model with non-stationary priors and an enhanced variational EM algorithm for learning stochastic dynamics effectively.

Findings

Provides fast and accurate predictions on complex stochastic systems

Produces parsimonious models with large training data

Demonstrates potential for real-time model-based reinforcement learning

Abstract

One of the challenges in model-based control of stochastic dynamical systems is that the state transition dynamics are involved, and it is not easy or efficient to make good-quality predictions of the states. Moreover, there are not many representational models for the majority of autonomous systems, as it is not easy to build a compact model that captures the entire dynamical subtleties and uncertainties. In this work, we present a hierarchical Bayesian linear regression model with local features to learn the dynamics of a micro-robotic system as well as two simpler examples, consisting of a stochastic mass-spring damper and a stochastic double inverted pendulum on a cart. The model is hierarchical since we assume non-stationary priors for the model parameters. These non-stationary priors make the model more flexible by imposing priors on the priors of the model. To solve the maximum likelihood (ML) problem for this hierarchical model, we use the variational expectation maximization (EM) algorithm, and enhance the procedure by introducing hidden target variables. The algorithm yields parsimonious model structures, and consistently provides fast and accurate predictions for all our examples involving large training and test sets. This demonstrates the effectiveness of the method in learning stochastic dynamics, which makes it suitable for future use in a paradigm, such as model-based reinforcement learning, to compute optimal control policies in real time.