A Probabilistic Model for Skill Acquisition with Switching Latent Feedback Controllers

TL;DR

This paper introduces a probabilistic model that interprets feedback controllers as latent skills, enabling robots to learn and switch between skills effectively, improving success rates and robustness from demonstrations.

Contribution

It reinterprets mixture density networks as feedback controllers, deriving a probabilistic model for skill segmentation and switching in robot learning from demonstrations.

Findings

Improves task success rate significantly.

Enhances robustness to observation noise.

Demonstrates effective deployment on physical robots.

Abstract

Manipulation tasks often consist of subtasks, each representing a distinct skill. Mastering these skills is essential for robots, as it enhances their autonomy, efficiency, adaptability, and ability to work in their environment. Learning from demonstrations allows robots to rapidly acquire new skills without starting from scratch, with demonstrations typically sequencing skills to achieve tasks. Behaviour cloning approaches to learning from demonstration commonly rely on mixture density network output heads to predict robot actions. In this work, we first reinterpret the mixture density network as a library of feedback controllers (or skills) conditioned on latent states. This arises from the observation that a one-layer linear network is functionally equivalent to a classical feedback controller, with network weights corresponding to controller gains. We use this insight to derive a probabilistic graphical model that combines these elements, describing the skill acquisition process as segmentation in a latent space, where each skill policy functions as a feedback control law in this latent space. Our approach significantly improves not only task success rate, but also robustness to observation noise when trained with human demonstrations. Our physical robot experiments further show that the induced robustness improves model deployment on robots.