DeepKoCo: Efficient latent planning with a task-relevant Koopman representation

TL;DR

DeepKoCo introduces a model-based agent that learns a task-relevant latent Koopman representation from images, enabling efficient planning and robustness to distractors in complex control tasks.

Contribution

It proposes a novel autoencoder-based method to learn task-focused latent dynamics for efficient planning with linear control methods.

Findings

Achieves comparable performance to model-free methods on complex tasks.

Demonstrates increased robustness to distractor dynamics.

Enables real-life application potential due to robustness.

Abstract

This paper presents DeepKoCo, a novel model-based agent that learns a latent Koopman representation from images. This representation allows DeepKoCo to plan efficiently using linear control methods, such as linear model predictive control. Compared to traditional agents, DeepKoCo learns task-relevant dynamics, thanks to the use of a tailored lossy autoencoder network that allows DeepKoCo to learn latent dynamics that reconstruct and predict only observed costs, rather than all observed dynamics. As our results show, DeepKoCo achieves similar final performance as traditional model-free methods on complex control tasks while being considerably more robust to distractor dynamics, making the proposed agent more amenable for real-life applications.