SeMAIL: Eliminating Distractors in Visual Imitation via Separated Models

TL;DR

SeMAIL is a novel imitation learning algorithm that separates environment dynamics into task-relevant and irrelevant parts, effectively removing distractors and enabling agents to imitate expert behavior efficiently in complex visual environments.

Contribution

The paper introduces SeMAIL, a new method that decouples environment dynamics into relevant and irrelevant components, improving imitation learning in high-dimensional visual tasks.

Findings

Achieves near-expert performance on complex visual control tasks.

Effectively removes distractors like moving backgrounds.

Improves sample efficiency in visual imitation learning.

Abstract

Model-based imitation learning (MBIL) is a popular reinforcement learning method that improves sample efficiency on high-dimension input sources, such as images and videos. Following the convention of MBIL research, existing algorithms are highly deceptive by task-irrelevant information, especially moving distractors in videos. To tackle this problem, we propose a new algorithm - named Separated Model-based Adversarial Imitation Learning (SeMAIL) - decoupling the environment dynamics into two parts by task-relevant dependency, which is determined by agent actions, and training separately. In this way, the agent can imagine its trajectories and imitate the expert behavior efficiently in task-relevant state space. Our method achieves near-expert performance on various visual control tasks with complex observations and the more challenging tasks with different backgrounds from expert observations.