Causal Navigation by Continuous-time Neural Networks

TL;DR

This paper introduces a continuous-time neural network framework for learning causal representations in vision-based navigation tasks, demonstrating improved robustness and generalization over discrete models in complex photorealistic environments.

Contribution

It presents a novel continuous-time neural network approach for causal representation learning, outperforming traditional discrete models in visual drone navigation tasks.

Findings

Continuous-time models outperform discrete models in navigation tasks.

Causal representations enable robust generalization to domain shifts.

Models learn from raw visual inputs to control complex drone behaviors.

Abstract

Imitation learning enables high-fidelity, vision-based learning of policies within rich, photorealistic environments. However, such techniques often rely on traditional discrete-time neural models and face difficulties in generalizing to domain shifts by failing to account for the causal relationships between the agent and the environment. In this paper, we propose a theoretical and experimental framework for learning causal representations using continuous-time neural networks, specifically over their discrete-time counterparts. We evaluate our method in the context of visual-control learning of drones over a series of complex tasks, ranging from short- and long-term navigation, to chasing static and dynamic objects through photorealistic environments. Our results demonstrate that causal continuous-time deep models can perform robust navigation tasks, where advanced recurrent models fail. These models learn complex causal control representations directly from raw visual inputs and scale to solve a variety of tasks using imitation learning.