Deep Bilinear Koopman Model for Real-Time Vehicle Control in Frenet Frame

TL;DR

This paper introduces a deep Koopman model within the Frenet frame for autonomous vehicle control, enabling real-time, accurate trajectory tracking by combining neural networks with invariant subspace learning and error regulation.

Contribution

It proposes a novel deep Koopman approach that learns invariant subspaces for vehicle dynamics, integrated with a cumulative error regulator for improved real-time control.

Findings

Achieved significant reduction in tracking error in hardware-in-the-loop tests.

Demonstrated real-time applicability on embedded systems.

Enhanced long-horizon prediction accuracy with multi-step loss.

Abstract

Accurate modeling and control of autonomous vehicles remain a fundamental challenge due to the nonlinear and coupled nature of vehicle dynamics. While Koopman operator theory offers a framework for deploying powerful linear control techniques, learning a finite-dimensional invariant subspace for high-fidelity modeling continues to be an open problem. This paper presents a deep Koopman approach for modeling and control of vehicle dynamics within the curvilinear Frenet frame. The proposed framework uses a deep neural network architecture to simultaneously learn the Koopman operator and its associated invariant subspace from the data. Input-state bilinear interactions are captured by the algorithm while preserving convexity, which makes it suitable for real-time model predictive control (MPC) application. A multi-step prediction loss is utilized during training to ensure long-horizon prediction capability. To further enhance real-time trajectory tracking performance, the model is integrated with a cumulative error regulator (CER) module, which compensates for model mismatch by mitigating accumulated prediction errors. Closed-loop performance is evaluated through hardware-in-the-loop (HIL) experiments using a CarSim RT model as the target plant, with real-time validation conducted on a dSPACE SCALEXIO system. The proposed controller achieved significant reductions in tracking error relative to baseline controllers, confirming its suitability for real-time implementation in embedded autonomous vehicle systems.