A Functional Learning Approach for Team-Optimal Traffic Coordination

TL;DR

This paper introduces a kernel-based functional learning framework for multi-agent traffic coordination, combining model-based and model-free approaches to optimize strategies in intersection scenarios.

Contribution

It develops a novel Hilbert space-based policy iteration method for team-optimal traffic management, incorporating nonlinear safety penalties and recursive system estimation.

Findings

Effective in signal-free intersection simulations

Validated with both model-based and model-free methods

Demonstrates improved traffic coordination strategies

Abstract

In this paper, we develop a kernel-based policy iteration functional learning framework for computing team-optimal strategies in traffic coordination problems. We consider a multi-agent discrete-time linear system with a cost function that combines quadratic regulation terms and nonlinear safety penalties. Building on the Hilbert space formulation of offline receding-horizon policy iteration, we seek approximate solutions within a reproducing kernel Hilbert space, where the policy improvement step is implemented via a discrete Fr\'echet derivative. We further study the model-free receding-horizon scenario, where the system dynamics are estimated using recursive least squares, followed by updating the policy using rolling online data. The proposed method is tested in signal-free intersection scenarios via both model-based and model-free simulations and validated in SUMO.