Transfer Learning versus Multi-agent Learning regarding Distributed Decision-Making in Highway Traffic

TL;DR

This paper compares transfer learning and multi-agent learning in mixed-intelligence highway traffic scenarios using deep reinforcement learning within the MIT DeepTraffic simulation, analyzing their impact on traffic flow and decision-making.

Contribution

It introduces a comparative analysis of transfer learning versus multi-agent learning for distributed decision-making in mixed traffic using deep reinforcement learning.

Findings

Transfer learning can effectively initialize multi-agent systems.

Multi-agent learning adapts better to high ratios of AI-driven traffic.

Traffic flow varies significantly with different learning strategies.

Abstract

Transportation and traffic are currently undergoing a rapid increase in terms of both scale and complexity. At the same time, an increasing share of traffic participants are being transformed into agents driven or supported by artificial intelligence resulting in mixed-intelligence traffic. This work explores the implications of distributed decision-making in mixed-intelligence traffic. The investigations are carried out on the basis of an online-simulated highway scenario, namely the MIT \emph{DeepTraffic} simulation. In the first step traffic agents are trained by means of a deep reinforcement learning approach, being deployed inside an elitist evolutionary algorithm for hyperparameter search. The resulting architectures and training parameters are then utilized in order to either train a single autonomous traffic agent and transfer the learned weights onto a multi-agent scenario or else to conduct multi-agent learning directly. Both learning strategies are evaluated on different ratios of mixed-intelligence traffic. The strategies are assessed according to the average speed of all agents driven by artificial intelligence. Traffic patterns that provoke a reduction in traffic flow are analyzed with respect to the different strategies.