DQN Control Solution for KDD Cup 2021 City Brain Challenge

TL;DR

This paper presents a DQN-based approach for real-time traffic signal control in a complex city network, achieving top rankings in a major competition and offering a potential baseline for future research.

Contribution

The paper adapts the deep Q-network framework with a novel reward function for real-world traffic signal control in complex networks.

Findings

Achieved 8th place in KDD Cup 2021 City Brain Challenge.

A single DQN agent can rank among top 15 teams with proper tuning.

Proposed a new reward function tailored for traffic signal control.

Abstract

We took part in the city brain challenge competition and achieved the 8th place. In this competition, the players are provided with a real-world city-scale road network and its traffic demand derived from real traffic data. The players are asked to coordinate the traffic signals with a self-designed agent to maximize the number of vehicles served while maintaining an acceptable delay. In this abstract paper, we present an overall analysis and our detailed solution to this competition. Our approach is mainly based on the adaptation of the deep Q-network (DQN) for real-time traffic signal control. From our perspective, the major challenge of this competition is how to extend the classical DQN framework to traffic signals control in real-world complex road network and traffic flow situation. After trying and implementing several classical reward functions, we finally chose to apply our newly-designed reward in our agent. By applying our newly-proposed reward function and carefully tuning the control scheme, an agent based on a single DQN model can rank among the top 15 teams. We hope this paper could serve, to some extent, as a baseline solution to traffic signal control of real-world road network and inspire further attempts and researches.