Cooperative Multi-Objective Reinforcement Learning for Traffic Signal Control and Carbon Emission Reduction

TL;DR

This paper introduces a cooperative multi-objective reinforcement learning framework for traffic signal control that improves traffic flow and reduces carbon emissions, using a decentralized approach with real-world data.

Contribution

It proposes MOMA-DDPG, a novel multi-agent RL architecture that optimizes local and global traffic objectives while minimizing emissions, a first in linking carbon reduction with traffic control.

Findings

Outperforms state-of-the-art traffic control methods.

Reduces waiting time and carbon emissions effectively.

Maintains decentralized inference despite global optimization.

Abstract

Existing traffic signal control systems rely on oversimplified rule-based methods, and even RL-based methods are often suboptimal and unstable. To address this, we propose a cooperative multi-objective architecture called Multi-Objective Multi-Agent Deep Deterministic Policy Gradient (MOMA-DDPG), which estimates multiple reward terms for traffic signal control optimization using age-decaying weights. Our approach involves two types of agents: one focuses on optimizing local traffic at each intersection, while the other aims to optimize global traffic throughput. We evaluate our method using real-world traffic data collected from an Asian country's traffic cameras. Despite the inclusion of a global agent, our solution remains decentralized as this agent is no longer necessary during the inference stage. Our results demonstrate the effectiveness of MOMA-DDPG, outperforming state-of-the-art methods across all performance metrics. Additionally, our proposed system minimizes both waiting time and carbon emissions. Notably, this paper is the first to link carbon emissions and global agents in traffic signal control.