Crafting desirable climate trajectories with RL explored socio-environmental simulations

TL;DR

This paper explores the use of multi-agent reinforcement learning in socio-environmental simulations to model complex stakeholder interactions and identify pathways toward desirable climate futures, highlighting the importance of cooperation versus competition.

Contribution

It introduces a multi-agent RL framework to simulate socio-environmental interactions in climate policy modeling, extending traditional IAM approaches with a focus on agent cooperation and competition.

Findings

Cooperative agents can find pathways to reduce emissions and improve economy.

Competitive agents often fail to reach desirable climate outcomes.

Visualisation of states reveals causes of uncertainty and algorithm failure.

Abstract

Climate change poses an existential threat, necessitating effective climate policies to enact impactful change. Decisions in this domain are incredibly complex, involving conflicting entities and evidence. In the last decades, policymakers increasingly use simulations and computational methods to guide some of their decisions. Integrated Assessment Models (IAMs) are one of such methods, which combine social, economic, and environmental simulations to forecast potential policy effects. For example, the UN uses outputs of IAMs for their recent Intergovernmental Panel on Climate Change (IPCC) reports. Traditionally these have been solved using recursive equation solvers, but have several shortcomings, e.g. struggling at decision making under uncertainty. Recent preliminary work using Reinforcement Learning (RL) to replace the traditional solvers shows promising results in decision making in uncertain and noisy scenarios. We extend on this work by introducing multiple interacting RL agents as a preliminary analysis on modelling the complex interplay of socio-interactions between various stakeholders or nations that drives much of the current climate crisis. Our findings show that cooperative agents in this framework can consistently chart pathways towards more desirable futures in terms of reduced carbon emissions and improved economy. However, upon introducing competition between agents, for instance by using opposing reward functions, desirable climate futures are rarely reached. Modelling competition is key to increased realism in these simulations, as such we employ policy interpretation by visualising what states lead to more uncertain behaviour, to understand algorithm failure. Finally, we highlight the current limitations and avenues for further work to ensure future technology uptake for policy derivation.