Computational Analysis of Climate Policy

TL;DR

This thesis demonstrates that GPT-4 can effectively analyze local government climate policies, revealing that councils with Climate Emergency Declarations tend to have more recent, urgent, and equitable climate policies, enabling large-scale policy assessment.

Contribution

Introduces PALLM, a GPT-4 based system for large-scale analysis of climate policies, validated with policymakers, and applies it to assess local government responses to climate emergencies.

Findings

Councils with CEDs have more recent climate policies.

Councils with CEDs emphasize urgency and social justice.

GPT-4 can perform high-level policy analysis at scale.

Abstract

This thesis explores the impact of the Climate Emergency movement on local government climate policy, using computational methods. The Climate Emergency movement sought to accelerate climate action at local government level through the mechanism of Climate Emergency Declarations (CEDs), resulting in a series of commitments from councils to treat climate change as an emergency. With the aim of assessing the potential of current large language models to answer complex policy questions, I first built and configured a system named PALLM (Policy Analysis with a Large Language Model), using the OpenAI model GPT-4. This system is designed to apply a conceptual framework for climate emergency response plans to a dataset of climate policy documents. I validated the performance of this system with the help of local government policymakers, by generating analyses of the climate policies of 11 local governments in Victoria and assessing the policymakers' level of agreement with PALLM's responses. Having established that PALLM's performance is satisfactory, I used it to conduct a large-scale analysis of current policy documents from local governments in the state of Victoria, Australia. This thesis presents the methodology and results of this analysis, comparing the results for councils which have passed a CED to those which did not. This study finds that GPT-4 is capable of high-level policy analysis, with limitations including a lack of reliable attribution, and can also enable more nuanced analysis by researchers. Its use in this research shows that councils which have passed a CED are more likely to have a recent and climate-specific policy, and show more attention to urgency, prioritisation, and equity and social justice, than councils which have not. It concludes that the ability to assess policy documents at scale opens up exciting new opportunities for policy researchers.