Justifications for Democratizing AI Alignment and Their Prospects

TL;DR

This paper explores democratic versus epistocratic approaches to AI alignment's normative problem, analyzing their justifications, challenges, and proposing hybrid frameworks to address legitimacy and coercion concerns.

Contribution

It provides a comprehensive analysis of democratic justifications for AI alignment, highlighting the need for hybrid approaches combining expertise and stakeholder participation.

Findings

Democratic approaches can better address normative uncertainties.

Challenges include preventing illegitimate coercion and AI monopolization.

Hybrid frameworks may offer balanced solutions.

Abstract

The AI alignment problem comprises both technical and normative dimensions. While technical solutions focus on implementing normative constraints in AI systems, the normative problem concerns determining what these constraints should be. This paper examines justifications for democratic approaches to the normative problem -- where affected stakeholders determine AI alignment -- as opposed to epistocratic approaches that defer to normative experts. We analyze both instrumental justifications (democratic approaches produce better outcomes) and non-instrumental justifications (democratic approaches prevent illegitimate authority or coercion). We argue that normative and metanormative uncertainty create a justificatory gap that democratic approaches aim to fill through political rather than theoretical justification. However, we identify significant challenges for democratic approaches, particularly regarding the prevention of illegitimate coercion through AI alignment. Our analysis suggests that neither purely epistocratic nor purely democratic approaches may be sufficient on their own, pointing toward hybrid frameworks that combine expert judgment with participatory input alongside institutional safeguards against AI monopolization.