Roadmap on Incentive Compatibility for AI Alignment and Governance in Sociotechnical Systems

TL;DR

This paper introduces the Incentive Compatibility Sociotechnical Alignment Problem (ICSAP), emphasizing the importance of integrating game theory principles to align AI systems with societal values and governance.

Contribution

It proposes a new sociotechnical alignment problem, exploring how incentive compatibility concepts can bridge technical and societal aspects of AI governance.

Findings

Identifies ICSAP as a key challenge in AI alignment.

Discusses classical game theory solutions like mechanism design and contract theory.

Provides preliminary ideas for implementing incentive-compatible AI systems.

Abstract

The burgeoning integration of artificial intelligence (AI) into human society brings forth significant implications for societal governance and safety. While considerable strides have been made in addressing AI alignment challenges, existing methodologies primarily focus on technical facets, often neglecting the intricate sociotechnical nature of AI systems, which can lead to a misalignment between the development and deployment contexts. To this end, we posit a new problem worth exploring: Incentive Compatibility Sociotechnical Alignment Problem (ICSAP). We hope this can call for more researchers to explore how to leverage the principles of Incentive Compatibility (IC) from game theory to bridge the gap between technical and societal components to maintain AI consensus with human societies in different contexts. We further discuss three classical game problems for achieving IC: mechanism design, contract theory, and Bayesian persuasion, in addressing the perspectives, potentials, and challenges of solving ICSAP, and provide preliminary implementation conceptions.