SurveyLM: A platform to explore emerging value perspectives in augmented language models' behaviors

TL;DR

SurveyLM is a platform that uses survey and experimental methods to analyze and understand the emergent social behaviors and alignment dynamics of augmented language models in complex social contexts.

Contribution

It introduces a novel platform that leverages ALMs' feedback to improve survey design and systematically study their alignment and emergent behaviors.

Findings

Provides unprecedented insights into ALMs' alignment behaviors

Demonstrates the platform's ability to enhance survey frameworks

Facilitates understanding of factors influencing ALMs' social behaviors

Abstract

This white paper presents our work on SurveyLM, a platform for analyzing augmented language models' (ALMs) emergent alignment behaviors through their dynamically evolving attitude and value perspectives in complex social contexts. Social Artificial Intelligence (AI) systems, like ALMs, often function within nuanced social scenarios where there is no singular correct response, or where an answer is heavily dependent on contextual factors, thus necessitating an in-depth understanding of their alignment dynamics. To address this, we apply survey and experimental methodologies, traditionally used in studying social behaviors, to evaluate ALMs systematically, thus providing unprecedented insights into their alignment and emergent behaviors. Moreover, the SurveyLM platform leverages the ALMs' own feedback to enhance survey and experiment designs, exploiting an underutilized aspect of ALMs, which accelerates the development and testing of high-quality survey frameworks while conserving resources. Through SurveyLM, we aim to shed light on factors influencing ALMs' emergent behaviors, facilitate their alignment with human intentions and expectations, and thereby contributed to the responsible development and deployment of advanced social AI systems. This white paper underscores the platform's potential to deliver robust results, highlighting its significance to alignment research and its implications for future social AI systems.