Perspective Dial: Measuring Perspective of Text and Guiding LLM Outputs

TL;DR

This paper introduces Perspective Dial, a method for quantifying and controlling the perspective of LLM outputs using a novel metric space and systematic prompt engineering, enabling bias detection and mitigation.

Contribution

It presents a new framework combining Perspective Space and systematic prompt engineering to measure and steer LLM perspectives, addressing bias and viewpoint issues.

Findings

Effective measurement of perspectives on various topics.

Ability to steer LLM outputs towards desired viewpoints.

Potential for bias detection and mitigation in LLMs.

Abstract

Large language models (LLMs) are used in a variety of mission-critical roles. Due to the rapidly developing nature of LLMs, there is a lack of quantifiable understanding of the bias and perspective associated with LLM output. Inspired by this need, this paper considers the broader issue of perspective or viewpoint of general text and perspective control of large-language model (LLM) output. Perspective-Dial consists of two main components: a (1) metric space, dubbed Perspective Space, that enables quantitative measurements of different perspectives regarding a topic, and the use of (2) Systematic Prompt Engineering that utilizes greedy-coordinate descent to control LLM output perspective based on measurement feedback from the Perspective Space. The empirical nature of the approach allows progress to side step a principled understanding of perspective or bias -- effectively quantifying and adjusting outputs for a variety of topics. Potential applications include detection, tracking and mitigation of LLM bias, narrative detection, sense making and tracking in public discourse, and debate bot advocating given perspective.