Can One-sided Arguments Lead to Response Change in Large Language Models?

TL;DR

This paper investigates whether providing one-sided arguments can steer large language models to adopt specific viewpoints in their responses, demonstrating that opinion steering is effective across various models and conditions.

Contribution

The study systematically analyzes how one-sided arguments influence LLM responses, revealing effective opinion steering methods and their dependence on question formulation and argument presentation.

Findings

Opinion steering occurs across diverse models and topics.

Switching arguments decreases opinion steering.

Steering effectiveness depends on question formulation and argument presentation.

Abstract

Polemic questions need more than one viewpoint to express a balanced answer. Large Language Models (LLMs) can provide a balanced answer, but also take a single aligned viewpoint or refuse to answer. In this paper, we study if such initial responses can be steered to a specific viewpoint in a simple and intuitive way: by only providing one-sided arguments supporting the viewpoint. Our systematic study has three dimensions: (i) which stance is induced in the LLM response, (ii) how the polemic question is formulated, (iii) how the arguments are shown. We construct a small dataset and remarkably find that opinion steering occurs across (i)-(iii) for diverse models, number of arguments, and topics. Switching to other arguments consistently decreases opinion steering.