Self-Agreement: A Framework for Fine-tuning Language Models to Find Agreement among Diverse Opinions

TL;DR

This paper introduces Self-Agreement, a framework that fine-tunes language models to autonomously identify agreement among diverse opinions by generating and evaluating opinions with minimal human data.

Contribution

The paper presents a novel self-supervised framework that enables LLMs to find agreements among opinions without human annotations, using data generated by the models themselves.

Findings

Fine-tuned LLMs achieve performance comparable to GPT-3 with fewer parameters.

The framework reduces reliance on human-annotated data.

Effective in discovering agreement among diverse opinions.

Abstract

Finding an agreement among diverse opinions is a challenging topic in multiagent systems. Recently, large language models (LLMs) have shown great potential in addressing this challenge due to their remarkable capabilities in comprehending human opinions and generating human-like text. However, they typically rely on extensive human-annotated data. In this paper, we propose Self-Agreement, a novel framework for fine-tuning LLMs to autonomously find agreement using data generated by LLM itself. Specifically, our approach employs the generative pre-trained transformer-3 (GPT-3) to generate multiple opinions for each question in a question dataset and create several agreement candidates among these opinions. Then, a bidirectional encoder representations from transformers (BERT)-based model evaluates the agreement score of each agreement candidate and selects the one with the highest agreement score. This process yields a dataset of question-opinion-agreements, which we use to fine-tune a pre-trained LLM for discovering agreements among diverse opinions. Remarkably, a pre-trained LLM fine-tuned by our Self-Agreement framework achieves comparable performance to GPT-3 with only 1/25 of its parameters, showcasing its ability to identify agreement among various opinions without the need for human-annotated data.