EVINCE: Optimizing Multi-LLM Dialogues Using Conditional Statistics and Information Theory

TL;DR

EVINCE is a framework that uses information theory and conditional statistics to dynamically optimize multi-LLM dialogues, balancing diversity and consensus for more effective collaboration.

Contribution

It introduces a novel entropy-based optimization method for regulating LLM behaviors in multi-agent dialogues, addressing previous limitations in behavior modulation and information assessment.

Findings

Promotes diverse, contentious dialogues when mutual information is low.

Facilitates consensus and compromise as mutual information stabilizes.

Provides quantitative tools for dynamic regulation of LLM interactions.

Abstract

EVINCE (Entropy and Variation IN Conditional Exchanges) is a novel framework for optimizing multi-LLM dialogues using conditional statistics and information theory. It addresses limitations in multi-agent debate (MAS) frameworks, where multiple LLMs ``chat'' without behavior modulation or mutual information quality assessment. Using dual entropy optimization to balance perspective diversity and prior knowledge, $\EVINCE$ provides quantitative tools to dynamically regulate LLM linguistic behaviors. When mutual information is low and both cross-entropy and Wasserstein distance are high, EVINCE promotes contentious dialogues to expose diverse perspectives and uncover inconsistencies. Conversely, as cross-entropy decreases and mutual information stabilizes, it transitions discussions into a conciliatory phase, encouraging compromise and acknowledgment of valid points. Using information-theoretic metrics and optimizing mutual information, $\EVINCE$ emerges as a structured and highly effective framework for multi-LLM collaboration.