WISE: Weighted Iterative Society-of-Experts for Robust Multimodal Multi-Agent Debate
Anoop Cherian, River Doyle, Eyal Ben-Dov, Suhas Lohit, Kuan-Chuan Peng

TL;DR
This paper introduces WISE, a modular multi-agent debate framework for multimodal reasoning that improves accuracy by leveraging heterogeneous experts and a novel aggregation method.
Contribution
The paper presents WISE, a generalized MAD framework for multimodal tasks, incorporating heterogeneous agents and a modified Dawid-Skene aggregation algorithm, advancing multi-agent debate capabilities.
Findings
WISE improves accuracy by 2-7% over state-of-the-art methods.
The framework effectively handles heterogeneous multimodal experts.
Results demonstrate robustness across diverse datasets and LLM configurations.
Abstract
Recent large language models (LLMs) are trained on diverse corpora and tasks, leading them to develop complementary strengths. Multi-agent debate (MAD) has emerged as a popular way to leverage these strengths for robust reasoning, though it has mostly been applied to language-only tasks, leaving its efficacy on multimodal problems underexplored. In this paper, we study MAD for solving vision-and-language reasoning problems. Our setup enables generalizing the debate protocol with heterogeneous experts that possess single- and multi-modal capabilities. To this end, we present Weighted Iterative Society-of-Experts (WISE), a generalized and modular MAD framework that partitions the agents into Solvers, that generate solutions, and Reflectors, that verify correctness, assign weights, and provide natural language feedback. To aggregate the agents' solutions across debate rounds, while…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Topic Modeling · Multi-Agent Systems and Negotiation
