Brainstorming Brings Power to Large Language Models of Knowledge Reasoning

TL;DR

This paper introduces a multi-model brainstorming approach for large language models, enhancing reasoning accuracy by collaborative discussion, and enabling smaller models to perform comparably to larger ones through iterative consensus-building.

Contribution

It proposes a novel multi-model brainstorming method that improves reasoning accuracy and allows small models to match larger models' performance via collaborative reasoning.

Findings

Brainstorming significantly improves logical reasoning accuracy.

Small models can achieve performance comparable to large models through brainstorming.

The method enhances fact extraction and reasoning stability.

Abstract

Large Language Models (LLMs) have demonstrated amazing capabilities in language generation, text comprehension, and knowledge reasoning. While a single powerful model can already handle multiple tasks, relying on a single perspective can lead to biased and unstable results. Recent studies have further improved the model's reasoning ability on a wide range of tasks by introducing multi-model collaboration. However, models with different capabilities may produce conflicting answers on the same problem, and how to reasonably obtain the correct answer from multiple candidate models has become a challenging problem. In this paper, we propose the multi-model brainstorming based on prompt. It incorporates different models into a group for brainstorming, and after multiple rounds of reasoning elaboration and re-inference, a consensus answer is reached within the group. We conducted experiments on three different types of datasets, and demonstrate that the brainstorming can significantly improve the effectiveness in logical reasoning and fact extraction. Furthermore, we find that two small-parameter models can achieve accuracy approximating that of larger-parameter models through brainstorming, which provides a new solution for distributed deployment of LLMs.