An Introduction to Collective Intelligence
David H. Wolpert, Kagan Tumer

TL;DR
This paper introduces the science of designing collective intelligence systems composed of multiple agents using reinforcement learning, focusing on how to choose reward functions that lead to high overall system utility.
Contribution
It presents a survey of the emerging field of COIN design, highlighting the use of RL algorithms for implicit system optimization and discussing potential applications and future research directions.
Findings
Successful applications in packet-routing and leader-follower problems
Identification of key challenges in reward function design
Potential for expanding to complex real-world systems
Abstract
This paper surveys the emerging science of how to design a ``COllective INtelligence'' (COIN). A COIN is a large multi-agent system where: (i) There is little to no centralized communication or control; and (ii) There is a provided world utility function that rates the possible histories of the full system. In particular, we are interested in COINs in which each agent runs a reinforcement learning (RL) algorithm. Rather than use a conventional modeling approach (e.g., model the system dynamics, and hand-tune agents to cooperate), we aim to solve the COIN design problem implicitly, via the ``adaptive'' character of the RL algorithms of each of the agents. This approach introduces an entirely new, profound design problem: Assuming the RL algorithms are able to achieve high rewards, what reward functions for the individual agents will, when pursued by those agents, result in high…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGame Theory and Applications · Evolutionary Game Theory and Cooperation · Opinion Dynamics and Social Influence
