SoccerRef-Agents: Multi-Agent System for Automated Soccer Refereeing

TL;DR

This paper introduces SoccerRef-Agents, a comprehensive multi-agent system for automated soccer refereeing that combines multimodal data, knowledge bases, and reasoning to improve decision accuracy and explainability.

Contribution

It presents a new holistic framework with a benchmark, knowledge base, and multi-agent architecture for AI-assisted soccer refereeing, advancing beyond isolated perception tasks.

Findings

System significantly outperforms general-purpose MLLMs in decision accuracy.

Achieves higher quality explanations for refereeing decisions.

Provides a new multimodal benchmark and knowledge base for soccer refereeing.

Abstract

Refereeing is vital in sports, where fair, accurate, and explainable decisions are fundamental. While intelligent assistant technologies are being widely adopted in soccer refereeing, current AI-assisted approaches remain preliminary. Existing research mostly focuses on isolated video perception tasks and lacks the ability to understand and reason about foul scenarios. To fill this gap, we propose SoccerRef-Agents, a holistic and explainable multi-agent decision-making framework for soccer refereeing. The main contributions are: (i) constructing the multimodal benchmark SoccerRefBench with over 1,200 referee theory questions and 600 foul video clips; (ii) building a vector-based knowledge base RefKnowledgeDB using the latest "Laws of the Game" and a classic case database for precise, knowledge-driven reasoning; (iii) designing a novel multi-agent architecture that collaborates via cross-modal RAG to bridge the semantic gap between visual content and regulatory texts. This work explores the technical capability of integrating MLLMs with refereeing expertise, and evaluations show our system significantly outperforms general-purpose MLLMs in decision accuracy and explanation quality. All databases, benchmarks, and code will be made available.