Multi-Agent VQA: Exploring Multi-Agent Foundation Models in Zero-Shot Visual Question Answering

TL;DR

This paper introduces a multi-agent system for zero-shot visual question answering that leverages foundation models with specialized agents, aiming to improve robustness and practicality without dataset fine-tuning.

Contribution

It proposes a novel adaptive multi-agent framework for zero-shot VQA, addressing limitations of foundation models in object detection and counting.

Findings

Preliminary results demonstrate potential in zero-shot scenarios

System shows robustness without dataset fine-tuning

Identifies failure cases to guide future research

Abstract

This work explores the zero-shot capabilities of foundation models in Visual Question Answering (VQA) tasks. We propose an adaptive multi-agent system, named Multi-Agent VQA, to overcome the limitations of foundation models in object detection and counting by using specialized agents as tools. Unlike existing approaches, our study focuses on the system's performance without fine-tuning it on specific VQA datasets, making it more practical and robust in the open world. We present preliminary experimental results under zero-shot scenarios and highlight some failure cases, offering new directions for future research.