VQA Training Sets are Self-play Environments for Generating Few-shot Pools
Tautvydas Misiunas, Hassan Mansoor, Jasper Uijlings, Oriana, Riva, Victor Carbune

TL;DR
This paper introduces a method to transform existing VQA training sets into self-play environments, enabling models to autonomously improve their reasoning skills by iteratively selecting training examples that maximize task metrics.
Contribution
The authors propose a novel technique that leverages existing datasets for self-play training, reducing dataset construction costs and enhancing model performance in visual-question answering tasks.
Findings
Models can learn to use themselves or other models as tools.
The approach improves zero-shot performance on various VQA datasets.
Self-play training enhances reasoning capabilities in visual question answering.
Abstract
Large-language models and large-vision models are increasingly capable of solving compositional reasoning tasks, as measured by breakthroughs in visual-question answering benchmarks. However, state-of-the-art solutions often involve careful construction of large pre-training and fine-tuning datasets, which can be expensive. The use of external tools, whether other ML models, search engines, or APIs, can significantly improve performance by breaking down high-level reasoning questions into sub-questions that are answerable by individual tools, but this approach has similar dataset construction costs to teach fine-tuned models how to use the available tools. We propose a technique in which existing training sets can be directly used for constructing computational environments with task metrics as rewards. This enables a model to autonomously teach itself to use itself or another model as…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Testing and Debugging Techniques · Software Engineering Techniques and Practices · Educational Games and Gamification
