LLaVA-Critic: Learning to Evaluate Multimodal Models

TL;DR

LLaVA-Critic is an open-source multimodal model that evaluates and guides the performance of other multimodal models, achieving competitive evaluation accuracy and improving model alignment through preference learning.

Contribution

It introduces the first open-source multimodal evaluator capable of assessing diverse tasks and providing reward signals for preference learning.

Findings

Performs on par or better than GPT models in evaluation benchmarks.

Effectively generates reward signals for model alignment.

Demonstrates the potential of open-source LMMs in self-critique.

Abstract

We introduce LLaVA-Critic, the first open-source large multimodal model (LMM) designed as a generalist evaluator to assess performance across a wide range of multimodal tasks. LLaVA-Critic is trained using a high-quality critic instruction-following dataset that incorporates diverse evaluation criteria and scenarios. Our experiments demonstrate the model's effectiveness in two key areas: (1) LMM-as-a-Judge, where LLaVA-Critic provides reliable evaluation scores, performing on par with or surpassing GPT models on multiple evaluation benchmarks; and (2) Preference Learning, where it generates reward signals for preference learning, enhancing model alignment capabilities. This work underscores the potential of open-source LMMs in self-critique and evaluation, setting the stage for future research into scalable, superhuman alignment feedback mechanisms for LMMs.