Assessing Color Vision Test in Large Vision-language Models

TL;DR

This paper introduces a new color vision testing framework for large vision-language models, creating a dataset and analyzing their performance to improve their color perception abilities.

Contribution

It defines a novel color vision test for large models, constructs a comprehensive dataset, and proposes fine-tuning strategies to enhance their color recognition capabilities.

Findings

Models exhibit specific error patterns in color recognition.

Fine-tuning improves model performance on color vision tasks.

The dataset enables systematic evaluation of color perception in models.

Abstract

With the widespread adoption of large vision-language models, the capacity for color vision in these models is crucial. However, the color vision abilities of large visual-language models have not yet been thoroughly explored. To address this gap, we define a color vision testing task for large vision-language models and construct a dataset \footnote{Anonymous Github Showing some of the data https://anonymous.4open.science/r/color-vision-test-dataset-3BCD} that covers multiple categories of test questions and tasks of varying difficulty levels. Furthermore, we analyze the types of errors made by large vision-language models and propose fine-tuning strategies to enhance their performance in color vision tests.