Iterative Self-Improvement of Vision Language Models for Image Scoring and Self-Explanation

TL;DR

This paper introduces an iterative self-training approach for Vision Language Models to improve image scoring and generate natural language justifications, enhancing both accuracy and interpretability without external data.

Contribution

It presents a novel self-improvement training method for VLMs that enhances image scoring and explanation generation using only internal datasets and iterative optimization.

Findings

Improved image scoring accuracy.

Enhanced coherence of generated explanations.

Effective self-training without external data.

Abstract

Image scoring is a crucial task in numerous real-world applications. To trust a model's judgment, understanding its rationale is essential. This paper proposes a novel training method for Vision Language Models (VLMs) to generate not only image scores but also corresponding justifications in natural language. Leveraging only an image scoring dataset and an instruction-tuned VLM, our method enables self-training, utilizing the VLM's generated text without relying on external data or models. In addition, we introduce a simple method for creating a dataset designed to improve alignment between predicted scores and their textual justifications. By iteratively training the model with Direct Preference Optimization on two distinct datasets and merging them, we can improve both scoring accuracy and the coherence of generated explanations.