Cascaded Self-Evaluation Augmented Training for Lightweight Multimodal LLMs

TL;DR

This paper introduces Cas-SEAT, a novel training method for lightweight multimodal LLMs that enhances self-evaluation and reasoning abilities through cascaded prompts and data filtering, leading to significant performance improvements.

Contribution

The paper proposes Cas-SEAT, a cascaded self-evaluation training approach with double-level data filtering, to improve reasoning and self-evaluation in lightweight multimodal LLMs.

Findings

Cas-SEAT improves model performance by 22.16% on average across datasets.

DDF reduces training resource consumption significantly.

Enhanced self-evaluation capabilities lead to better reasoning accuracy.

Abstract

Efficient Multimodal Large Language Models (EMLLMs) can improve performance through Chain-of-Thought (CoT) reasoning, but they have poor self-evaluation capabilities during the CoT reasoning process. This is due to their tendency to simplify the reasoning process and the degradation of self-evaluation ability during downstream task fine-tuning. To address this, we intuitively propose \textit{Self-Evaluation Augmented Training (SEAT)}, which uses more powerful EMLLMs to evaluate CoT reasoning data. The evaluation data is then used to train EMLLMs. However, due to the difficulties EMLLMs face with processing long token input-output sequences, and the degradation of self-evaluation ability as a basis for CoT reasoning, the SEAT method is not fully adapted. Therefore, we further propose \textit{Cascaded Self-Evaluation Augmented Training (Cas-SEAT)}, which converts long prompts into cascaded short prompts, each focusing on a specific task. Additionally, we mix CoT reasoning and self-evaluation data to preserve its CoT reasoning ability while enhancing the self-evaluation capability of EMLLMs. We also conduct \textit{Double-level Data Filtering (DDF)}, which includes source data filtering and labeled data filtering, using both manual selection and MLLMs for filtering. Cas-SEAT and DDF work together to improve the performance of EMLLMs. Experiments show that Cas-SEAT achieves an average improvement of 22.16% across multiple datasets, and DDF significantly reduces the resource consumption of training