LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-Language Models

TL;DR

This paper introduces LongWriter-V-22k, a dataset and training method enabling vision-language models to generate ultra-long, high-fidelity outputs, significantly improving their long-generation capabilities beyond 1,000 words.

Contribution

The authors present a new dataset, LongWriter-V-22k, and a training approach using IterDPO, to enhance vision-language models' ability to produce coherent, high-quality outputs over thousands of words.

Findings

Model trained with LongWriter-V-22k outperforms larger proprietary models on long-generation tasks.

IterDPO effectively optimizes long output generation without extensive human feedback.

The approach achieves high-fidelity, coherent long outputs up to 10,000 words.

Abstract

Existing Large Vision-Language Models (LVLMs) can process inputs with context lengths up to 128k visual and text tokens, yet they struggle to generate coherent outputs beyond 1,000 words. We find that the primary limitation is the absence of long output examples during supervised fine-tuning (SFT). To tackle this issue, we introduce LongWriter-V-22k, a SFT dataset comprising 22,158 examples, each with multiple input images, an instruction, and corresponding outputs ranging from 0 to 10,000 words. Moreover, to achieve long outputs that maintain high-fidelity to the input images, we employ Direct Preference Optimization (DPO) to the SFT model. Given the high cost of collecting human feedback for lengthy outputs (e.g., 3,000 words), we propose IterDPO, which breaks long outputs into segments and uses iterative corrections to form preference pairs with the original outputs. Additionally, we develop MMLongBench-Write, a benchmark featuring six tasks to evaluate the long-generation capabilities of VLMs. Our 7B parameter model, trained with LongWriter-V-22k and IterDPO, achieves impressive performance on this benchmark, outperforming larger proprietary models like GPT-4o. Code and data: https://github.com/THU-KEG/LongWriter-V