Fine-Tuning Qwen 2.5 3B for Realistic Movie Dialogue Generation

TL;DR

This paper presents the fine-tuning of the Qwen 2.5 3B model on movie dialogue data, achieving realistic and engaging conversations suitable for real-time applications, while addressing computational constraints.

Contribution

It introduces a fine-tuning approach for small open-source models on movie dialogue, demonstrating high-quality output despite limited GPU resources.

Findings

Small models can generate realistic movie dialogue.

Progressive scaling improves training efficiency.

Qwen 2.5 outperforms some larger models in creative tasks.

Abstract

The Qwen 2.5 3B base model was fine-tuned to generate contextually rich and engaging movie dialogue, leveraging the Cornell Movie-Dialog Corpus, a curated dataset of movie conversations. Due to the limitations in GPU computing and VRAM, the training process began with the 0.5B model progressively scaling up to the 1.5B and 3B versions as efficiency improvements were implemented. The Qwen 2.5 series, developed by Alibaba Group, stands at the forefront of small open-source pre-trained models, particularly excelling in creative tasks compared to alternatives like Meta's Llama 3.2 and Google's Gemma. Results demonstrate the ability of small models to produce high-quality, realistic dialogue, offering a promising approach for real-time, context-sensitive conversation generation.