MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learning

TL;DR

This paper introduces MMDuet2, a proactive video multimodal large language model that autonomously decides when to respond during video playback using multi-turn reinforcement learning, improving response timing and quality.

Contribution

MMDuet2 is the first to apply multi-turn RL for proactive interaction in Video MLLMs, eliminating manual threshold tuning and precise response annotations.

Findings

Outperforms existing proactive Video MLLMs in response timing and quality

Achieves state-of-the-art on ProactiveVideoQA benchmark

Trained on 52k videos with dialogue data

Abstract

Recent advances in video multimodal large language models (Video MLLMs) have significantly enhanced video understanding and multi-modal interaction capabilities. While most existing systems operate in a turn-based manner where the model can only reply after user turns, proactively deciding when to reply during video playback presents a promising yet challenging direction for real-time applications. In this work, we propose a novel text-to-text approach to proactive interaction, where the model autonomously determines whether to respond or remain silent at each turn based on dialogue history and visual context up to current frame of an streaming video. To overcome difficulties in previous methods such as manually tuning response decision thresholds and annotating precise reply times, we introduce a multi-turn RL based training method that encourages timely and accurate responses without requiring precise response time annotations. We train our model MMDuet2 on a dataset of 52k videos with two types of dialogues via SFT and RL. Experimental results demonstrate that MMDuet2 outperforms existing proactive Video MLLM baselines in response timing and quality, achieving state-of-the-art performance on the ProactiveVideoQA benchmark.