# MMG-Vid: Maximizing Marginal Gains at Segment-level and Token-level for Efficient Video LLMs

## TL;DR

MMG-Vid introduces a training-free token pruning method for Video LLMs that dynamically allocates tokens based on segment and token-level marginal gains, significantly improving efficiency while preserving performance.

## Contribution

The paper presents a novel, training-free token pruning framework that considers temporal dynamics and inter-frame diversity for efficient video understanding in LLMs.

## Key findings

- Maintains over 99.5% of original performance
- Reduces 75% of visual tokens
- Accelerates prefilling stage by 3.9x

## Abstract

Video Large Language Models (VLLMs) excel in video understanding, but their excessive visual tokens pose a significant computational challenge for real-world applications. Current methods aim to enhance inference efficiency by visual token pruning. However, they do not consider the dynamic characteristics and temporal dependencies of video frames, as they perceive video understanding as a multi-frame task. To address these challenges, we propose MMG-Vid, a novel training-free visual token pruning framework that removes redundancy by Maximizing Marginal Gains at both segment-level and token-level. Specifically, we first divide the video into segments based on frame similarity, and then dynamically allocate the token budget for each segment to maximize the marginal gain of each segment. Subsequently, we propose a temporal-guided DPC algorithm that jointly models inter-frame uniqueness and intra-frame diversity, thereby maximizing the marginal gain of each token. By combining both stages, MMG-Vid can maximize the utilization of the limited token budget, significantly improving efficiency while maintaining strong performance. Extensive experiments demonstrate that MMG-Vid can maintain over 99.5% of the original performance, while effectively reducing 75% visual tokens and accelerating the prefilling stage by 3.9x on LLaVA-OneVision-7B. Code will be released soon.