# TrimTokenator: Towards Adaptive Visual Token Pruning for Large Multimodal Models

## TL;DR

This paper introduces TrimTokenator, an adaptive visual token pruning method for large multimodal models that significantly reduces token count and inference time while maintaining performance by selectively removing redundant visual tokens.

## Contribution

It proposes a novel visual token pruning strategy based on mutual information and diversity maximization, explicitly preserving cross-modal alignment and intra-modal information.

## Key findings

- Reduces visual tokens by 88.9%
- Improves inference speed by 56.7%
- Maintains strong model performance

## Abstract

Large Multimodal Models (LMMs) have achieved significant success across various tasks. These models usually encode visual inputs into dense token sequences, which are then concatenated with textual tokens and jointly processed by a language model. However, the increased token count substantially raises computational and memory costs during inference. Token pruning has emerged as a promising approach to address this issue. Existing token pruning methods often rely on costly calibration or suboptimal importance metrics, leading to redundant retained tokens. In this paper, we analyze the redundancy differences between visual and textual tokens and propose pruning exclusively on visual tokens. Based on this, we propose a visual token pruning strategy that explicitly preserves both cross-modal alignment and intra-modal informational diversity. We introduce a mutual information-based token pruning strategy that removes visual tokens semantically misaligned with textual tokens, effectively preserving the alignment between the visual and textual modalities. To further improve the representational quality of the retained tokens, we additionally prune redundant visual tokens by maximizing the expected pairwise distances in the embedding space, which is solved efficiently with a greedy algorithm. Extensive experiments demonstrate that our method maintains strong performance while reducing tokens by 88.9% on models such as LLaVA-1.5-7B and LLaVA-NEXT-7B, resulting in a 56.7% improvement in inference speed.