Topology-Aware Layer Pruning for Large Vision-Language Models

TL;DR

This paper introduces a topology-aware layer pruning method for large vision-language models that preserves critical representational transitions using simplicial complexes and zigzag persistent homology, improving efficiency without sacrificing performance.

Contribution

It proposes a novel topology-based framework for adaptive layer pruning in LVLMs, capturing global representation evolution to maintain model effectiveness.

Findings

Outperforms existing pruning methods across various benchmarks.

Effectively preserves critical model transitions during pruning.

Achieves better sparsity-performance trade-offs.

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities in natural language understanding and reasoning, while recent extensions that incorporate visual inputs enable them to process multimodal information. Despite these advances, Large Vision-Language Models (LVLMs) incur substantial computational and memory costs, hindering deployment in resource-constrained scenarios. Existing layer pruning methods typically rely on local similarity metrics or static proxy signals, failing to capture the global and dynamic evolution of representations across model depth, which often leads to the removal of transition-critical layers. To address this limitation, we propose a topology-aware layer pruning framework for LVLMs. Specifically, we represent layer wise hidden states as point clouds and models their evolution using \textit{simplicial complexes}. By leveraging \textit{zigzag persistent homology}, we quantify inter-layer topological consistency and enable adaptive pruning that preserves critical representational transitions. Extensive experiments on diverse multimodal benchmarks demonstrate that the proposed framework consistently outperforms existing pruning methods across a wide range of sparsity ratios. Our code is available at https://github.com/zpc456/TopoVLM.