FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment

TL;DR

FlexRank introduces a method to extract importance-ordered nested submodels from pretrained large neural networks, enabling flexible deployment across various computational budgets without retraining, thus improving practicality and efficiency.

Contribution

It proposes a novel low-rank decomposition technique with nested importance-based consolidation for adaptive model deployment from pretrained models.

Findings

Enables cost-performance trade-offs without retraining.

Extracts nested submodels of increasing capabilities.

Facilitates 'train-once, deploy-everywhere' deployment paradigm.

Abstract

The growing scale of deep neural networks, encompassing large language models (LLMs) and vision transformers (ViTs), has made training from scratch prohibitively expensive and deployment increasingly costly. These models are often used as computational monoliths with fixed cost, a rigidity that does not leverage overparametrized architectures and largely hinders adaptive deployment across different cost budgets. We argue that importance-ordered nested components can be extracted from pretrained models, and selectively activated on the available computational budget. To this end, our proposed FlexRank method leverages low-rank weight decomposition with nested, importance-based consolidation to extract submodels of increasing capabilities. Our approach enables a "train-once, deploy-everywhere" paradigm that offers a graceful trade-off between cost and performance without training from scratch for each budget - advancing practical deployment of large models.