OneComp: One-Line Revolution for Generative AI Model Compression

TL;DR

OneComp is an open-source framework that automates and optimizes model compression for generative AI, making deployment more efficient across different hardware.

Contribution

It introduces a resource-adaptive, reproducible pipeline that automates mixed-precision quantization and refinement stages for model compression.

Findings

Automates model inspection and quantization planning based on hardware.

Progressively refines model compression while maintaining performance.

Bridges research and production with an extensible, hardware-aware pipeline.

Abstract

Deploying foundation models is increasingly constrained by memory footprint, latency, and hardware costs. Post-training compression can mitigate these bottlenecks by reducing the precision of model parameters without significantly degrading performance; however, its practical implementation remains challenging as practitioners navigate a fragmented landscape of quantization algorithms, precision budgets, data-driven calibration strategies, and hardware-dependent execution regimes. We present OneComp, an open-source compression framework that transforms this expert workflow into a reproducible, resource-adaptive pipeline. Given a model identifier and available hardware, OneComp automatically inspects the model, plans mixed-precision assignments, and executes progressive quantization stages, ranging from layer-wise compression to block-wise refinement and global refinement. A key architectural choice is treating the first quantized checkpoint as a deployable pivot, ensuring that each subsequent stage improves the same model and that quality increases as more compute is invested. By converting state-of-the-art compression research into an extensible, open-source, hardware-aware pipeline, OneComp bridges the gap between algorithmic innovation and production-grade model deployment.