SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models

TL;DR

This paper introduces SAFE-SVD, a novel compression method for physics foundation models that preserves physical fidelity by accounting for layer sensitivity, enabling higher compression ratios without sacrificing accuracy.

Contribution

The paper presents a sensitivity-aware, fidelity-enforcing compression framework specifically designed for physics foundation models, addressing the challenge of preserving physical fidelity during compression.

Findings

Achieves significantly higher compression ratios while maintaining accuracy.

Outperforms existing methods across multiple models and datasets.

Potentially enables more efficient and sustainable scientific models.

Abstract

We propose a new method for compressing physics foundation models (PFMs) which is a new trend in AI for Science. While model compression is essential for reducing memory use and accelerating inference in large foundation models, it remains under-explored for PFMs, where preserving physical fidelity is crucial. The challenge lies in the functional nature of physics data, where partial derivatives encode spatiotemporal dynamics and exhibit high sensitivity to compression. Conventional compression methods ignore this structure, often causing severe performance degradation or failure. To address this, we introduce a sensitivity-aware fidelity-enforcing compression framework that explicitly models loss-aware layer sensitivity in the output function space during compression. This provides a new route to compressing scientific foundation models while preserving accuracy and physical fidelity. Experiments show substantial gains over existing methods across multiple models and datasets, achieving significantly higher compression ratios while maintaining accuracy, in some cases by orders of magnitude. More broadly, the work potentially leads to a new subfield of efficient, deployable, and sustainable scientific foundation models in AI for Science.