PRISM: Structured Optimization via Anisotropic Spectral Shaping

TL;DR

PRISM is a novel optimizer that integrates partial second-order information into spectral descent methods, adaptively shaping updates to improve convergence with minimal overhead.

Contribution

It introduces a low-rank quasi-second-order preconditioner using polar decomposition for anisotropic spectral shaping in spectral optimization.

Findings

Enhances spectral descent methods with curvature-adaptive updates.

Maintains computational efficiency with minimal overhead.

Demonstrates improved optimization performance in practice.

Abstract

We propose PRISM, an optimizer that enhances first-order spectral descent methods like Muon with partial second-order information. It constructs an efficient, low-rank quasi-second-order preconditioner via innovation-augmented polar decomposition. This mechanism enables PRISM to perform anisotropic spectral shaping, which adaptively suppresses updates in high-variance subspaces while preserving update strength in signal-dominated directions. Crucially, this is achieved with minimal computational overhead and zero additional memory compared to first-order baselines. PRISM demonstrates a practical strategy for integrating curvature-adaptive properties into the spectral optimization paradigm.