OmniSpectra: A Unified Foundation Model for Native Resolution Astronomical Spectra

TL;DR

OmniSpectra is a pioneering foundation model that processes astronomical spectra at their native resolution, enabling versatile, zero-shot applications across multiple surveys and tasks without resampling or fixed input constraints.

Contribution

It introduces a novel architecture with adaptive patching and encoding techniques that handle variable-length spectra from diverse surveys simultaneously.

Findings

Demonstrates excellent zero-shot generalization across tasks.

Achieves state-of-the-art performance in source classification and redshift estimation.

Reduces need for task-specific model training.

Abstract

We present OmniSpectra, the first native-resolution foundation model for astronomy spectra. Unlike traditional models, which are limited to fixed-length input sizes or configurations, OmniSpectra handles spectra of any length at their original size, without resampling or interpolation. Despite the large-scale spectroscopic data from diverse surveys fueling the rapid growth of astronomy, existing foundation models are limited to a fixed wavelength range and specific instruments. OmniSpectra is the first foundation model to learn simultaneously from multiple real-world spectra surveys with different configurations at a large scale. We achieve this by designing a novel architecture with adaptive patching across variable lengths, sinusoidal global wavelength encoding, local positional embeddings through depthwise convolution, and validity-aware self-attention masks. Allowing us to learn multi-scale spatial patterns while skipping attention for invalid patches. Even with a limited training example, OmniSpectra demonstrates excellent zero-shot generalization compared to methods tailored for specific tasks. This transfer learning capability makes this model the state-of-the-art across various astronomy tasks, including source classification, redshift estimation, and properties prediction for stars and galaxies. OmniSpectra reduces the need for training individual models for different tasks from scratch, establishing itself as the next-generation astronomy foundation model.