AstroMAE: Redshift Prediction Using a Masked Autoencoder with a Novel Fine-Tuning Architecture

TL;DR

AstroMAE introduces a novel masked autoencoder approach with a specialized fine-tuning architecture for redshift prediction, leveraging unlabeled SDSS images to improve accuracy and generalization in astronomical data analysis.

Contribution

This paper presents the first application of masked autoencoders to astronomical data, enhancing redshift prediction through pretraining on unlabeled images and a new fine-tuning architecture.

Findings

AstroMAE outperforms traditional CNN and vision transformer models.

Pretraining with masked autoencoders improves data understanding without labels.

The approach achieves superior accuracy in redshift prediction tasks.

Abstract

Redshift prediction is a fundamental task in astronomy, essential for understanding the expansion of the universe and determining the distances of astronomical objects. Accurate redshift prediction plays a crucial role in advancing our knowledge of the cosmos. Machine learning (ML) methods, renowned for their precision and speed, offer promising solutions for this complex task. However, traditional ML algorithms heavily depend on labeled data and task-specific feature extraction. To overcome these limitations, we introduce AstroMAE, an innovative approach that pretrains a vision transformer encoder using a masked autoencoder method on Sloan Digital Sky Survey (SDSS) images. This technique enables the encoder to capture the global patterns within the data without relying on labels. To the best of our knowledge, AstroMAE represents the first application of a masked autoencoder to astronomical data. By ignoring labels during the pretraining phase, the encoder gathers a general understanding of the data. The pretrained encoder is subsequently fine-tuned within a specialized architecture tailored for redshift prediction. We evaluate our model against various vision transformer architectures and CNN-based models, demonstrating the superior performance of AstroMAEs pretrained model and fine-tuning architecture.