MAX: Masked Autoencoder for X-ray Fluorescence in Geological Investigation

TL;DR

This paper introduces MAX, a self-supervised masked autoencoder model for XRF spectra in geological studies, which improves quantification accuracy and generalizability with limited data, aiding geological discovery.

Contribution

The paper presents MAX, a novel self-supervised learning approach using masked autoencoders for XRF data, enhancing transferability and performance in geological applications with scarce data.

Findings

MAX outperforms non-pretrained models in geochemical quantification.

Model generalizes better by over 60% in zero-shot tests.

Masking 50% of spectra provides effective self-supervision.

Abstract

Pre-training foundation models has become the de-facto procedure for deep learning approaches, yet its application remains limited in the geological studies, where in needs of the model transferability to break the shackle of data scarcity. Here we target on the X-ray fluorescence (XRF) scanning data, a standard high-resolution measurement in extensive scientific drilling projects. We propose a scalable self-supervised learner, masked autoencoders on XRF spectra (MAX), to pre-train a foundation model covering geological records from multiple regions of the Pacific and Southern Ocean. In pre-training, we find that masking a high proportion of the input spectrum (50\%) yields a nontrivial and meaningful self-supervisory task. For downstream tasks, we select the quantification of XRF spectra into two costly geochemical measurements, CaCO$_3$ and total organic carbon, due to their importance in understanding the paleo-oceanic carbon system. Our results show that MAX, requiring only one-third of the data, outperforms models without pre-training in terms of quantification accuracy. Additionally, the model's generalizability improves by more than 60\% in zero-shot tests on new materials, with explainability further ensuring its robustness. Thus, our approach offers a promising pathway to overcome data scarcity in geological discovery by leveraging the self-supervised foundation model and fast-acquired XRF scanning data.