Do Masked Autoencoders Improve Downhole Prediction? An Empirical Study on Real Well Drilling Data

TL;DR

This study empirically evaluates masked autoencoder pretraining for downhole drilling metric prediction, demonstrating significant error reduction and insights into architectural choices using real well data.

Contribution

First empirical assessment of MAE pretraining for downhole drilling prediction, showing its effectiveness over supervised models and analyzing key design factors.

Findings

MAE reduces test MAE by 19.8% compared to supervised GRU.

Latent space width is the most influential architectural parameter.

Masking ratio has negligible effect due to high temporal redundancy.

Abstract

Downhole drilling telemetry presents a fundamental labeling asymmetry: surface sensor data are generated continuously at 1~Hz, while labeled downhole measurements are costly, intermittent, and scarce. Current machine learning approaches for downhole metric prediction universally adopt fully supervised training from scratch, which is poorly suited to this data regime. We present the first empirical evaluation of masked autoencoder (MAE) pretraining for downhole drilling metric prediction. Using two publicly available Utah FORGE geothermal wells comprising approximately 3.5 million timesteps of multivariate drilling telemetry, we conduct a systematic full-factorial design space search across 72 MAE configurations and compare them against supervised LSTM and GRU baselines on the task of predicting Total Mud Volume. Results show that the best MAE configuration reduces test mean absolute error by 19.8\% relative to the supervised GRU baseline, while trailing the supervised LSTM baseline by 6.4\%. Analysis of design dimensions reveals that latent space width is the dominant architectural choice (Pearson $r = -0.59$ with test MAE), while masking ratio has negligible effect, an unexpected finding attributed to high temporal redundancy in 1~Hz drilling data. These results establish MAE pretraining as a viable paradigm for drilling analytics and identify the conditions under which it is most beneficial.