Distance Preserving Machine Learning for Uncertainty Aware Accelerator Capacitance Predictions

TL;DR

This paper introduces a distance-preserving neural network approach for uncertainty-aware capacitance prediction in accelerator systems, enhancing Gaussian process approximations with spectral normalization techniques.

Contribution

It compares spectral-normalized layers to SVD for feature extraction, improving distance preservation in neural Gaussian process models for high-dimensional data.

Findings

Achieved less than 1% error in capacitance predictions.

Improved distance preservation over standard neural network layers.

Demonstrated effectiveness in a real accelerator system application.

Abstract

Providing accurate uncertainty estimations is essential for producing reliable machine learning models, especially in safety-critical applications such as accelerator systems. Gaussian process models are generally regarded as the gold standard method for this task, but they can struggle with large, high-dimensional datasets. Combining deep neural networks with Gaussian process approximation techniques have shown promising results, but dimensionality reduction through standard deep neural network layers is not guaranteed to maintain the distance information necessary for Gaussian process models. We build on previous work by comparing the use of the singular value decomposition against a spectral-normalized dense layer as a feature extractor for a deep neural Gaussian process approximation model and apply it to a capacitance prediction problem for the High Voltage Converter Modulators in the Oak Ridge Spallation Neutron Source. Our model shows improved distance preservation and predicts in-distribution capacitance values with less than 1% error.