glassoformer: a query-sparse transformer for post-fault power grid voltage prediction

TL;DR

GLassoformer is a new transformer model that uses group Lasso regularization to sparsify queries, making it more efficient and more accurate for post-fault power grid voltage prediction.

Contribution

It introduces a novel transformer architecture with query sparsity via group Lasso, improving efficiency and prediction performance in power grid applications.

Findings

Outperforms benchmark algorithms in accuracy

Demonstrates higher stability in voltage prediction

Reduces computational cost compared to standard transformers

Abstract

We propose GLassoformer, a novel and efficient transformer architecture leveraging group Lasso regularization to reduce the number of queries of the standard self-attention mechanism. Due to the sparsified queries, GLassoformer is more computationally efficient than the standard transformers. On the power grid post-fault voltage prediction task, GLassoformer shows remarkably better prediction than many existing benchmark algorithms in terms of accuracy and stability.