Learning the Markov Decision Process in the Sparse Gaussian Elimination

TL;DR

This paper introduces a novel learning-based approach using Q-Learning to optimize key modules in sparse Gaussian Elimination, aiming to enhance solver performance by framing the problem within a Markov Decision Process.

Contribution

It is the first to connect Gaussian Elimination with Markov Decision Processes and proposes Q-Learning algorithms for core sparse solver modules.

Findings

Improved performance in sparse solver tasks

Effective Q-Learning algorithms for ordering and pivoting

First integration of MDP with Gaussian Elimination

Abstract

We propose a learning-based approach for the sparse Gaussian Elimination. There are many hard combinatorial optimization problems in modern sparse solver. These NP-hard problems could be handled in the framework of Markov Decision Process, especially the Q-Learning technique. We proposed some Q-Learning algorithms for the main modules of sparse solver: minimum degree ordering, task scheduling and adaptive pivoting. Finally, we recast the sparse solver into the framework of Q-Learning. Our study is the first step to connect these two classical mathematical models: Gaussian Elimination and Markov Decision Process. Our learning-based algorithm could help improve the performance of sparse solver, which has been verified in some numerical experiments.