Krylov Solvers for Interior Point Methods with Applications in Radiation Therapy and Support Vector Machines

TL;DR

This paper explores the use of Krylov subspace iterative solvers, specifically preconditioned conjugate gradient methods, within interior point algorithms for large-scale optimization problems in radiation therapy and support vector machines, demonstrating promising results on real data.

Contribution

It introduces a prototype interior point method employing Krylov solvers with Jacobi preconditioning for large optimization problems, showing potential for GPU acceleration and improved efficiency.

Findings

Jacobi preconditioning improves system conditioning for iterative solving.

The method achieves acceptable accuracy within reasonable time.

Potential for further improvements with advanced preconditioners and GPU use.

Abstract

Interior point methods are widely used for different types of mathematical optimization problems. Many implementations of interior point methods in use today rely on direct linear solvers to solve systems of equations in each iteration. The need to solve ever larger optimization problems more efficiently and the rise of hardware accelerators for general purpose computing has led to a large interest in using iterative linear solvers instead, with the major issue being inevitable ill-conditioning of the linear systems arising as the optimization progresses. We investigate the use of Krylov solvers for interior point methods in solving optimization problems from radiation therapy and support vector machines. We implement a prototype interior point method using a so called doubly augmented formulation of the Karush-Kuhn-Tucker linear system of equations, originally proposed by Forsgren and Gill, and evaluate its performance on real optimization problems from radiation therapy and support vector machines. Crucially, our implementation uses a preconditioned conjugate gradient method with Jacobi preconditioning internally. Our measurements of the conditioning of the linear systems indicate that the Jacobi preconditioner improves the conditioning of the systems to a degree that they can be solved iteratively, but there is room for further improvement in that regard. Furthermore, profiling of our prototype code shows that it is suitable for GPU acceleration, which may further improve its performance in practice. Overall, our results indicate that our method can find solutions of acceptable accuracy in reasonable time, even with a simple Jacobi preconditioner.