Solving large linear least squares problems with linear equality constraints

TL;DR

This paper introduces new methods for efficiently solving large-scale linear least squares problems with linear equality constraints, focusing on maintaining sparsity and small residuals, and demonstrates their effectiveness through practical experiments.

Contribution

It proposes novel modifications and solution strategies, including null-space and direct elimination methods, for constrained least squares problems, especially when constraints involve dense rows.

Findings

Null-space method combined with a null space basis algorithm is effective.

Direct elimination with careful pivoting transforms the problem into a sparse-dense least squares problem.

Solution variants using augmented systems are effective for related problem sequences.

Abstract

We consider the problem of efficiently solving large-scale linear least squares problems that have one or more linear constraints that must be satisfied exactly. Whilst some classical approaches are theoretically well founded, they can face difficulties when the matrix of constraints contains dense rows or if an algorithmic transformation used in the solution process results in a modified problem that is much denser than the original one. To address this, we propose modifications and new ideas, with an emphasis on requiring the constraints are satisfied with a small residual. We examine combining the null-space method with our recently developed algorithm for computing a null space basis matrix for a "wide" matrix. We further show that a direct elimination approach enhanced by careful pivoting can be effective in transforming the problem to an unconstrained sparse-dense least squares problem that can be solved with existing direct or iterative methods. We also present a number of solution variants that employ an augmented system formulation, which can be attractive when solving a sequence of related problems. Numerical experiments using problems coming from practical applications are used throughout to demonstrate the effectiveness of the different approaches.