Dual Hierarchical Least-Squares Programming with Equality Constraints

TL;DR

This paper introduces a dual formulation-based solver for hierarchical least-squares programming with equality constraints, enabling differentiability and faster computation suitable for neural networks and distributed systems.

Contribution

The paper develops a convex, differentiable solver for HLSP with equality constraints using dual formulation and ADMM, significantly improving computational speed over existing methods.

Findings

D-HLSP-E is convex and differentiable.

D-HADM is about ten times faster than interior-point methods.

The solver efficiently handles multiple priority levels in HLSPs.

Abstract

Hierarchical least-squares programming (HLSP) is an important tool in optimization as it enables the stacking of any number of priority levels in order to reflect complex constraint relationships, for example in physical systems like robots. Existing solvers typically address the primal formulation of HLSP's, which is computationally efficient due to sequential treatment of the priority levels. This way, already identified active constraints can be eliminated after each priority level, leading to smaller problems as the solver progresses through the hierarchy. However, this sequential progression makes the solvers discontinuous and therefore not differentiable. This prevents the incorporation of HLSP's as neural network neurons, or solving HLSP's in a distributed fashion. In this work, an efficient solver based on the dual formulation of HLSP's with equality constraints (D-HLSP-E) is developed. D-HLSP-E is a convex and differentiable quadratically constrained least-squares program (QCLSP), which is solved by an Alternating Direction Method of Multipliers (ADMM). By introducing appropriate operator splitting, primal-dual variables, which link each priority level with all their respective higher priority levels, can be eliminated from the main computation step of computing a matrix factorization. The proposed solver D-HADM is about one magnitude faster than a comparable D-HLSP-E solver based on the interior-point method (D-HIPM), where the primal-dual variables enter the computational complexity in a cubic fashion.