Quaternion Matrix Optimization and The Underlying Calculus

TL;DR

This paper develops calculus tools like derivatives and subdifferentials for real functions of quaternion matrices, enabling advanced optimization models in color image processing and engineering.

Contribution

It introduces first and second order derivatives, generalized subdifferentials, and optimality conditions for quaternion matrix functions, advancing the mathematical foundation for quaternion matrix optimization.

Findings

Established derivatives and calculus rules for quaternion matrix functions.

Analyzed optimality conditions for sparse low-rank color image denoising.

Proved the closedness and semi-algebraic nature of low-rank quaternion matrix sets.

Abstract

Optimization models involving quaternion matrices are widely used in color image process and other engineering areas. These models optimize real functions of quaternion matrix variables. In particular, $\ell_0$-norms and rank functions of quaternion matrices are discrete. Yet calculus with derivatives, subdifferentials and generalized subdifferentials of such real functions is needed to handle such models. In this paper, we introduce first and second order derivatives and establish their calculation rules for such real functions. Our approach is consistent with the subgradient concept for norms of quaternion matrix variables, recently introduced in the literature. We develop the concepts of generalized subdifferentials of proper functions of quaternion matrices, and use them to analyze the optimality conditions of a sparse low rank color image denoising model. We introduce R-product for two quaternion matrix vectors, as a key tool for our calculus. We show that the real representation set of low-rank quaternion matrices is closed and semi-algebraic. We also establish first order and second order optimality conditions for constrained optimization problems of real functions in quaternion matrix variables.