Efficient evaluation of noncommutative polynomials using tensor and noncommutative Waring decompositions
Eric Evert, J. William Helton, Shiyuan Huang, Jiawang Nie
TL;DR
This paper explores efficient evaluation methods for noncommutative polynomials using tensor and Waring decompositions, aiming to reduce matrix multiplications and improve computational speed.
Contribution
It introduces a noncommutative Waring decomposition framework, compares it with classical approaches, and provides methods for computing these decompositions to enhance evaluation efficiency.
Findings
Decomposition reduces matrix multiplications needed for evaluation.
Comparison shows noncommutative polynomials differ from commutative ones in decomposability.
Proposed methods improve evaluation speed for generic noncommutative polynomials.
Abstract
This paper analyses a Waring type decomposition of a noncommuting (NC) polynomial with respect to the goal of evaluating efficiently on tuples of matrices. Such a decomposition can reduce the number of matrix multiplications needed to evaluate a noncommutative polynomial and is valuable when a single polynomial must be evaluated on many matrix tuples. In pursuit of this goal we examine a noncommutative analog of the classical Waring problem and various related decompositions. For example, we consider a "Waring decomposition" in which each product of linear terms is actually a power of a single linear NC polynomial or more generally a power of a homogeneous NC polynomial. We describe how NC polynomials compare to commutative ones with regard to these decompositions, describe a method for computing the NC decompositions and compare the effect of various decompositions on the…
| no. of evals. for LPS to break even vs. | generic | tensor decomp. | |||||
| (g,d) | Horner | naive | tensor | time | rel. | ||
| rank | (s) | error | |||||
| (3,3)444The space is defective and the generic rank for tensors of this size is rather than the expected . | 9,000 | 430 | 409 | 20 | 5 | 0.025 | |
| (4,4) | 54,751 | 2,618 | 1,856 | 89 | 20 | 1.85 | |
| (8,4) | 139,136 | 6,654 | 1,853 | 89 | 142 | 30.9 | |
| (5,5) | 290,994 | 13,916 | 4,498 | 215 | 149 | 75.3 | |
| (3,6) | 171,110 | 8,183 | 3,972 | 190 | 57 | 18.8 | |
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTensor decomposition and applications · Matrix Theory and Algorithms · graph theory and CDMA systems
Efficient evaluation of noncommutative polynomials using tensor and noncommutative Waring decompositions
Eric Evert1
Eric Evert, Group Science, Engineering and Technology
KU Leuven Kulak,
E. Sabbelaan 53, 8500 Kortrijk, Belgium
and
Electrical Engineering ESAT/STADIUS
KU Leuven,
Kasteelpark Arenberg 10, 3001 Leuven, Belgium
,
J. William Helton1
J. William Helton, Department of Mathematics
University of California
San Diego
,
Shiyuan Huang1
Shiyuan Huang, Department of Computer Science
Columbia University
New York City
and
Jiawang Nie
Jiawang Nie, Department of Mathematics
University of California
San Diego
Abstract.
This paper analyses a Waring type decomposition of a noncommuting (NC) polynomial with respect to the goal of evaluating efficiently on tuples of matrices. Such a decomposition can reduce the number of matrix multiplications needed to evaluate a noncommutative polynomial and is valuable when a single polynomial must be evaluated on many matrix tuples.
In pursuit of this goal we examine a noncommutative analog of the classical Waring problem and various related decompositions. For example, we consider a “Waring decomposition” in which each product of linear terms is actually a power of a single linear NC polynomial or more generally a power of a homogeneous NC polynomial. We describe how NC polynomials compare to commutative ones with regard to these decompositions, describe a method for computing the NC decompositions and compare the effect of various decompositions on the speed of evaluation of generic NC polynomials.
Key words and phrases:
noncommutative polynomials, Waring problem, sums of powers, matrix variables, symmetric tensors
2010 Mathematics Subject Classification:
Primary 11P05, 46L52. Secondary 15A69, 47A56
1Research supported by the NSF grant DMS-1500835
1. Introduction
This paper concerns decompositions of noncommutative polynomials as sums of products of linear polynomials. The goal is to find ways of quickly evaluating noncommutative polynomials on tuples of matrices.
A place where efficient evaluations matter comes in numerical solution of problems arising in linear systems and control. Problems which are completely specified by signal flow diagrams having signals all take the form of solving collections of matrix inequalities based on polynomial matrix inequalities. For example, see [CHS06].
After changes of variables, some basic problems of this type convert to solving Linear Matrix Inequalities (whose coefficients are functions of the given system parameters) and for these there are numerous numerical optimization schemes [WSV00]. As with all optimization algorithms these require very many function evaluations.
[CHS06] showed how, using NC symbolic software, one could produce optimization algorithms whose linear subproblem has coefficients which are NC polynomials in the current iterate . As progresses toward the optimum, many function evaluations of NC polynomials are required.
The striking fact is that the NC polynomials which must be evaluated depend only on the signal flow diagram and on the numerical optimization algorithm in the package. They do not depend on what is being designed, e.g.. a ship controller, airplane controller or helicopter controller (not to mention which ship, which plane etc).
Thus in the lifetime of a popular software toolbox a few specific polynomials must be evaluated billions (at least) of times on matrices of various sizes.
Pursuits involving noncommutative polynomials are in the spirit of the burgeoning area called free analysis. Here one takes classical problems and works out analogues with noncommutative variables, which are free of constraints. These free analogues typically have interpretations for matrix or operator variables and their development often impacts various areas.
One of the original efforts here was Voiculescu’s free probability, which started by developing a notion of entropy for operator variables and which has a become a big area having many associations to random matrix theory, [MS17]. Some other directions are free analytic function theory, cf. [KVV14] and free real algebraic geometry [BKP16] with some consequences for system engineering being [HMPV09]. Our paper concerns and gives applications for the noncommutative variant of the classical Waring problem.
1.1. Noncommutative polynomials
We work with functions of noncommutative variables
[TABLE]
and are interested in powers of linear functions
[TABLE]
where is an index and for .
For any (index) tuple , where for are integers between 1 and g, we denote
[TABLE]
We say the monomial has degree . For example, if , then is a degree monomial.
A noncommutative (NC) polynomial is a formal sum of the form
[TABLE]
where or for each and only finitely many of the are nonzero. The degree of a NC polynomial is equal to that of its highest degree monomial which has a nonzero coefficient. If all monomials of a NC polynomial with nonzero coefficients have the same degree, then the NC polynomial is homogeneous.
Let be a noncommutative polynomial in noncommutative variables. Then for any and for any -tuple of matrices , we define the evaluation of on by
[TABLE]
where . A question of practical interest is how to efficiently evaluate a NC polynomial on a collection of matrix tuples.
In this article we show that tensor decompositions may be used to significantly reduce the number of matrix multiplications needed to evaluate a noncommutative polynomial. Here a tensor is a multiindexed array with entries where is a -tuple of integers between and .
Our general strategy is as follows. First one associates a homogeneous noncommutative polynomial to a tensor . By computing the tensor decomposition of the associated tensor, one gets a decomposition that expresses the NC polynomial as a sum of products of linear terms. This reduces the number of matrix multiplications needed to evaluate .
The nonhomogeneous setting can easily be handled can easily be handled by sorting as a sum of homogeneous polynomials. Additionally, one could homogenize the polynomials with a dummy variable (say ), then replace with after a factorization is obtained.
1.1.1. Evaluation using tensor decompositions
Let
[TABLE]
be a homogeneous degree noncommutative polynomial in variables . We can associate to the tensor . Suppose that has a rank decomposition
[TABLE]
for each and . Then we have
[TABLE]
We call a decomposition of the form (1.1.2) a linear product sum for the NC polynomial . Additionally, if is as small as possible, we say has product sum rank . Before continuing we give an example.
1.1.2. Example
Consider the noncommutative polynomial
[TABLE]
Think of its coefficients for as entries of a tensor with frontal slices
[TABLE]
and
[TABLE]
where is the standard Matlab index notation. One can check that has the rank 2 decomposition
[TABLE]
It follows from (1.1.4) that has the rank linear product sum decomposition
[TABLE]
which one can check using NCAlgebra [OHMS17].
In this case, evaluating as it is written in equation (1.1.3) requires matrix multiplications and matrix additions. However, using equation (1.1.5) one needs only matrix multiplications and matrix additions, so our complexity is reduced by an order of .
As we illustrate later, a low rank tensor decomposition like (1.1.4) can be computed by standard numerical software packages such as Tensorlab. Accuracy of the decompositions will be discussed in section 2.3.4.
1.1.3. A basic NC Horner method
One may also evaluate a NC polynomial using a basic extension of Horner’s method to the NC setting. Given a degree NC polynomial in variables, one may first write
[TABLE]
where is a constant and the degree of is less than for each . One may then recursively apply this method to each until all polynomials appearing in the summation have degree equal to one. For example, for the polynomial in equation (1.1.3), is equal to
[TABLE]
Writing in this form allows to be evaluated using matrix multiplications and matrix additions, thus this method offers a significant improvement over naive evaluation. While this basic Horner method greatly improves on naive evaluation, the linear product sum decomposition for this NC polynomial is still notably more efficient.
Section 2.2 contains a more detailed comparison of the computational complexity of these three methods for generic homogeneous NC polynomials. We thank a referee for urging us to compare this method to linear product sums. Schrempf in [S19] subsequent to this paper introduced an interesting and natural method for evaluation. It heavily uses ‘linear system realizations’, known in the algebra community as ‘linearization’ or the ‘linearization trick’.
1.2. Waring decompositions of noncommutative polynomials
The case where a homogeneous noncommutative polynomial can be expressed as a sum of powers of linear forms adds further advantage for efficient numerical evaluation, as the th power of a matrix can be computed more efficiently than the product of matrices. This calls for a natural noncommutative generalization of the classical Waring problem.
The problem is as follows: Given a NC polynomial in the NC indeterminates , determine if there exist linear functions
[TABLE]
such that
[TABLE]
where or for . We call a decomposition of the form (1.2.1) a rank real (resp. complex) Waring decomposition of . If is as small as possible then we say has Waring rank .
In the spirit of the NC Waring problem, we also consider the more general problem of determining if a homogeneous NC polynomial of degree can be decomposed as a sum of th powers of homogeneous degree NC polynomials. That is, supposing is a homogeneous degree NC polynomial, we wish to determine if there are homogeneous degree polynomials
[TABLE]
such that
[TABLE]
where each is in or . We call a decomposition of the form (1.2.2) a rank real (resp. complex) -Waring decomposition of or sometimes a general Waring decomposition.
The NC Waring problem reduces to the classical commutative variable Waring problem, thereby effectively solving it over . In a similar spirit, we reduce the NC general Waring problem to a classical Waring problem, but in more variables, see Section 4.
1.3. The NC Waring decomposition
Before stating a result we need a definition. Define an indicator function on an index -tuple by first defining
[TABLE]
Then the indicator function which gives the number of ’s appearing in is
[TABLE]
We caution the reader that the superscript appearing on the indicator function is not interpreted as a power.
A corollary for of Theorem 3.5 is:
Corollary 1.1**.**
Suppose a NC homogeneous polynomial , where , satisfies for any index sets such that for all . Then has a NC complex coefficient Waring decomposition with linear powers. Moreover, for a generic NC homogeneous polynomial, the number of terms needed is
[TABLE]
except in the cases
- •
, where terms are needed
- •
* where terms are needed.*
Proof.
This corollary is a combination of Theorem 3.5, the main result in Section 3.3.2, and the solutions for the classical Waring Problem [AH95, OO12]. ∎
Here the term generic means that the set of exceptions is contained in a proper closed algebraic variety, i.e., in the zero set of a nontrivial system of polynomial equations.
Each term in a Waring decomposition of a NC polynomial can be evaluated by computing the th power of a matrix rather than computing the product of different matrices. This gives Waring decompositions an additional computational advantage over linear product sum decompositions when the number of terms needed for each decomposition is the same as one typically expects.
The authors thank Ignat Domanov for discussion related to NC Waring decompositions and efficient polynomial evaluations.
1.4. Guide to readers
In Section 2 we examine in more detail the use of linear product sum decompositions to evaluate NC polynomials on matrix variables. We then discus computation of NC Waring and linear product sum decompositions. Additionally we estimate the expected computational savings when evaluating a NC polynomial using one of these decompositions and provide timing comparisons for naive evaluation and evaluation using and linear product sum decompositions.
Section 3 shows that the NC Waring problem reduces to the classical Waring problem. The section begins by introducing a compatibility condition which is necessary for a NC homogeneous polynomial to have a Waring decomposition. Theorem 3.5 shows that a NC homogeneous polynomial has a -term Waring decomposition if and only if it satisfies our compatibility condition and its commutative collapse has a -term Waring decomposition.
Section 4 considers the general NC Waring problem. Similar to the case, we begin by introducing a general -compatibility condition which is necessary for the existence of a -NC Waring decomposition. Theorem 4.9 shows that, under the -compatibility condition, the general NC Waring problem is equivalent to a commutative Waring problem for a polynomial with an increased number of variables. We end with Section 4.5 which illustrates that an increase in our number of variables is necessary to reduce the general NC Waring decomposition to a commutative Waring decomposition.
2. Accelerating NC polynomial evaluation using tensor and Waring decompositions
In this section we will establish a connection between general tensor decompositions and decompositions of noncommutative polynomials. Using this connection we describe how to use tensor decomposition to efficiently evaluate noncommutative polynomials on matrix variables. Having discussed general tensor decompositions in the introduction, we first consider polynomials with a NC Waring decomposition. Next in Section 2.2 we compare the computational cost of using the various decompositions. Also we discus issues of accuracy.
2.1. NC Waring decompositions and symmetric tensors
It is well known that the classical polynomial Waring problem is equivalent to the problem of symmetric tensor decomposition. Let be a symmetric tensor, i.e. a symmetric multiidexed array, with entries where is a -tuple of integers between and . Here symmetric means that for any permutation , we have where . We may associate to a homogeneous degree d polynomial in the commutative variables by setting
[TABLE]
Suppose has rank symmetric tensor decomposition
[TABLE]
Here for each . Then it is straightforward to check that
[TABLE]
That is, a rank symmetric tensor decomposition of corresponds to a rank Waring decomposition for . By reversing this correspondence one sees that a rank Waring decomposition for a homogeneous polynomial gives a rank symmetric tensor decomposition for the associated symmetric tensor. The fact that the tensor corresponding to a NC polynomial with a Waring decomposition is symmetric is a consequence of Theorem 3.5.
2.1.1. Numerical computation of NC Waring decompositions
We now give an example which computes an NC Waring decomposition by using popular tensor decomposition software. Consider the homogeneous noncommutative polynomial
[TABLE]
We associate to the symmetric tensor defined by its frontal slices
[TABLE]
and
[TABLE]
Using Tensorlab111A matlab script which computes this decomposition using Tensorlab is avaliable on GitHub at https://github.com/NCAlgebra/UserNCNotebooks. [VDSBL16] we compute that is a rank tensor and has symmetric tensor decomposition
[TABLE]
where
[TABLE]
and
[TABLE]
It follows that has the rank 4 NC Waring decomposition
[TABLE]
This is easy to numerically verify using NCAlgebra [OHMS17].
A naive evaluation of on a matrix tuple using the original definition of requires matrix multiplications. In contrast, evaluating on a matrix tuple using its NC Waring decomposition only requires matrix multiplications.
2.2. Computational savings
We now examine the computational costs for each of the the Waring, linear product sum, and basic Horner methods for NC polynomial evaluation.
2.2.1. Linear product sum
The maximum rank of a tensor is not known, however it is conjectured [AOP09] that the rank of a generic tensor is equal to
[TABLE]
except in a small number of defective spaces where most commonly one additional term is needed.
Each term in the linear product sum decomposition is an NC monomial of degree and may be evaluated in multiplications. Therefore, if this conjecture holds, then it follows that generic homogeneous noncommutative polynomials of degree in variables may be evaluated using approximately
[TABLE]
matrix multiplications.
2.2.2. Waring
We now consider the case where has a -term degree NC Waring decomposition as in equation (1.2.1). In this case, for any matrix tuple we may evaluate using matrix additions and matrix exponentiations of degree , where for generic NC polynomials by Corollary 1.1. We note that powers of a matrix may be efficiently computed either by decomposing the exponent as a sum of powers of two, or by first computing the Jordan form of the matrix.
Using repeated squaring methods, a matrix exponentiation of degree can be evaluated with at most matrix multiplications. In addition, using Stirling’s approximation one can show that
[TABLE]
It follows that if an NC polynomial has an NC Waring decomposition, then one may evaluate on matrix variables using approximately
[TABLE]
matrix multiplications. Here .
2.2.3. Horner’s method
Using equation (1.1.6), one sees that if denotes the number of matrix multiplications needed to evaluate a degree NC polynomial in variables using this basic Horner method, then
[TABLE]
Using , one then has
[TABLE]
with equality for generic NC polynomials.
2.2.4. The case
In the case that is a homogeneous NC polynomial of degree in variables, the tensor corresponding to is in fact a matrix. It follows that has linear product sum rank less than or equal to , hence may be evaluated using at most matrix multiplications. Horner’s method also generically requires matrix multiplications in the case, while naive evaluation generically requires matrix multiplications.
2.3. Comparison of computational costs
In this subsection we compare computational costs for the various methods.
2.3.1. Comparison of efficiency: Linear product sum vs. Horner
We now briefly compare the various methods. Supposing that the generic rank of a tensor is in fact given by equation (2.2.1) and using the approximation in equation (2.2.2), one finds that for generic homogeneous NC polynomials, the basic Horner method requires approximately
[TABLE]
more matrix multiplications to evaluate a NC polynomial than the linear product sum method. The above shows that evaluation with linear product sum is more efficient for generic homogeneous NC polynomials than evaluation with Horner for all provided . This increased efficiency leads to a notable improvement for NC polynomials requiring millions of evaluations, see Table 1.
While the more practical point is that linear product sum is more efficient than Horner for all fixed , it gives perspective to look at extremes of ratios. The asymptotic ratio of the number of matrix multiplications needed by linear product sum to that of Horner approaches as tends to infinity. Thus, for high degree homogeneous NC polynomials requiring smaller numbers of evaluations, Horner is likely more appropriate due to the computational cost associated with computing a linear product sum decomposition. In contrast, for fixed this ratio approaches as tends to infinity.
2.3.2. Comparison of efficiency: Waring vs. linear product sum
The main advantage of a Waring decomposition compared to linear product sum is that a th power of a linear form may be evaluated (by repeated squaring) using no more than matrix multiplications. In contrast, the product of distinct linear forms naively requires matrix multiplications to evaluate.
The rank of a tensor is necessarily less than or equal to the symmetric rank of a tensor. It follows that if has a Waring decomposition, hence the corresponding tensor is symmetric, then the ratio of the number of matrix multiplications needed by the Waring method and the linear product sum method is bounded below by
[TABLE]
with equality if the rank of is equal to the symmetric rank of .
An example of a tensor whose rank is strictly less than its symmetric rank has only recently been produced [S18]. The example is of a symmetric tensor of size with rank 903 and symmetric rank greater than 903. The corresponding NC polynomial is a homogeneous degree polynomial in variables.
Since the degree of is , in both the Waring method and the linear product sum method, each monomial requires matrix multiplications to evaluate. As a consequence, in this example, using the linear product sum decomposition allows for to be evaluated in strictly fewer matrix multiplications than the Waring decomposition.
Although an example of a tensor with rank less than symmetric rank is known, there are various results showing that rank is equal to symmetric rank for generic tensors having small rank, e.g. see [COV17, F16]. Additionally, we note that it remains unknown if generic symmetric tensors have rank equal to symmetric rank.
In the case , there is no advantage of using a Waring decomposition over a linear product sum decomposition in terms of number of multiplications required for evaluation. However, we expect that as grows large, even if there is a gap between the Waring rank and linear product sum rank of a given NC polynomial, a NC Waring decomposition will outperform a linear product sum decomposition in terms of efficiency due to the ability to efficiently evaluate matrix powers.
It is also worth pointing out that the generic symmetric rank for symmetric tensors in is strictly less than the generic rank for arbitrary tensors in provided , with the gap becoming increasingly significant as and grow. In contrast, Horner’s method sees no notable improvement when used on NC polynomials which have a Waring decomposition. Thus both Waring and linear product sum decompositions significantly outperform Horner’s method in this setting.
2.3.3. Comparison to naive
All three methods offer a serious improvement over naive evaluation. Since a naive evaluation of a single degree NC monomial requires matrix multiplications, the naive approach to evaluating a NC polynomial on a matrix tuple generically requires
[TABLE]
matrix multiplications. It follows that for a NC polynomial with linear product sum rank given by equation (2.2.1), the ratio of the number of matrix multiplications used in the linear product sum method to those in the naive method is approximately
[TABLE]
Similarly, for NC polynomials with a Waring decomposition, the ratio of the number of matrix multiplications used in the Waring method to those in the naive method is then approximately bounded above by
[TABLE]
a quantity that rapidly approaches zero as or increase, provided .
2.3.4. Accuracy of computations
The tensor in Example 1.1.3 has a unique rank decomposition (up to scaling) which can be shown using Kruskal’s condition for uniqueness of tensor decompositions [K77]. Indeed, when the example is treated with Tensorlab a rank decomposition which is the same (up to scaling) as the decomposition in (1.1.4) is produced.
For display purposes in equation (2.1.1) and above we have truncated the coefficients in the decompositions for and at the thousandths place which leads to a small round off error. If we use the long form coefficients computed by Tensorlab, then the decomposition for and is highly accurate. Note that has infinitely many rank tensor decompositions. The computed tensor decomposition depends on the initialization of the algorithm used in the computation.
Although highly accurate decompositions can be computed for small tensors, when working with large tensors of generic rank, one should not expect to exactly compute a tensor decomposition. However, in early steps of noncommutative optimization algorithms, a small amount of error in the computed descent directions is unlikely to cause serious difficulty. Exact evaluations may be used in later steps when near an optimum. Amounts of relative error averaged over our experiments in tensor decompositions for tensors of the various selected and are reported in Table 1.
2.3.5. Experiment comparing run times of linear product sum to other methods
We now give a brief illustration of experimental timing where we for evaluating homogeneous NC polynomials on and matrices using linear product sums , Horner, and naive evaluation.
Table 1 selects several values of and , in column , and presents properties of the tensor decomposition in the space in the last columns: generic tensor rank, time to find a decomposition, and accuracy of the decomposition. This is the tensor decomposition used for the linear product sum method.
Columns and list how many polynomial evaluations are needed for linear product sum to overcome its tensor decomposition cost, and hence to outperform Horner’s method222The cost of computing a Horner decomposition is assumed to be negligible in this comparison.. Similarly, columns and show when linear product sum breaks even with the naive method333The estimates are generated as follows: We randomly generate pairs of matrices and compute the average amount of time needed for a single multiplication of a pair matrices. The number of matrix multiplications needed for a generic rank linear product sum evaluation or a naive evaluation is multiplied by the average amount of time needed for a single matrix multiplication to compute the expected time needed for a single evaluation on matrices. Using this methodology the average time needed for multiplication of a pair of matrices or matrices was found to be seconds or seconds, respectively..
In the case that a NC polynomial has low Waring or linear product sum rank, evaluation using these methods will be much more efficient. Also, the tensor decomposition needed to compute the NC polynomial decomposition takes significantly less time to compute and the error in the decomposition will be significantly lower.
3. The noncommutative Waring problem
In this section we examine when a noncommutative polynomial has a NC Waring decomposition. Two approaches are considered. First we consider a noncommutative algebra approach. In this approach, we show that if a noncommutative polynomial has a Waring decomposition, then its coefficients must satisfy a compatibility condition. If this condition is satisfied, then we prove that has a -term Waring decomposition if and only if the restriction of to commuting variables has a classical -term Waring decomposition.
The second approach makes use of identification of noncommutative polynomials and tensors and known results for tensor decompositions. To an expert in both tensor theory and in NC polynomials the use of this approach and results on NC Waring decompositions may not come as a surprise. However, for our (main) NC polynomial audience we include a self contained NC polynomial proof.
Before proceeding with proofs we briefly discuss the history of the polynomial Waring problem.
3.1. History of the Waring decomposition
The polynomial Waring problem concerns the question whether a given polynomial, , can be represented by sums of powers of polynomials, where ’s are variables which commute. In this form, the Waring problem is closely related to symmetric tensor decomposition. The polynomial Waring problem for powers of linear forms was treated successfully in [AH95] and subsequently in [RS00] and [FOS12] and has been studied extensively, as is shown, for example, in [BC13] and [GV08].
3.2. A basic definition
Noncommutative Waring decompositions are associated with commutative Waring decompositions through a correspondence we now describe.
For a NC polynomial , the associated commutative collapse, , is the commutative polynomial obtained by considering the variables of to be commutative. Our notation for commutative collapse for a NC monomial is . For example, when , collapses to .
We impose an equivalence relation on NC monomials by saying that and are commutative equivalent if they have the same commutative collapse:
[TABLE]
Moreover, we say two index tuples and are commutative equivalent, denoted , iff . Note that
[TABLE]
3.3. NC polynomial proof of the NC Waring decomposition
Our presentation contains two parts. First we state a compatibility condition necessary for the existence of a Waring decomposition, §3.3.1. Second, if the compatibility condition holds, we reduce the NC Waring problem to the classical commutative Waring problem, §3.3.2.
3.3.1. The Compatibility Condition
As we next see the following condition is necessary for existence of a NC Waring decomposition.
Definition 3.1**.**
We say a noncommutative homogeneous degree polynomial
[TABLE]
satisfies the compatibility condition if
[TABLE]
Sometimes we say that is compatible. ∎
We note that a noncommutative homogeneous polynomial satisfies the compatibility condition if and only if the corresponding tensor described in Section 1.1.1 is symmetric. To see this, given a tuple of length and a permutation define
[TABLE]
It is straight forward to check that and if and only if there is a permutation such that .
Extend the action of to noncommutative homogeneous polynomials of degree by
[TABLE]
Then meets the compatibility condition if and only if
[TABLE]
for all permutations . That is, for all and all , we have . It follows that the corresponding tensor is symmetric.
The following lemma shows that the compatibility condition is necessary for existence of a NC Waring decomposition.
Lemma 3.2**.**
If a NC homogeneous polynomial of degree has a -term NC Waring decomposition, then the compatibility condition (3.3.1) holds. Moreover, if meets the compatibility condition, then has a -term NC Waring decomposition over the complex numbers (resp. real numbers) if and only if
[TABLE]
has a solution
Proof.
By definition, has a -term Waring decomposition if and only if
[TABLE]
Comparing the coefficients of on both sides, we get
[TABLE]
This also implies if for all . ∎
Example 3.3**.**
A NC homogeneous polynomial has the complex (resp. real) -term Waring decomposition
[TABLE]
if and only if is compatible and
[TABLE]
has a solution (resp. ). ∎
3.3.2. Reduction of NC Waring to Classical Waring
We see in this section that the NC Waring problem reduces to the commutative one.
Lemma 3.4**.**
For an index tuple , denote as the number of ’s that satisfy for all . Then
[TABLE]
Proof.
The problem is equivalent to calculating how many d-tuples can be formed by elements from , which is equivalent to
[TABLE]
∎
Theorem 3.5**.**
Suppose is a homogeneous NC polynomial which satisfies the compatibility conditions (3.3.1). Then the commutative collapse has the Waring decomposition
[TABLE]
(with being commuting variables) if and only if has the NC Waring decomposition
[TABLE]
Note that the number of terms is the same and the real coefficients (resp. complex coefficients) are the same.
Proof.
The proof begins by laying out the algebraic connection between and . Let denote a set consisting of one representative from each equivalence class. Then from (3.3.1), the NC polynomial has commutative collapse satisfying
[TABLE]
where .
Thus if satisfies the compatibility condition (3.3.1), then
[TABLE]
Therefore, is the commutative collapse of a compatible NC homogeneous degree polynomial iff for all index tuples of length .
Now we proceed to prove our theorem. Assume has the NC Waring decomposition (3.3.6), we shall obtain a reversible formula for the Waring decomposition of . By equation (3.3.7) and Lemma 3.2, the commutative collapse is
[TABLE]
Thus
[TABLE]
On the other hand, suppose ’s commutative collapse, , has the commutative Waring decomposition (3.3.5), then the calculations in (3.3.8) and (3.3.9) can be reversed. By comparing coefficients, this is equivalent to
[TABLE]
for all . Therefore by (3.3.7), satisfies
[TABLE]
for all index tuples of length . Hence by Lemma 3.2, has the Waring decomposition (3.3.6). Thus under the compatibility condition (3.3.1), the NC polynomial has a Waring decomposition iff its commutative collapse has the same Waring decomposition. ∎
3.4. NC Waring decompositions and symmetric tensors
A tensor based approach to the noncommutative Waring problem that can be used to prove Theorem 3.5 is as follows. By considering the correspondence of NC polynomials and tensors described in Section 2 as well as the relationship between NC polynomial decompositions and tensor decompositions, one sees that a NC polynomial has a NC Waring decomposition if and only if the corresponding tensor has a symmetric tensor decomposition.
It is well known that a tensor has a symmetric tensor decomposition if and only if the tensor itself is symmetric, e.g. see [CGLM08, Lemma 4.2] . Therefore, a NC polynomial has a NC Waring decomposition if and only if the corresponding tensor is symmetric. One may check that the tensor is symmetric if and only if satisfies the compatibility condition.
4. The general noncommutative Waring problem
We now consider a more general situation of which the problem in the preceding section is the base case. As you will see, the bookkeeping and notation is formidable, so it is very helpful to have done a simpler case. In the previous section our focus was to determine if a degree noncommutative homogeneous polynomial can be expressed as sums of powers of linear terms. Now we examine when a degree noncommutative homogeneous polynomial can be expressed as sums of powers of homogeneous degree terms.
As in the last section, we consider both noncommutative algebra and (for the tensor proficient) tensor based approaches.
4.1. Classical General Waring Problem.
The classical commutative Waring problem can be generalized from representation by sums of powers of linear functions to representation by sums of powers of homogeneous polynomials. The generalized classical Waring problem has also been well studied. According to Theorem 4 in [FOS12], there is an upper bound for the number of terms needed for such problems:
Theorem 4.1**.**
A general homogeneous polynomial of degree in variables, where , can be expressed as a sum of at most powers of degree homogeneous complex coefficient polynomials. Moreover, for a fixed , this bound is sharp for all sufficiently large .
4.2. Problem formulation and notation
Let be the set of all possible -tuples whose elements are integers between and , i.e.,
[TABLE]
Additionally, define by
[TABLE]
That is, is the set of -tuples of tuples of indices. For any , where we can write
[TABLE]
That is, is the monomial
[TABLE]
Recall our notation for a degree homogeneous polynomial
[TABLE]
where .
Remark 4.2**.**
For any , we can identify
[TABLE]
with
[TABLE]
On the other hand, for any element of , we can reverse this identification and form groups of size to get a -tuple of -tuples. We let denote the bijection
[TABLE]
which accomplishes this grouping. ∎
The General NC Waring Problem:
Given a NC homogeneous degree polynomial p, does it have a t-term power real NC Waring (resp. complex NC Waring) decomposition of degree . That is, can be written as
[TABLE]
We call this problem the -NC Waring problem and say a decomposition of the form (4.2.1) is a -term -NC Waring decomposition. Similarly for a commutative polynomial , we say a decomposition of the form (4.2.1) (with replaced by ) is a -term -Waring decomposition. Note that the problem treated in Section 3 is exactly the -NC Waring problem.
An obvious fact is, if is a degree NC homogeneous polynomial and has a -term -NC Waring decomposition, then its commutative collapse has a -term -Waring decomposition. For a conjecture on the generic value of in this commutative case, see [LORS19, Conjecture 1.2].
4.2.1. Tuple indicator functions
We now extend the notion of indicator function to tuples of -tuples. For two tuples , denote
[TABLE]
Then for an index tuple , the number of times a particular tuple appears in is
[TABLE]
Furthermore, denote
[TABLE]
as the number of integers appearing in all the -tuples in .
4.3. Main results on the general Waring decomposition
Similar to Section 2, we first state a compatibility condition which is necessary for the existence of a generalized NC Waring decomposition. We then prove that, if this condition holds, then we can reduce the generalized NC Waring problem to a commutative one at the price of increasing our number of variables.
4.3.1. The Compatibility Condition
The generalized version of the compatibility condition is defined as follows:
Definition 4.3**.**
We say a noncommutative homogeneous polynomial of degree in g variables of the form
[TABLE]
satisfies the -compatibility condition* if*
[TABLE]
for all index sets, , such that . Consistent with this, we define the -equivalence relation*, denoted , on by*
[TABLE]
for all . ∎
Remark 4.4**.**
Here are a few bookkeeping properties of -equivalences.
- (1)
We have if and only if . 2. (2)
Let and let . If divides , then implies . In the case where this follows from equation (4.2.2). The general case is similar. 3. (3)
Let and let be a degree NC homogeneous polynomial. If divides and satisfies the -compatibility condition then satisfies the -compatibility condition.
Items (2) and (3) highlight that, as grows, it becomes increasingly difficult for fixed monomials and of degree divisible by to be -equivalent. As an immediate consequence, as grows, it become more likely that a fixed NC homogeneous polynomial of degree divisible by satisfies the -compatibility condition. In the extreme case, monomials and of degree are -equivalent if and only if . As a result, every degree NC homogeneous polynomial satisfies the -compatibility condition. ∎
Example 4.5**.**
Let
[TABLE]
Then
[TABLE]
Now let be the degree four homogeneous NC polynomial
[TABLE]
Then satisfies the -compatibility condition and the -compatibility condition. However, does not satisfy the -compatibility condition, since the coefficient of in is [math] but the coefficient of is and
[TABLE]
The following lemma shows that the -compatibility condition is necessary for the general NC Waring problem.
Lemma 4.6**.**
Suppose a NC homogeneous polynomial of degree in variables has a -term -NC Waring decomposition, then satisfies the -compatibility condition. That is, if . Here has coefficients .
Moreover, the -NC Waring problem has a solution over the complex numbers (resp. real numbers) if and only if the equation
[TABLE]
has a solution (resp. ).
Proof.
The polynomial has a -term -NC Waring decomposition iff degree homogeneous polynomials, satisfying
[TABLE]
Comparing coefficients we see, equivalent to the -NC Waring decomposition is:
[TABLE]
yielding (4.3.3).
As a consequence for any satisfying for every , yielding the first assertion of the theorem. ∎
Example 4.7**.**
Let
[TABLE]
Then is an example where there is no -NC Waring decomposition; indeed the -compatibility condition is violated because . However, its commutative collapse does have the (2,2)-Waring decomposition:
[TABLE]
4.4. Reduction to classical Waring in more variables
To solve the general -noncommutative Waring problem we reduce to the case solved by Theorem 3.5. This reduction is accomplished by identifying a monomial with a new variable . Namely, fix and define the map on monomials of the form for by
[TABLE]
where the are noncommutative indeterminates indexed by elements of .
We extend our definition of to a noncommutative homogeneous polynomial
[TABLE]
of degree by
[TABLE]
Lemma 4.8**.**
The map as defined in equation (4.4.1) defines an algebra isomorphism on the algebra of noncommutative homogeneous polynomials of degree divisible by in the noncommutative indeterminate which maps to the algebra of noncommutative homogeneous polynomials in the noncommutative indeterminates .
Proof.
This is straightforward from the definition of on a noncommutative homogeneous polynomial of degree . ∎
Note that in the case of commutative , substitution of by a commutative is sometimes used, however, the isomorphism property in Lemma 4.8 fails, so conclusions are much less precise than what we get here.
We now give our main result for the -NC Waring problem.
Theorem 4.9**.**
Let be a noncommutative homogeneous polynomial of degree in the indeterminate , and let be as defined in equation (4.4.1). Then we have the following.
- (1)
* has a -term -noncommutative Waring decomposition if and only if has a -term -noncommutative Waring decomposition.* 2. (2)
* satisfies the -compatibility condition if and only if satisfies the -compatibility condition.* 3. (3)
* has a -term -noncommutative Waring decomposition if and only if satisfies the -compatibility condition and the commutative collapse of has a -term -Waring decomposition.*
Proof.
To prove item (1), assume has a -term -noncommutative Waring decomposition
[TABLE]
By Lemma 4.8, is an algebra isomorphism so
[TABLE]
This shows has a -term noncommutative Waring decomposition. The reverse direction follows the same reasoning using instead of .
To prove item (2) let
[TABLE]
Then
[TABLE]
Observe
[TABLE]
where the are viewed as elements of the index set if and only if
[TABLE]
where the are viewed as as tuples of elements of . It follows that
[TABLE]
where the are viewed as elements of the index set if and only if
[TABLE]
where the are viewed as as tuples of elements of .
Item (3) is an immediate consequence of items (1) and (2) with Theorem 3.5, our main result for -NC Waring decompositions. ∎
4.5. Additional variables are necessary for the reduction
It is tempting to try to solve the general -NC Waring problem by reducing to the commutative case without introducing additional variables. This section will show that this is not possible.
One may hope that the following are true:
- (1)
If is a degree NC homogeneous polynomial, which satisfies the -compatibility condition (4.3.2), then its commutative collapse has the Waring decomposition
[TABLE]
(with being commuting variables) if and only if has the NC Waring decomposition
[TABLE] 2. (2)
The commutative collapse of has a -term -NC Waring decomposition iff the commutative collapse of has a -term -NC Waring decomposition.
The following polynomial gives a counter example to both items. Let
[TABLE]
and let . Then satisfies the -compatibility condition. We will show that the commutative collapse of has a two term -Waring decomposition but that does not have a two term -NC Waring decomposition.
It is straight forward to check
[TABLE]
Item (1) would imply that
[TABLE]
which is contradiction. This shows that item (1) cannot be correct.
In fact, does not have a two term -NC Waring decomposition. To check this set
[TABLE]
Then satisfies the -compatibility condition but does not have a two term -Waring decomposition. To see this, note that the tensor corresponding to is a symmetric matrix which has rank , hence a NC Waring decomposition for requires four terms. It follows from Theorem 4.9 (1) that does not have a two term -NC Waring decomposition.
4.6. General NC Waring and tensors
Standard tensor techniques can also be used to address the general NC Waring problem and to derive Theorem 4.9. One may identify the space of NC homogeneous polynomials of degree with the space of tensors . Requiring that a NC polynomial satisfies the -compatibility condition then corresponds to requiring that the corresponding tensor satisfies a restricted symmetry condition. In standard tensor notation one must have . In words, is a symmetric tensor in the space where is the space . The result again follows from the fact that a tensor in has a symmetric tensor decomposition if and only if it is symmetric.
While this is an expedient approach for those familiar with tensor methods, we expect the noncommutative algebra approach to be more clear for NC algebra experts who are not familiar with tensor methods. Furthermore, the tensor based approach does not easily convert to a condensed statement of Theorem 4.9 which only uses the language of noncommutative polynomials.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[AH 95] J. Alexander and A. Hirschowitz. Polynomial interpolation in several variables. J. Algebraic Geom., 4 (1995), pp. 201-222.
- 2[AOP 09] H. Abo, G. Ottaviani and C. Peterson. Induction for secant varieties of Segre varieties. Trans. Am. Math. Soc., 361 (2009), pp. 767-792.
- 3[B 02] D.J. Bernstein, Pippenger’s Exponentiation Algorithm. Preprint, (2002). http://cr.yp.to/papers.html#pippenger
- 4[BC 13] A. Bodin and M. Car, Waring’s problem for polynomials in two variables. Proc. Amer. Math. Soc., 141 (2013), pp. 1577–1589.
- 5[BKP 16] S. Burgdorf, I. Klep and J. Povh, Optimization of Polynomials in Noncommuting Variables. Springer, 2016.
- 6[CHS 06] J.F. Camino, J.W. Helton and R.E. Skelton, Solving matrix inequalities whose unknowns are matrices. SIAM Jour. of Optimization, 17 (2006), no 1, pp. 1-36.
- 7[COV 17] L. Chiantini, G. Ottaviani, and N. Vannieuwenhoven, Effective Criteria for Specific Identifiability of Tensors and Forms. SIAM J. Matrix Anal. Appl., 38 (2017), pp. 656-681.
- 8[CGLM 08] P. Comon, G.H. Golub, L.-H. Lim, B. Mourrain, Symmetric tensors and symmetric tensor rank. SIAM J. Matrix Anal. Appl., 30 (2008), no. 3, pp. 1254–1279.