Matrix scaling limits in finitely many iterations

TL;DR

This paper studies the convergence behavior of the Sinkhorn scaling algorithm, providing explicit examples of matrices that become doubly stochastic after exactly one column scaling, highlighting specific limits of the iterative process.

Contribution

It constructs a two-parameter family of positive matrices that reach doubly stochastic form after a single column scaling, revealing new insights into the algorithm's convergence properties.

Findings

Matrices that become doubly stochastic after one column scaling

Explicit construction of such matrices for any size n

Insights into the limits of the Sinkhorn scaling algorithm

Abstract

The alternate row and column scaling algorithm applied to a positive $n\times n$ matrix $A$ converges to a doubly stochastic matrix $S(A)$, sometimes called the \emph{Sinkhorn limit} of $A$. For every positive integer $n$, a two parameter family of row but not column stochastic $n\times n$ positive matrices is constructed that become doubly stochastic after exactly one column scaling.

Equations

$${\mathrm{rowsum}}_i(A)=\sum _{j=1}^n{a}_{i,j}.$$

$${\mathrm{colsum}}_j(A)=\sum _{i=1}^n{a}_{i,j}.$$

$$X(A)=\mathrm{diag}\left(\frac{1}{{\mathrm{rowsum}}_1(A)},\frac{1}{{\mathrm{rowsum}}_2(A)},\dots ,\frac{1}{{\mathrm{rowsum}}_n(A)}\right).$$

$${\mathrm{rowsum}}_i\left(X(A)A\right)$$

$$Y(A)=\mathrm{diag}\left(\frac{1}{{\mathrm{colsum}}_1(A)},\frac{1}{{\mathrm{colsum}}_2(A)},\dots ,\frac{1}{{\mathrm{colsum}}_n(A)}\right).$$

$${\mathrm{colsum}}_j(AY(A))=\sum _{i=1}^n(AY(A){)}_{i,j}=\sum _{i=1}^n\frac{a_{i,j}}{{\mathrm{colsum}}_j(A)}=\frac{{\mathrm{colsum}}_j(A)}{{\mathrm{colsum}}_j(A)}=1$$

$$A=\left(\begin{array}{ccc}1/5& 1/5& 3/5\\ 2/5& 1/5& 2/5\\ 3/5& 1/5& 1/5\end{array}\right).$$

$$AY(A)=\left(\begin{array}{ccc}1/6& 1/3& 3/6\\ 2/6& 1/3& 2/6\\ 3/6& 1/3& 1/6\end{array}\right).$$

$${\mathrm{row}}_i(A)=\left(\begin{array}{c}{a}_{i,1},a_{i,2},\dots ,a_{i,n}\end{array}\right).$$

$$0<x+z<\frac{1}{k}\mathrm{and}x+z\ne \frac{2}{n}$$

$$y=\frac{x+z}{2}\mathrm{and}w=\frac{1-k(x+z)}{n-2k}.$$

$${\mathrm{row}}_i(A)=\{\begin{array}{ll}(\underset{k}{\underbrace{x,x,\dots ,x}}\underset{n-2k}{\underbrace{w,w,\dots ,w}}\underset{k}{\underbrace{z,z,\dots ,z}}& \text{ if i∈{1,2,…,ℓ}\}\\ & \\ (\underset{k}{\underbrace{y,y,\dots ,y}}\underset{n-2k}{\underbrace{w,w,\dots ,w}}\underset{k}{\underbrace{y,y,\dots ,y}}& \text{ if }i\in \{\ell +1,\ell +2,\dots ,n-\ell \}\\ & \\ (\underset{k}{\underbrace{z,z,\dots ,z}}\underset{n-2k}{\underbrace{w,w,\dots ,w}}\underset{k}{\underbrace{x,x,\dots ,x}}& \text{ if }i\in \{n-\ell +1,n-\ell +2,\dots ,n\}\end{array}$$

$$i\in \{1,2,\dots ,\ell \}\cup \{n-\ell +1,n-\ell +2,\dots ,n\}$$

$${\mathrm{rowsum}}_i(A)=k(x+z)+(n-2k)w=1.$$

$$i\in \{\ell +1,\ell +2,\dots ,n-\ell \}$$

$${\mathrm{rowsum}}_i(A)=2ky+(n-2k)w=1.$$

$$j\in \{1,2,\dots ,k\}\cup \{n-k+1,n-k+2,\dots ,n\}$$

$${\mathrm{colsum}}_j(A)=\ell x+(n-2\ell )y+\ell z=ny\ne 1$$

$$j\in \{k+1,k+2,\dots ,n-k\}$$

$${\mathrm{colsum}}_j(A)=nw\ne 1.$$

$$Y(A)$$

$$i\in \{1,2,\dots ,\ell \}\cup \{n-\ell +1,n-\ell +2,\dots ,n\}$$

$${\mathrm{rowsum}}_i(AY(A))=\frac{kx}{ny}+\frac{(n-2k)w}{nw}+\frac{kz}{ny}=\frac{k(x+z)}{ny}+1-\frac{2k}{n}=1.$$

$$i\in \{\ell +1,\ell +2,\dots ,n-\ell \}$$

$${\mathrm{rowsum}}_i(A)=\frac{2ky}{ny}+\frac{(n-2k)w}{nw}=\frac{2k}{n}+1-\frac{2k}{n}=1.$$

$$0<x+z<1,x+z\ne \frac{2}{3}$$

$$y=\frac{x+z}{2}\mathrm{and}w=1-x-z.$$

$$A=\left(\begin{array}{ccc}x& w& z\\ y& w& y\\ z& w& x\end{array}\right),$$

$$x=\frac{1}{4},y=\frac{3}{16},z=\frac{1}{8},w=\frac{1}{12}$$

$$A=\left(\begin{array}{ccccccc}1/4& 1/4& 1/12& 1/12& 1/12& 1/8& 1/8\\ 1/4& 1/4& 1/12& 1/12& 1/12& 1/8& 1/8\\ 1/4& 1/4& 1/12& 1/12& 1/12& 1/8& 1/8\\ 3/16& 3/16& 1/12& 1/12& 1/12& 3/16& 3/16\\ 1/8& 1/8& 1/12& 1/12& 1/12& 1/4& 1/4\\ 1/8& 1/8& 1/12& 1/12& 1/12& 1/4& 1/4\\ 1/8& 1/8& 1/12& 1/12& 1/12& 1/4& 1/4\end{array}\right).$$

Full text

1. The alternate scaling algorithm

A positive matrix is a matrix with positive coordinates. A nonnegative matrix is a matrix with nonnegative coordinates. Let $D=\operatorname{\text{diag}}(x_{1},\ldots,x_{n})$ denote the $n\times n$ diagonal matrix with coordinates $x_{1},\ldots,x_{n}$ on the main diagonal. The diagonal matrix $D$ is positive if its coordinates $x_{1},\ldots,x_{n}$ are positive. If $A=(a_{i,j})$ is an $m\times n$ positive matrix, if $X=\operatorname{\text{diag}}(x_{1},\ldots,x_{m})$ is an $m\times m$ positive diagonal matrix, and if $Y=\operatorname{\text{diag}}(y_{1},\ldots,y_{n})$ is an $n\times n$ positive diagonal matrix, then $XA=(x_{i}a_{i,j})$, $AY=(a_{i,j}y_{j})$, $XAY=(x_{i}a_{i,j}y_{j})$ are $m\times n$ positive matrices.

Let $A=(a_{i,j})$ be an $n\times n$ matrix. The $i$th row sum of $A$ is

[TABLE]

The $j$th column sum of $A$ is

[TABLE]

The matrix $A$ is row stochastic if it is nonnegative and $\operatorname{\text{rowsum}}_{i}(A)=1$ for all $i\in\{1,\ldots,n\}$. The matrix $A$ is column stochastic if it is nonnegative and $\operatorname{\text{colsum}}_{j}(A)=1$ for all $j\in\{1,\ldots,n\}$. The matrix $A$ is doubly stochastic if it is both row stochastic and column stochastic.

Let $A=(a_{i,j})$ be a nonnegative $n\times n$ matrix such that $\operatorname{\text{rowsum}}_{i}(A)>0$ and $\operatorname{\text{colsum}}_{j}(A)>0$ for all $i,j\in\{1,\ldots,n\}$. Define the $n\times n$ positive diagonal matrix

[TABLE]

Multiplying $A$ on the left by $X(A)$ multiplies each coordinate in the $i$th row of $A$ by $1/\operatorname{\text{rowsum}}_{i}(A)$, and so

[TABLE]

for all $i\in\{1,2,\ldots,n\}$. The process of multiplying $A$ on the left by $X(A)$ to obtain the row stochastic matrix $X(A)A$ is called row scaling. We have $X(A)A=A$ if and only if $A$ is row stochastic if and only if $X(A)=I$. Note that the row stochastic matrix $X(A)A$ is not necessarily column stochastic.

Similarly, we define the $n\times n$ positive diagonal matrix

[TABLE]

Multiplying $A$ on the right by $Y(A)$ multiplies each coordinate in the $j$th column of $A$ by $1/\operatorname{\text{colsum}}_{j}(A)$, and so

[TABLE]

for all $j\in\{1,2,\ldots,n\}$. The process of multiplying $A$ on the right by $Y(A)$ to obtain a column stochastic matrix $AY(A)$ is called column scaling. We have $AY(A)=A$ if and only if $Y(A)=I$ if and only if $A$ is column stochastic. The column stochastic matrix $AY(A)$ is not necessarily row stochastic.

Let $A$ be a positive $n\times n$ matrix. Alternately row scaling and column scaling the matrix $A$ produces an infinite sequence of matrices that converges to a doubly stochastic matrix This result (due to Sinkhorn [7], Knopp-Sinkhorn [8], Brualdi, Parter, and Schnieder [1]), Menon [4], Letac [3], Tverberg [9], and others) is classical.

Nathanson [5, 6] proved that if $A$ is a $2\times 2$ positive matrix that is not doubly stochastic but becomes doubly stochastic after a finite number $L$ of scalings, then $L$ is at most 2, and the $2\times 2$ row stochastic matrices that require exactly one scaling were computed explicitly. An open question was to describe $n\times n$ matrices with $n\geq 3$ that are not doubly stochastic but become doubly stochastic after finitely many scalings. Ekhad and Zeilberger [2] discovered the following row-stochastic but not column stochastic $3\times 3$ matrix, which requires exactly one column scaling to become doubly stochastic:

[TABLE]

Column scaling $A$ produces the doubly stochastic matrix

[TABLE]

The following construction generalizes this example. For every $n\geq 3$, there is a two parameter family of row-stochastic $n\times n$ matrices that require exactly one column scaling to become doubly stochastic

Let $A=\left(\begin{matrix}a_{i,j}\end{matrix}\right)$ be an $m\times n$ matrix. For $i=1,\ldots,m$, we denote the $i$th row of $A$ by

[TABLE]

Theorem 1**.**

Let $k$ and $\ell$ be positive integers, and let $n>\max(2k,2\ell)$. Let $x$ and $z$ be positive real numbers such that

[TABLE]

and let

[TABLE]

The $n\times n$ matrix $A$ such that

[TABLE]

is row stochastic but not column stochastic. The matrix obtained from $A$ after one column scaling is doubly stochastic.

Proof.

If

[TABLE]

then

[TABLE]

If

[TABLE]

then

[TABLE]

Thus, the matrix $A$ is row stochastic.

If

[TABLE]

then

[TABLE]

and if

[TABLE]

then

[TABLE]

Thus, matrix $A$ is not column stochastic.

The column scaling matrix for $A$ is the positive diagonal matrix

[TABLE]

For the column scaled matrix $AY(A)$, we have the following row sums. If

[TABLE]

then

[TABLE]

If

[TABLE]

then

[TABLE]

Thus, the matrix $AY(A)$ is row stochastic. This completes the proof. ∎

For example, let $k=\ell=1$ and $n=3$, and let $w,x,y,z$ be positive real numbers such that

[TABLE]

[TABLE]

The matrix

[TABLE]

is row stochastic but not column stochastic. By Theorem 1, column scaling $A$ produces a doubly stochastic matrix. Choosing $x=1/5$ and $z=3/5$, we obtain the matrix (1).

Let $k=2$, $\ell=3$, and $n=7$. Choosing

[TABLE]

we obtain the row but not column stochastic matrix

[TABLE]

Column scaling produces the doubly stochastic matrix

[TABLE]

Theorem 2**.**

Every $n\times n$ matrix $A$ constructed in Theorem 1 satisfies $\det(A)=0$.

Proof.

There are three cases.

If $k>1$ or $n-2k>1$, then $A$ has two equal columns and $\det(A)=0$.

If $\ell>1$ or $n-2\ell>1$, then $A$ has two equal rows and $\det(A)=0$.

If $k=\ell=1$ and $n=3$, then

[TABLE]

and

[TABLE]

This completes the proof. ∎

Theorem 2 is important for the following reason. Let $A$ be an $n\times n$ positive matrix. If $\det(A)\neq 0$, then (by solving a system of linear equations) there exist a unique $n\times n$ diagonal matrix $Z=\operatorname{\text{diag}}(z_{1},\ldots,z_{n})$ and a column stochastic $n\times n$ matrix $B$ such that $B=ZA$. If $z_{i}\neq 0$ for all $i\in\{1,\ldots,n\}$, then $Z$ is invertible. Setting $X=Z^{-1}$, we have $XB=A$. Because $A$ is positive, the matrices $X$ and $B$ can be adjusted so that $X$ is a positive diagonal matrix and $B$ is a positive $n\times n$ matrix. If $A$ is row stochastic, then $X$ is the row scaling matrix associated to $B$. Thus, if $A$ is a row stochastic matrix such that column scaling $A$ produces a doubly stochastic matrix, then we have pulled $A$ back to a column stochastic matrix $B$, and we have increased by 1 the number of scalings needed to get a doubly stochastic matrix.

Unfortunately, by Theorem 2, the matrices constructed above all have determinant 0.

2. Open problems

1. (1)

Does there exist a positive $3\times 3$ row stochastic but not column stochastic matrix $A$ with nonzero determinant such that $A$ becomes doubly stochastic after one column scaling? 2. (2)

Let $A$ be a positive $3\times 3$ row stochastic but not column stochastic matrix that becomes doubly stochastic after one column scaling. Does $\det(A)=0$ imply that $A$ has the shape of matrix (4)? 3. (3)

Here is the inverse problem: Let $A$ be an $n\times n$ row-stochastic matrix. Does there exist a column stochastic matrix $B$ such that row scaling $B$ produces $A$ (equivalently, such that $X(B)B=A$)? Compute $B$. 4. (4)

Modify the above problems so that the matrices are required to have rational coordinates. 5. (5)

Determine if, for positive integers $L\geq 3$ and $n\geq 3$, there exists a positive $n\times n$ matrix that requires exactly $L$ scalings to reach a doubly stochastic matrix. 6. (6)

Classify all matrices for which the alternate scaling algorithm terminates in finitely many steps.