Interesting Open Problem Related to Complexity of Computing the Fourier Transform and Group Theory

TL;DR

This paper discusses an open problem in the complexity of computing the Fourier Transform, highlighting its connection to group theory and representation theory, and emphasizing the gap between known bounds and theoretical limits.

Contribution

It introduces a natural open problem linking Fourier Transform complexity to group theory, expanding understanding of computational lower bounds.

Findings

Established an $ ext{O}(n ext{log} n)$ algorithm for Fourier Transform

Presented a lower bound of $ ext{Omega}(n ext{log} n)$ under certain conditions

Highlighted the gap and open questions in the complexity of Fourier Transform computation

Abstract

The Fourier Transform is one of the most important linear transformations used in science and engineering. Cooley and Tukey's Fast Fourier Transform (FFT) from 1964 is a method for computing this transformation in time $O(n\log n)$. From a lower bound perspective, relatively little is known. Ailon shows in 2013 an $\Omega(n\log n)$ bound for computing the normalized Fourier Transform assuming only unitary operations on pairs of coordinates is allowed. The goal of this document is to describe a natural open problem that arises from this work, which is related to group theory, and in particular to representation theory.

Equations

$$F(k,\ell )=n^{-1/2}{e}^{-i2\pi k\ell /n}.$$

$$H(k,\ell )=n^{-1/2}(-1{)}^{⟨k,\ell ⟩},$$

$$R_m\cdot R_{m-1}\dots R_2\cdot R_1$$

$$R_t=\left(\begin{array}{ccccc}\begin{array}{ccc}1& & \\ & \ddots & \\ & & 1\end{array}& \begin{array}{c}{}^{i_t}\\ \\ \end{array}& & \begin{array}{c}{}^{j_t}\\ \\ \end{array}& \\ i_t& A_t(1,1)& & A_t(1,2)& \\ & & \begin{array}{ccc}1& & \\ & \ddots & \\ & & 1\end{array}& & \\ j_t& A_t(2,1)& & A_t(2,2)& \\ & & & & \begin{array}{ccc}1& & \\ & \ddots & \\ & & 1\end{array}\end{array}\right).$$

$$\Phi (U)=-\sum _{i,j}|U(i,j){|}^2\log |U(i,j){|}^2.$$

$$|M_t(i_t,k){|}^2+|M_t(j_t,k){|}^2=|M_{t-1}(i_t,k){|}^2+|M_{t-1}(j_t,k){|}^2.$$

$$|(x^2\log x^2+y^2\log y^2)-(z^2\log z^2+w^2\log w^2)|\le r^2.$$

$$\sum _k(|M_t(i_t,k){|}^2+|M_t(j_t,k){|}^2)=2.$$

$$G=\left(\begin{array}{cc}0& F\\ -F^{\ast }& 0\end{array}\right).$$

$$X_{\alpha }=(\cos \alpha )\mathrm{Id}+(\sin \alpha )G.$$

$$X_{{\alpha }^{\prime }}{X}_{\alpha }=X_{\alpha +{\alpha }^{\prime }}.$$

$$\Phi (X_{\alpha })=\Theta ({\alpha }^{2}n\log n).$$

$$\Phi (X_{\pi /4-\alpha /2})=\Theta \left({\sin }^2(\pi /4-\alpha /2)\cdot n\log n\right)\Phi (X_{\pi /4+\alpha /2})=\Theta \left({\sin }^2(\pi /4+\alpha /2)\cdot n\log n\right),$$

$$\left|\Phi (X_{\pi /4+\alpha /2})-\Phi (X_{\pi /4-\alpha /2})\right|$$

$$({\Psi }_{det}^k(U){)}_{I,J}=\det U_{I,J},$$

Full text

1 Introduction

The (discrete) normalized Fourier transform is a complex linear mapping sending an input $x\in\mathbb{C}^{n}$ to $y=Fx\in\mathbb{C}^{n}$, where $F$ is an $n\times n$ unitary matrix defined by

[TABLE]

If $n$ is a power of $2$, then Walsh-Hadamard transform is a real, orthogonal mapping $H$, with the element in position $(k,\ell)$ given by:

[TABLE]

where $\langle k,\ell\rangle$ is the dot-product modulo $2$ of the binary representations of the integers $k-1$ and $\ell-1$.

More genererally, both $F$ and $H$ (resp.) are defined by the characters of underlying abelian groups ${\mathbb{Z}}/n{\mathbb{Z}}$ (integers modulo $n$, under addition) and the $\log n$ dimensional binary cube $({\mathbb{Z}}/2{\mathbb{Z}})^{\log n}$ ($\log n$ dimensional vector space over bits). Given an input vector $x\in\mathbb{C}^{n}$, it is possible to compute $Fx$ and $Hx$ (resp.) in time $O(n\log n)$ using the Fast Fourier-Transform [Cooley and Tukey(1964)] , or the Walsh-Hadamard transform (resp.).

As for computational lower bounds, it is trivial that computing both $Fx$ and $Hx$ requires a linear number of steps, because each coordinate of the output depends on all the input coordinates.

There has not been much prior work on better bounds. We refer the reader to [Ailon(2013)] for a brief history of this line of work, and concentrate on a recent lower bound.

The work [Ailon(2013)] provides a lower bound of $\Omega(n\log n)$ operations for computing $Fx$ (or $Hx$) given $x$, assuming that at each step the computer can perform a unitary operation affecting at most $2$ rows. In other words, the algorithm, running in $m$ steps, is viewed as a product

[TABLE]

of matrices $R_{t}$, each $R_{t}$ a block-diagonal matrix with $n-2$ blocks equalling $1$, and one block equalling a $2\times 2$ unitary matrix $A_{t}$:

[TABLE]

The justification for this model of computation is threefold:

• •

Similarly to matrices of the form (3), any basic operation of a modern computer (e.g., addition of numbers) acts on only a fixed number of inputs.

• •

The Fast Fourier-Transform, as well as the Walsh-Hadamard transform, operate in this model, and

• •

The set of matrices of the form (3) generate the group of unitary matrices.

Thus the question of computational complexity of of the Fourier transform becomes that of computing distances between elements of a group, namely the unitary group, with respect to a set of generators that is computationally simple.

Obtaining the lower bound of $\Omega(n\log n)$ in [Ailon(2013)] is done by defining a potential function $\Phi$ for unitary matrices, as follows:

[TABLE]

With this potential function, one shows that:

1. (a)

$\Phi(\operatorname{Id})=0$ 2. (b)

$\Phi(F)=\Phi(H)=\Omega(n\log n)$ 3. (c)

$|\Phi(M_{t})-\Phi(M_{t-1})|\leq 2$, where $M_{t}=R_{t}\cdot R_{t-1}\dots R_{1}$ is the state of the algorithm after $t$ steps.

Indeed, if the potentail $\Phi$ grows from [math] to $\Omega(n\log n)$ changing (in absolute value) by no more than $2$ at each step, then the number of steps must be $\Omega(n\log n)$. Showing $(c)$ is done using two observations. The first is that $M_{t}$ defers from $M_{t-1}$ in at most $2$ rows $i_{t}$ and $j_{t}$, and that for each column $k$, due to unitarity of $A_{t}$,

[TABLE]

The next observation is that any $4$ numbers $x,y,z,w$ satisfying $x^{2}+y^{2}=z^{2}+w^{2}=:r^{2}$, also satisfy

[TABLE]

Combining the observations, we conclude that the total change in the potential function can be at most

[TABLE]

2 An Interesting Problem

The advantage of the method just described is that it reduces a computational problem to that of computing distance between two elements of a group, with respect to a chosen set of generators of the group. We now define a more general problem within the same group theoretical setting.

Consider the $2n\times 2n$ matrix $G$ defined as

[TABLE]

(One may replace $F$ with $H$, but we work with $F$ henceforth.) The matrix $G$ is skew-Hermitian. Let $\operatorname{Id}$ denote the $2n\times 2n$ identity matrix, and finally define for a real angle $\alpha$ the following matrix:

[TABLE]

It is easy to verify that $X_{\alpha}$ is unitary for all $\alpha$. It is also easy to verify that

[TABLE]

Also, using the potential function $\Phi$ defined above, we see that

[TABLE]

Hence, using the argument as above, the number of steps required to compute $X_{\alpha}$ must be at least $\Omega(\alpha^{2}n\log n)$. However, it is unreasonable that it should be possible to compute $X_{\alpha}$ faster than the time it takes to compute $F$, by a factor of $1/\alpha^{2}$. Indeed, given an input $x\in\mathbb{C}^{n}$, we could simply embed it as $\tilde{x}\in\mathbb{C}^{2n}$ by padding with $n$ [math]’s, then compute $\tilde{y}=X_{\alpha}\tilde{x}$ and then retrieve $y=Fx$ from $\tilde{y}$ and $\tilde{x}$ by a simple arithmetic manipulation. Hence, we conjecture that the number of steps required to compute $X_{\alpha}$ should be not much smaller than $\Omega(n\log n)$. 111The author conjectures $\Omega((n\log n)/\log(1/\alpha))$ to be the correct bound.

2.1 A slight improvement: Lower bound of $\Omega(\alpha n\log n)$.

It is possible to get a better bound than $\Omega(\alpha^{2}n\log n)$, as follows. Instead of starting the computation at state $\operatorname{Id}$ and finishing at $X_{\alpha}$, we can opportunistically choose a starting point $M$ (and finish at $X_{\alpha}M$).

If we choose the state $M=X_{\pi/4-\alpha/2}$ then it is trivial to verify that the computation ends at state $X_{\alpha}M$, which equals $X_{\pi/4+\alpha/2}$ by (4). We then observe that

[TABLE]

and hence,

[TABLE]

2.2 Stronger improvements?

Is it possible to get a stronger lower bound than $\alpha n\log n$? One approach for solving this problem might be using group representation theory. If $\Psi:U(n)\mapsto U(n^{\prime})$ is any unitary representation of $U(n)$, then we could define a new potential function $\Phi\circ\Psi$ on $U(n)$, and use it to obtain possibly better lower bounds.

An interesting representation is related to determinants. We let the order $k$ determinant representation of a unitary matrix $U$ be the matrix $\Psi_{det}^{k}(U)$ of shape ${n\choose k}\times{n\choose k}$, defined by

[TABLE]

where $I,J$ are subsets of size exactly $k$ of $[n]$, $U_{I,J}$ is the $k$-by-$k$ submatrix defined by row set $I$ and column set $J$. The fact that $\Psi_{det}^{k}(U)$ is a unitary matrix coming from a group representation is non-trivial, and we refer the reader to resources on representation theory for more details.

So far I have not been able to make progress on the problem using this (quite natural) representation, but I am not convinced that this direction is futile either.

2.3 Important Note: Even the case $\alpha=\pi/4$ is Interesting

Note that although the main problem proposed here is to understand the asymptotic behviour of the complexity of $X_{\alpha}$, as $\alpha$ tends to [math], even the case of finding a lower bound for computation of $X_{\pi/4}$ is not trivial, in the sense that it is not clear how (and whether it is at all possible) to get a bound better than $\frac{1}{2\sqrt{2}}n\log_{2}n$, which is the best possible using the “vanilla” entropy function $\Phi$.