When Slepian Meets Fiedler: Putting a Focus on the Graph Spectrum

TL;DR

This paper introduces Slepian graph signals that maximize energy concentration in subgraphs, linking spectral graph theory with clustering and embedding to enhance understanding of graph-localized signals.

Contribution

It presents a novel design of Slepian graph signals and establishes a new connection between localized spectral signals and graph clustering.

Findings

Slepian graph signals optimize energy concentration in subgraphs.

A new link between spectral localization and graph clustering is established.

The approach enhances analysis of localized signals on graphs.

Abstract

The study of complex systems benefits from graph models and their analysis. In particular, the eigendecomposition of the graph Laplacian lets emerge properties of global organization from local interactions; e.g., the Fiedler vector has the smallest non-zero eigenvalue and plays a key role for graph clustering. Graph signal processing focusses on the analysis of signals that are attributed to the graph nodes. The eigendecomposition of the graph Laplacian allows to define the graph Fourier transform and extend conventional signal-processing operations to graphs. Here, we introduce the design of Slepian graph signals, by maximizing energy concentration in a predefined subgraph for a graph spectral bandlimit. We establish a novel link with classical Laplacian embedding and graph clustering, which provides a meaning to localized graph frequencies.

Figures

Equations

$$\mathbf{F}=\frac{1}{\sqrt{N}}[e^{j{\omega }_l(k-1)}{]}_{k,l},k,l=1,\dots ,N,$$

$$\mu =\frac{{\sum }_{k\in \mathcal{S}}|g[k]{|}^2}{{\sum }_{k=1}^N|g[k]{|}^2}.$$

$$\mu =\frac{{\hat{\mathbf{g}}}^H\mathbf{C}\hat{\mathbf{g}}}{{\hat{\mathbf{g}}}^H\hat{\mathbf{g}}},$$

$$\lambda =\frac{{\mathbf{u}}^T\mathbf{L}\mathbf{u}}{{\mathbf{u}}^T\mathbf{u}}$$

$$\mu =\frac{{\hat{\mathbf{g}}}^T\mathbf{C}\hat{\mathbf{g}}}{{\hat{\mathbf{g}}}^T\hat{\mathbf{g}}},$$

$$\arg \underset{\mathbf{g}}{\min }\sum _{i,j=1}^N{A}_{i,j}(g_i-g_j{)}^2=\arg \underset{\mathbf{g}}{\min }{\mathbf{g}}^T\mathbf{L}\mathbf{g},$$

$$\xi ={\hat{\mathbf{g}}}^T{\mathbf{\Lambda }}_W^{1/2}\underset{\mathbf{C}}{\underbrace{{\mathbf{W}}^T{\mathbf{U}}^T\mathbf{S}\mathbf{U}\mathbf{W}}}{\mathbf{\Lambda }}_W^{1/2}\hat{\mathbf{g}},$$

Full text

I Introduction

Graphs are powerful mathematical models to represent complex data structures [1]. Spectral graph theory supports an important class of methods for studying graph topology as well as analyzing graph signals. Graph partitioning problems [2] can be solved as the convex relaxation of the graph-cut criterion, which reverts to an eigendecomposition of the graph Laplacian. Thresholding the eigenvector with the smallest non-zero eigenvalue—commonly called the Fiedler vector—leads to a graph bipartition [3]. These eigenvectors also solve the embedding problem; i.e., mapping the nodes onto a line (or higher-dimensional space) such that the distance between connected nodes is minimized [4]. In addition, the same eigenvectors are also used to define the graph Fourier transform (GFT), which has attracted a lot of attention from the signal processing community as many Fourier-domain operations have been generalized to graphs using the GFT [5]. One notable example is the spectral graph wavelet transform [6] that defines graph-domain wavelets by using a window function in the spectral domain.

In this Letter, we propose the graph extension of Slepian functions, which were introduced on regular domains by Slepian and colleagues [7, 8], to find a trade-off between temporal and spectral energy concentration. These functions have been extended to several other domains, including spherical ones in the context of geophysics [9]. Balancing the spread of graph signals in the original and the spectral domain relates to extensions of uncertainty principles [10], which have been explored by Tsitsvero et al. [11] based on a Slepian-type of constraints. Here, we take the design of graph Slepians one step further by establishing a link between the criterion of energy concentration and the one of Laplacian embedding and graph clustering. This leads to Slepians that are optimized w.r.t. a modified embedded distance, and that are particularly interesting to generalize graph signal processing operations.

II Defining Slepian Functions on Graphs

II-A Slepian’s Time-Frequency Concentration Problem

In seminal work, Slepian, Landau, and Pollak [7, 8] solved the fundamental problem of optimally concentrating a signal jointly in temporal and spectral domains. Here we briefly review the 1-D discrete-discrete case, which is directly relevant for the generalization that we will introduce in this work. We define the unitary discrete Fourier transform (DFT) for a length-$N$ signal by the $N\times N$ matrix

[TABLE]

where $\omega_{l}=2\pi(l-\lceil(N-1)/2\rceil)/N$. Then, the signal represented as a vector ${\bf f}$ can be Fourier transformed as $\hat{{\bf f}}={\bf F}^{H}{\bf f}$, where $\cdot^{H}$ indicates the Hermitian transpose. Due to Parseval, we have that $\|{\bf f}\|^{2}=\|\hat{{\bf f}}\|^{2}$, where the $\ell_{2}$-norm is defined as $\|{\bf f}\|^{2}=\left<{\bf f},{\bf f}\right>={\bf f}^{H}{\bf f}$.

The problem at hand is now to optimally concentrate a strictly band-limited signal ${\bf g}$ in a predefined interval defined by the set of indices $\mathcal{S}$ that does not necessarily need to be contiguous. Finding such an optimal signal in terms of energy concentration in $\mathcal{S}$ can be formulated as maximizing

[TABLE]

By definition, the band-limited signal can be represented by a linear combination of a truncated Fourier basis; i.e., we write ${\bf g}={\bf F}{\bf W}\hat{{\bf g}}$, where ${\bf W}$ of size $N\times N_{W}$ selects the first $N_{W}$ Fourier vectors with smallest $|\omega_{l}|$. In addition, we introduce the $N\times N$ diagonal selection matrix ${\bf S}$ with elements $S_{k,k}=0/1$ that indicate the presence of index $k$ in $\mathcal{S}$; so the number of selected indexes is $\text{trace}({\bf S})=N_{S}$. This allows us now to rewrite Eq. (1) as the Rayleigh quotient

[TABLE]

where ${\bf C}={\bf W}^{T}{\bf F}^{H}{\bf S}{\bf F}{\bf W}$ is the concentration matrix of size $N_{W}\times N_{W}$. Finding the optimal solution translates into an eigendecomposition problem, ${\bf C}\hat{{\bf s}}_{k}=\mu_{k}\hat{{\bf s}}_{k}$, where $\mu_{k}$ represents the energy concentration of the Slepian vector ${\bf s}_{k}={\bf F}{\bf W}\hat{{\bf s}}_{k}$ in $\mathcal{S}$.

The Slepian vectors are illustrated for a signal of length $N=512$. The selected interval is chosen centered at $k=256$ with length $N_{S}=129$. The bandlimited space contains $N_{W}=17$ Fourier basis vectors ($\left|\omega_{l}\right|\leq\pi/32$). In Fig. 1, the first Slepian vector ${\bf s}_{1}$ has a Gaussian shape and is well localized in the interval. The second Slepian vector ${\bf s}_{2}$ is orthogonal to the first one (both over the original domain and the selected interval as shown above) and still well localized. We then show ${\bf s}_{5}$ which has a weaker energy concentration in the interval. By convention, the eigenvalues of the Slepian eigendecomposition are sorted according to decreasing energy concentration ${1>\mu_{1}\geq\mu_{2}\geq\ldots>0}$ in $\mathcal{S}$. The eigenvalue spectrum presents a sudden transition between well-localized and poorly-localized eigenvectors; this phase transition has been well studied and occurs at the time-bandwidth product or Shannon number $K=N_{W}N_{S}/N$. The inset of Fig. 1 illustrates this phenomenon which happens around $K=4.3$ for this example.

II-B Graph Fourier Spectrum

For an undirected weighted graph with $N$ nodes, the connectivity is characterized by an $N\times N$ symmetric adjacency matrix ${\bf A}$, where the elements $A_{i,j}$ indicate the edge weights between nodes $i,j=1,\ldots,N$. Here we assume the graph is connected and the edge weights are positive. Next to the graph and its topology, we can also consider graph signals that define a mapping from the nodes to a length-$N$ vector associating a value with every node; e.g., the graph topology models the presence of communication links between nodes, while the graph signal represents measures taken at the nodes.

The graph Laplacian is defined as ${\bf L}^{\prime}={\bf D}-{\bf A}$, where ${\bf D}$ is the diagonal degree matrix with elements $D_{i,i}=\sum_{j=1}^{N}A_{i,j}$. We further consider the normalized graph Laplacian ${\bf L}={\bf D}^{-1/2}{\bf L}^{\prime}{\bf D}^{-1/2}$ that factors out differences in degree and thus is only reflecting relative connectivity.

Spectral methods for graphs are based on the insight that the eigendecomposition of the graph Laplacian gives access to a GFT. The eigenvectors ${\bf u}_{k}$ of ${\bf L}$ minimize

[TABLE]

and can be ordered according to increasing eigenvalues ${\lambda_{1}=0\leq\lambda_{2}\leq\ldots\leq\lambda_{N}}$. The eigenvectors play the role of basis vectors of the graph spectrum, and the associated eigenvalues of frequencies [12].

II-C Slepians on Graphs

To generalize Slepians to graphs, we follow the approach by [11], which is based on introducing selectivity and bandwidth. First, selectivity can be specified by a subset $\mathcal{S}$ that contains the $N_{S}$ nodes in which we want the energy concentration to be optimal. Similar to the 1-D case, we represent the subgraph by the selection matrix ${\bf S}$. Second, for the notion of “bandwidth”, we propose to restrict the spectrum to the eigenvectors with the $N_{W}<N$ smallest eigenvalues $\lambda$. We then consider the truncated graph spectrum matrix ${\bf U}{\bf W}$ of size $N\times N_{W}$; i.e., any band-limited graph signal can be represented by ${\bf g}={\bf U}{\bf W}\hat{{\bf g}}$.

Finding the linear combination of eigenvectors within the bandlimit $N_{W}$ and with maximal energy in $\mathcal{S}$ reverts to optimizing the Rayleigh quotient

[TABLE]

where ${\bf C}={\bf W}^{T}{\bf U}^{T}{\bf S}{\bf U}{\bf W}$ is the concentration matrix. Since ${\bf L}$ is real and symmetric, ${\bf U}$ is real as well and we revert to the regular transpose $\cdot^{T}$. The Slepian vectors are orthonormal over the entire graph as well as orthogonal over the interval $\mathcal{S}$; i.e., we have ${\bf g}_{k}^{T}{\bf g}_{l}=\delta_{k-l}$ as well as ${\bf g}_{k}^{T}{\bf S}{\bf g}_{l}=\mu_{k}\delta_{k-l}$, where $\delta$ is the Kronecker delta.

III Link with Laplacian Embedding and

Graph Clustering

The GFT plays a central role in Laplacian embedding and graph clustering. Specifically, in Laplacian embedding, the aim is to find a mapping from the nodes onto a line such that strongly connected nodes stay as close as possible, which can be expressed as [4]:

[TABLE]

where ${\bf g}^{T}{\bf g}=1$ and ${\bf g}^{T}{\bf 1}=0$. The solution is the eigenvector of the Laplacian with the smallest non-zero eigenvalue, the Fiedler vector [3]. The optimization (5) is also related to the classical graph cut problem that consists of partitioning the graph into clusters of nodes such that the cut size is minimal. In this setting, $g_{i}=\pm 1$ indicate the labels of the nodes, which can be relaxed to take any real values.

We now use the Slepian design to generalize the criterion (5) as to find a bandlimited solution that is optimized for a selected set of nodes. To reveal this link, we first rewrite the quadratic form (5) as ${\bf g}^{T}{\bf L}{\bf g}={\bf g}^{T}{\bf U}{\bf\Lambda}{\bf U}^{T}{\bf g}=\hat{{\bf g}}^{T}{\bf\Lambda}\hat{{\bf g}}.$ By observing the identity ${\bf\Lambda}={\bf\Lambda}^{1/2}{\bf U}^{T}{\bf U}{\bf\Lambda}^{1/2}$, we propose to generalize the criterion by introducing the bandwidth selection in the spectral, and node selection in the graph domain, respectively:

[TABLE]

where ${\bf\Lambda}_{W}={\bf W}^{T}{\bf\Lambda}{\bf W}$ is the band-limited $N_{W}\times N_{W}$ diagonal matrix. Therefore, we propose to solve the eigendecomposition of the modified concentration matrix ${\bf C}_{\text{emb}}={\bf\Lambda}^{1/2}_{W}{\bf C}{\bf\Lambda}^{1/2}_{W}$. As before, these Slepian vectors are double orthogonal as we have ${\bf g}_{k}^{T}{\bf g}_{l}=\delta_{k-l}$ and ${\bf g}_{k}^{T}{\bf S}{\bf g}_{l}=\xi_{k}\delta_{k-l}$.

This demonstrates that Laplacian embedding can be generalized as a Slepian problem with the additional weighting of the Laplacian eigenvalues. It is interesting to note that the eigenvalues $\xi$ of ${\bf C}_{\text{emb}}$ represent the modified embedded distance, or, equivalently, a “frequency” that is localized in the subgraph $\mathcal{S}$, whereas the eigenvalues $\mu$ of ${\bf C}$ represent the energy concentration in the subgraph.

IV Results & Discussion

Practically, the design only requires a limited number of eigenvectors $N_{W}$ that can be obtained using efficient large-scale solvers [13] such as ARPACK. The Slepian optimization itself is an eigendecomposition of a matrix of size $N_{W}\times N_{W}$.

We build the graph of a mesh structure with 4’567 nodes and 13’650 edges (9’078 faces), see Fig. 2a. In Fig. 3, the first $9$ Laplacian eigenvectors with smallest eigenvalues are represented. These graph signals correspond to our intuition about low-frequency Fourier basis vectors that represent slow variations along the principle “axes of variation”. The Fiedler vector is #2, which means that 1-D Laplacian embedding projects nodes according to the posterior-anterior axis; i.e., an optimal graph cut separates the front from the back.

We selected a subgraph that corresponds to the head of the animal (i.e., $N_{S}=1^{\prime}534$ nodes), see Fig. 2a. The bandwidth ratio $N_{W}/N$ is a design choice, which we choose here as $9/4^{\prime}567=0.2\%$. The Slepian vectors based on the concentration criterion of Eq. (4) are illustrated in Fig. 4a ordered with increasing eigenvalues; e.g., vector #9 has highest eigenvalue (0.995) and a strong positive signal in the lower part of the head, in particular, the snout region. The vectors based on the modified embedded distance of Eq. (6) are shown in Fig. 4b. For all solutions, we computed the Laplacian embedded distance ($\lambda$), the energy concentration ($\mu$), and the modified embedded distance ($\xi$). The values typeset in bold are true eigenvalues.

We then increased the bandwidth to $N_{W}=1^{\prime}000$ and observed how the eigenvalues distribute for the graph Laplacian and both Slepian designs. In Fig. 5, we plotted the corresponding values of $\lambda$, $\mu$, and $\xi$. We observe that the eigenvalues of the graph Laplacian are nearly linear in this part of the spectrum. In terms of energy concentration, both Slepian designs exhibit a phase transition known from the 1-D case, approximately at the classical Shannon number $K=N_{W}N_{S}/N=336$. The Slepian design based on energy concentration is showing a sharper transition band. Finally, the Slepian design based on the modified embedded distance leads to an almost linear behavior of the eigenvalues $\xi$ for well-concentrated Slepian vectors, which shows a similar behavior of the localized spectrum as for the graph Laplacian.

While the Slepian design based on energy concentration is directly relevant for extending uncertainty principles for graphs [10, 11], the proposed design based on the modified embedded distance is particularly interesting for graph signal processing since the eigenvalues reflect a localized frequency [5]. For instance, using again $N_{W}=1^{\prime}000$, the graph signal shown in Fig. 2b is filtered by applying the same window function to the graph spectrum (Fig. 2c) and to the Slepian spectrum (Fig. 2d). This demonstrates how the Slepian design can be used to combine spectral operations with controlled localization, and thus are an alternative to graph wavelet approaches [6, 14, 15]. Future work can focus on applications including signal recovery and interpolation [16], adapting the design for directed graphs [17], or investigating how the Slepian designs depend on graph regularity.