On the expressive power of linear algebra on graphs

TL;DR

This paper explores how linear algebra operations in the MATLANG language affect the ability to distinguish between graphs, providing a comprehensive analysis of its expressive power in graph query languages.

Contribution

It characterizes the expressive power of linear algebra-based graph query languages, specifically MATLANG, in differentiating graphs.

Findings

Linear algebra operations influence graph distinguishability.

Complete characterization of MATLANG's expressive power.

Implications for graph query language design.

Abstract

Most graph query languages are rooted in logic. By contrast, in this paper we consider graph query languages rooted in linear algebra. More specifically, we consider MATLANG, a matrix query language recently introduced, in which some basic linear algebra functionality is supported. We investigate the problem of characterising equivalence of graphs, represented by their adjacency matrices, for various fragments of MATLANG. A complete picture is painted of the impact of the linear algebra operations in MATLANG on their ability to distinguish graphs.

Equations

$$e:=X\,|\,\mathsf{op}_{1}\bigl{(}e_{1},\ldots,e_{p_{1}}\bigr{)}\,|\,\cdots\,|\,\mathsf{op}_{k}\bigr{(}e_{1},\ldots,e_{p_{k}}\bigr{)},$$

$$\mathsf{op}_{i}(e_{1}(X),\ldots,e_{p_{i}}(X))(\nu(X)):=\mathsf{op}_{i}\bigl{(}e_{1}(\nu(X)),\ldots,e_{p_{i}}(\nu(X))\bigr{)}$$

$$(e(A_G){)}^{\ast }\cdot T=(T^{\ast }\cdot e(A_G){)}^{\ast }=(e(A_H)\cdot T^{\ast }{)}^{\ast }=T\cdot (e(A_H){)}^{\ast },$$

$$(e(A_H){)}^{\ast }\cdot T^{\ast }=(T\cdot e(A_H){)}^{\ast }=(e(A_G)\cdot T{)}^{\ast }=T^{\ast }\cdot (e(A_G){)}^{\ast }.$$

$$\#{\mathsf{walk}}_k(X):=(1(X){)}^{\ast }\cdot X^k\cdot 1(X)$$

$$\left[\begin{array}{cccccc}0& 0& 1& 0& 0& 1\\ 0& 0& 0& 1& 1& 0\\ 1& 0& 0& 0& 1& 0\\ 0& 1& 0& 0& 0& 1\\ 0& 1& 1& 0& 0& 0\\ 1& 0& 0& 1& 0& 0\end{array}\right]\cdot \left[\begin{array}{cccccc}0& 0& \frac{1}{2}& 0& 0& \frac{1}{2}\\ 0& 0& 0& \frac{1}{2}& \frac{1}{2}& 0\\ \frac{1}{2}& 0& 0& 0& \frac{1}{2}& 0\\ 0& \frac{1}{2}& 0& 0& 0& \frac{1}{2}\\ 0& \frac{1}{2}& \frac{1}{2}& 0& 0& 0\\ \frac{1}{2}& 0& 0& \frac{1}{2}& 0& 0\end{array}\right]=\left[\begin{array}{cccccc}0& 0& \frac{1}{2}& 0& 0& \frac{1}{2}\\ 0& 0& 0& \frac{1}{2}& \frac{1}{2}& 0\\ \frac{1}{2}& 0& 0& 0& \frac{1}{2}& 0\\ 0& \frac{1}{2}& 0& 0& 0& \frac{1}{2}\\ 0& \frac{1}{2}& \frac{1}{2}& 0& 0& 0\\ \frac{1}{2}& 0& 0& \frac{1}{2}& 0& 0\end{array}\right]\cdot \left[\begin{array}{cccccc}0& 0& 1& 0& 0& 1\\ 0& 0& 0& 1& 1& 0\\ 1& 0& 0& 0& 1& 0\\ 0& 1& 0& 0& 0& 1\\ 0& 1& 1& 0& 0& 0\\ 1& 0& 0& 1& 0& 0\end{array}\right],$$

$$\left(\frac{1}{6}\right)\times(\mathbb{1}(X))^{*}\mskip 2.0mu{\cdot}\mskip 2.0mu\bigl{(}\operatorname{\mathsf{diag}}(X\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}(X)-0\times\mathbb{1}(X))\mskip 2.0mu{\cdot}\mskip 2.0mu\operatorname{\mathsf{diag}}(X\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}(X)-1\times\mathbb{1}(X))\\ {}\mskip 2.0mu{\cdot}\mskip 2.0mu\operatorname{\mathsf{diag}}(X\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}(X)-2\times\mathbb{1}(X))\bigr{)}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}(X),$$

$$\mathrm{diag}(1_{V_i})\cdot A_G\cdot 1_{V_j}=\mathsf{deg}(v,V_j)\times 1_{V_i},$$

$$m_j^{(i)}(X):=X\cdot b_j^{(i-1)}(X),\text{for }j=1,\dots ,{\ell }_{i-1}.$$

$$\mathbb{1}_{=c}^{(i),j}(X)=\left(\frac{1}{\prod_{c^{\prime}\in D^{(i)}_{j},c^{\prime}\neq c}(c-c^{\prime})}\right)\times\biggl{(}\Bigl{(}\!\!\!\!\!\!\prod_{c^{\prime}\in D^{(i)}_{j}\!\!,c^{\prime}\neq c}\operatorname{\mathsf{diag}}\bigl{(}m^{(i)}_{j}(X)-c^{\prime}\times\mathbb{1}(X)\bigr{)}\Bigr{)}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}(X)\biggr{)},$$

$$1_{=(c_1,\dots ,c_{{\ell }_{i-1}})}^{(i)}(X)=\mathrm{diag}(1_{=c_1}^{(i),1}(X))\cdot \cdots \cdot \mathrm{diag}(1_{=c_{{\ell }_{i-1}}}^{(i),{\ell }_{i-1}}(X))\cdot 1(X).$$

$${\mathsf{eqpart}}_i(X):=b_i^{(n)}(X),$$

$$\mathsf{binary\_diag}(X):=(\mathbb{1}(X))^{*}\mskip 2.0mu{\cdot}\mskip 2.0mu\bigl{(}(X\mskip 2.0mu{\cdot}\mskip 2.0muX-X)\mskip 2.0mu{\cdot}\mskip 2.0mu(X\mskip 2.0mu{\cdot}\mskip 2.0muX-X)\bigr{)}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}(X).$$

$$\mathsf{binary\_diag}\bigl{(}\operatorname{\mathsf{diag}}(\mathsf{eqpart}_{i}(A_{G}))\bigr{)}=[0]=\mathsf{binary\_diag}\bigl{(}\operatorname{\mathsf{diag}}(\mathsf{eqpart}_{i}(A_{H}))\bigr{)},$$

$$\mathbb{1}(X)^{*}\mskip 2.0mu{\cdot}\mskip 2.0mu\big{(}\sum_{i=1}^{\ell}\mathsf{equit}_{i}(A_{G})\bigr{)}=[n]=\mathbb{1}(X)^{*}\mskip 2.0mu{\cdot}\mskip 2.0mu\big{(}\sum_{i=1}^{\ell}\mathsf{equit}_{i}(A_{H})\bigr{)},$$

$$\mathrm{diag}({\mathsf{eqpart}}_i(A_G))\cdot A_G\cdot \mathrm{diag}({\mathsf{eqpart}}_j(A_G))\cdot 1-\mathrm{deg}(v,V_j)\times {\mathsf{eqpart}}_i(A_G)$$

$$\operatorname{\mathsf{diag}}\Bigl{(}\operatorname{\mathsf{diag}}(\mathsf{eqpart}_{i}(X))\mskip 2.0mu{\cdot}\mskip 2.0muX\mskip 2.0mu{\cdot}\mskip 2.0mu\operatorname{\mathsf{diag}}(\mathsf{eqpart}_{j}(X))\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}(X)-\deg(v,V_{j})\times\mathsf{eqpart}_{i}(X)\Bigr{)}$$

$$\mathrm{diag}({\mathsf{eqpart}}_i(A_H))\cdot A_H\cdot \mathrm{diag}({\mathsf{eqpart}}_j(A_H))\cdot 1-\mathrm{deg}(v,V_j)\times {\mathsf{eqpart}}_i(A_H)$$

$$zerotest\_diag(X):=(1(X){)}^{\ast }\cdot X\cdot X\cdot 1(X),$$

$$zerotest\_diag({equi\_test}_{ij}(A_G))=[0]=zerotest\_diag({equi\_test}_{ij}(A_H)),$$

$$\mathrm{diag}(1_{V_i})\cdot T=T\cdot \mathrm{diag}(1_{W_i}).$$

$$e(A_G)=\sum _{i=1}^{\ell }{a}_i\times 1_{V_i},$$

$$e^{\prime }(A_G)=\mathrm{diag}(e(A_G))=e(A_G)=e(A_H)=\mathrm{diag}(e(A_H))=e^{\prime }(A_H).$$

$$1_{V_i}=\mathrm{diag}(1_{V_i})\cdot 1=\mathrm{diag}(1_{V_i})\cdot T\cdot 1=T\cdot \mathrm{diag}(1_{W_i})\cdot 1=T\cdot 1_{W_i}.$$

$$a_i\times |V_i|$$

$$=\sum _{j=1}^{\ell }{b}_j\times (1_{V_i}^{\mathsf{t}}\cdot T\cdot 1_{W_j}^{})=b_i\times |W_i|,$$

$$e^{\prime }(A_G)\cdot T$$

$$=\sum _{i=1}^{\ell }{a}_i\times (T\cdot \mathrm{diag}(1_{W_i}))=T\cdot \mathrm{diag}(e(A_H))=T\cdot e^{\prime }(A_H).$$

$$\mathsf{tr}(w(A_1,\dots ,A_p))=\mathsf{tr}(w(B_1,\dots ,B_p)),$$

$$A_G\cdot O$$

$$J\cdot O$$

$$\mathrm{diag}(1_{V_i})\cdot O$$

Full text

More specifically, we consider $\mathsf{MATLANG}$, a matrix query language recently introduced, in which some basic linear algebra functionality is supported. We investigate the problem of characterising the equivalence of graphs, represented by their adjacency matrices, for various fragments of $\mathsf{MATLANG}$. That is, we are interested in understanding when two graphs cannot be distinguished by posing queries in $\mathsf{MATLANG}$ on their adjacency matrices.

Surprisingly, a complete picture can be painted of the impact of each of the linear algebra operations supported in $\mathsf{MATLANG}$ on their ability to distinguish graphs. Interestingly, these characterisations can often be phrased in terms of spectral and combinatorial properties of graphs.

Furthermore, we also establish links to logical equivalence of graphs. In particular, we show that $\mathsf{MATLANG}$-equivalence of graphs corresponds to equivalence by means of sentences in the three-variable fragment of first-order logic with counting. Equivalence with regards to a smaller $\mathsf{MATLANG}$ fragment is shown to correspond to equivalence by means of sentences in the two-variable fragment of this logic.

1 Introduction

Motivated by the importance of linear algebra for machine learning on big data [9, 10, 17, 58, 67] there is a current interest in languages that combine matrix operations with relational query languages in database systems [28, 46, 52, 53, 56]. Such hybrid languages raise many interesting questions from a database theoretical point of view. The Lara language is one such proposal [46] and its connections to classical database query languages has been recently explored [7]. It seems natural, however, to first consider query languages for matrices alone. These are the focus of this paper.

More precisely, we continue the investigation of the expressive power of the matrix query language $\mathsf{MATLANG}$, recently introduced by Brijder et al. [11, 12], as an analog for matrices of the relational algebra on relations. Intuitively, queries in $\mathsf{MATLANG}$ are built up by composing several linear algebra operations commonly found in linear algebra packages. When arbitrary matrices are concerned, it is known that $\mathsf{MATLANG}$ is subsumed by aggregate logic with only three non-numerical variables. This implies, among other things, that when evaluated on adjacency matrices of graphs, $\mathsf{MATLANG}$ cannot compute the transitive closure of a graph and neither can it express the four-variable query asking if a graph contains a four-clique [11, 12].

In fact, it is implicit in the work by Brijder et al. that when two graphs $G$ and $H$ are indistinguishable by sentences in the three-variable fragment $\mathsf{C}^{3}$ of first-order logic with counting, denoted by $G\equiv_{\mathsf{C}^{3}}H$, then their adjacency matrices cannot be distinguished by $\mathsf{MATLANG}$ expressions that return scalars, henceforth referred to as sentences in $\mathsf{MATLANG}$. The equivalence with respect to such sentences is denoted by $G\equiv_{\mathsf{MATLANG}}H$. A natural question is whether the converse implication also holds, i.e., does $G\equiv_{\mathsf{MATLANG}}H$ also imply $G\equiv_{\mathsf{C}^{3}}H$? We answer this question affirmatively.

The underlying proof technique relies on a close connection between $\mathsf{C}^{3}$-equivalence and the indistinguishability of graphs by the $2$-dimensional Weisfeiler-Lehman ($2\mathsf{WL}$) algorithm, a result dating back to the seminal paper by Cai, Fürer and Immerman [15, 47]. Indeed, as we will see, the linear algebra operations supported in $\mathsf{MATLANG}$ have sufficient power to simulate the $2\mathsf{WL}$ algorithm. Hence, when $G\equiv_{\mathsf{MATLANG}}H$, then $G$ and $H$ cannot by distinguished by the $2\mathsf{WL}$ algorithm.

This combinatorial interpretation of $\mathsf{MATLANG}$-equivalence immediately provides an insight in which graph properties are preserved under $\mathsf{MATLANG}$-equivalence (see e.g., the work by Fürer [31, 32]). For example, when $G\equiv_{\mathsf{MATLANG}}H$, then $G$ and $H$ must be co-spectral (that is, their adjacency matrices have the same multi-set of eigenvalues) and have the same number of $s$-cycles, for $s\leq 6$, but not necessarily for $s>7$. As observed in the conference version of this paper [33], the case of $7$-cycles easily follows from the connection with $\mathsf{MATLANG}$. Indeed, the linear algebra expressions for counting $s$-cycles, for $s\leq 7$, given in Noga et al. [1] are expressible in $\mathsf{MATLANG}$ and hence, $7$-cycles are preserved by $2\mathsf{WL}$-equivalence. This has recently been verified using other techniques by Arvind et al. [3]. Although formulas exist for counting cycles of length greater than $7$ [1], they require counting the number of $k$-cliques, for $k\geq 4$, which is not possible in $\mathsf{MATLANG}$, as observed earlier.

Apart from the logical and spectral/combinatorial characterisation of $\mathsf{MATLANG}$-equivalence, we also point out the correspondence between $\mathsf{C}^{3}$-equivalence (and thus also $2\mathsf{WL}$-equivalence and $\mathsf{MATLANG}$-equivalence) and conjugacy conditions between adjacency matrices. Roughly speaking, a conjugacy condition refers to a relationship between adjacency matrices of the form $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muT=T\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ for some matrix $T$. Here, $A_{G}$ and $A_{H}$ denote the adjacency matrices of $G$ and $H$, respectively. As observed by Dawar et al. [25, 26], $G\equiv_{\mathsf{C}^{3}}H$ if and only if there exists a unitary matrix $U$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muU=U\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ and moreover, $U$ induces an algebraic isomorphism between the so-called coherent algebras of $A_{G}$ and $A_{H}$. We recall that a unitary matrix $U$ is a complex matrix whose inverse is its complex conjugate transpose $U^{*}$. Coherent algebras and their isomorphisms are detailed later in the paper.

All combined, we have a logical, combinatorial and conjugation-based characterisation of $\mathsf{MATLANG}$-equivalence. Surprisingly, similar characterisations hold also for fragments of $\mathsf{MATLANG}$. We define fragments of $\mathsf{MATLANG}$ by allowing only certain linear algebra operations in our expressions. Such fragments are denoted by $\mathsf{ML}({\cal L})$, with ${\cal L}$ the list of allowed operations. The corresponding notion of equivalence of graphs $G$ and $H$ will be denoted by $G\equiv_{\mathsf{ML}(\cal L)}H$. That is, $G\equiv_{\mathsf{ML}(\cal L)}H$ if any sentence in $\mathsf{ML}(\cal L)$ results in the same scalar when evaluated on $A_{G}$ and $A_{H}$. We investigate equivalence for all sensible $\mathsf{MATLANG}$ fragments. Our results are as follows:

For starters, we consider the fragment $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$ that allows for matrix multiplication ($\mskip 2.0mu{\cdot}\mskip 2.0mu$) and trace ($\mathsf{tr}$) computation (i.e., taking the sum of the diagonal elements of a matrix). Then, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})}H$ if and only if $G$ and $H$ are co-spectral, or equivalently, they have the same number of closed walks of any length, or $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ for some orthogonal matrix $O$. We recall that an orthogonal matrix $O$ is a matrix over the real numbers such that its inverse coincides with the transpose matrix $O^{\mathsf{t}}$ (Section 5).

Another small fragment, $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$, allows for matrix multiplication, conjugate transposition (∗) and the use of the column vector $\mathbb{1}$, consisting of all ones. Then, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})}H$ if and only if $G$ and $H$ are co-main (roughly speaking, they are co-spectral only for special “main” eigenvalues), or equivalently, they have the same number of (not necessarily closed) walks of any length, or $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muQ=Q\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ for some doubly quasi-stochastic matrix $Q$. A doubly quasi-stochastic matrix $Q$ is a matrix over the real numbers such that every of its columns and rows sums up to one (Section 6).

When allowing matrix multiplication, $\mathsf{tr}$, ∗, and $\mathbb{1}$, equivalence of graphs relative to $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},\allowbreak{}^{*},\mathbb{1})$ coincides, not surprisingly, to the graphs being both co-spectral and co-main, or equivalently, having the same number of closed walks of any length and the same number of non-closed walks of any length, or such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$, for an orthogonal doubly quasi-stochastic matrix $O$ (Section 6).

More interesting is the fragment $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$, which additionally allows for the operation $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ that turns a column vector into a diagonal matrix with that vector on its diagonal. For this fragment we can tie equivalence to indistinguishability by the $1$-dimensional Weisfeiler-Lehman ($1\mathsf{WL}$) algorithm (or colour refinement). This is known to coincide with the graphs having a common equitable partition, or the existence of a doubly stochastic matrix $S$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muS=S\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ (a.k.a. as a fractional isomorphism), or $\mathsf{C}^{2}$-equivalence. Here, $\mathsf{C}^{2}$ denotes the two-variable fragment of first-order logic with counting. We recall that a doubly stochastic matrix is a doubly quasi-stochastic matrix whose entries are all non-negative (Section 7).

In the former fragment, replacing the operation $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ with an operation ($\odot_{v}$) which pointwise multiplies vectors results in the same distinguishing power. By contrast, the combination of $\mathsf{tr}$ and the ability to pointwise multiply vectors results in a stronger notion of equivalence. That is, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})}H$ if and only if $G$ and $H$ are co-spectral and indistinguishable by $1\mathsf{WL}$. Also in this case, $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ for an orthogonal matrix $O$ that, in addition, needs to preserve equitable partitions. We define this preservation condition later in the paper (Section 8).

For the larger fragment $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$, no elegant combinatorial characterisation is obtained. Nevertheless, for equivalent graphs $G$ and $H$, $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ where $O$ is an orthogonal matrix that can be block-structured according to the equitable partitions. This is a stronger notion than the preservation of equitable partitions. Graphs equivalent with respect to this fragment have, for example, the same number of spanning trees. This is not necessarily true for all previous fragments (Section 7).

Finally, as we already mentioned, equivalence relative to $\mathsf{MATLANG}$ is shown to correspond to $\mathsf{C}^{3}$-equivalence and $2\mathsf{WL}$-equivalence. We additionally refine the conjugation-based characterisation given by Dawar et al. [25, 26] so that it compares more easily to the conjugacy notions used for all previous fragments. Furthermore, we show that pointwise multiplication of matrices (the Schur-Hadamard product) is crucial in this setting (Section 9).

Each of these fragments can be extended with addition and scalar multiplication at no increase in distinguishing power. It is also shown when fragments can be extended to accommodate for arbitrary pointwise function applications, on scalars, vectors or matrices. We furthermore exhibit example graphs separating all fragments.

For many of our characterisations we rely on the rich literature on spectral graph theory [14, 19, 20, 21, 35, 42, 65, 72] and the study of the equivalence by the Weisfeiler-Lehman algorithms and fixed-variable fragments of first-order logic with counting [25, 26, 27, 38, 47, 63, 69, 70, 73]. We describe the relevant results in these papers in due course. We also refer to work by Fürer [31, 32] for more examples of connections to graph invariants and to Dawar et al. [25, 26] for connections between logic, combinatorial and spectral invariants.

In some sense, we provide a unifying view of various existing results in the literature by grouping them according to the operations supported in $\mathsf{MATLANG}$. We remark that, recently, another unifying approach has been put forward by Dell et al. [27]. In that work, one considers indistinguishability of graphs in terms of homomorphism vectors. That is, one defines $\textsf{HOM}_{\cal F}(G):=(\textsf{Hom}(F,G))_{F\in{\cal F}}$ for some class ${\cal F}$ of graphs, where $\textsf{Hom}(F,G)$ is the number of homomorphisms from $F$ to $G$. Then $G$ and $H$ are indistinguishable for some class ${\cal F}$ of graphs when $\textsf{HOM}_{\cal F}(G)=\textsf{HOM}_{\cal F}(H)$. When ${\cal F}$ consists of all cycles, this notion of equivalence corresponds to $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$-equivalence (recall the closed walk characterisation of the latter); when ${\cal F}$ consists of all paths, we have a correspondence with $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$-equivalence (recall the walk characterisation of the latter); when ${\cal F}$ consists of trees, $G$ and $H$ are equivalent for the $1\mathsf{WL}$-algorithm and thus also for $\mathsf{C}^{2}$ and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$, and finally, when ${\cal F}$ consists of all graphs of tree-width at most $2$, $G$ and $H$ are equivalent for the $2\mathsf{WL}$-algorithm and thus also for $\mathsf{C}^{3}$ and $\mathsf{MATLANG}$. Our results can thus be regarded as a re-interpretation of the results in Dell et al. [27] in terms of $\mathsf{MATLANG}$.

We also remark that $\mathsf{C}^{k}$-equivalence, for $k\geq 4$, can be characterised in terms of solutions to linear problems which resemble conjugation-based characterisations [4, 39, 57]. We leave it to future work to identify which additional linear algebra operations to include in $\mathsf{MATLANG}$ such that $\mathsf{C}^{k}$-equivalence can be captured, for $k\geq 4$.

Although we made links to logics such as $\mathsf{C}^{2}$ and $\mathsf{C}^{3}$, the connection between $\mathsf{MATLANG}$, rank logics and fixed-point logics with counting, as studied in the context of the descriptive complexity of linear algebra [23, 22, 24, 37, 40, 45], is yet to be explored. Similarly for connections to logic-based graph query languages [2, 6]. We also mention that $\mathsf{MATLANG}$ can be interpreted as a relational query language on so-called $K$-relations. Such $K$-relations are standard database relations which are annotated with values from a semiring $K$ [13]. This connection provides an elegant formalism of bridging linear algebra and relational algebra. It further opens the way to explore $\mathsf{MATLANG}$ for matrices whose elements are semiring values.

We want to emphasise that in this work we only consider indistinguishability of graphs by means of matrix query languages. As such, our results do not directly imply which matrix functions can be computed by $\mathsf{MATLANG}$ expressions, in a uniform dimension-independent way. This is in contrast to the expressiveness results in Brijder et al. [11, 12]. Furthermore, we focus only on undirected graphs in this paper. Such graphs have symmetric adjacency matrices which have many desirable linear algebra properties, diagonalisability being the most important one. Finally, $\mathsf{MATLANG}$ is a language in which expressions can combine multiple input matrices. Since our focus is on distinguishing graphs, in this work we restrict $\mathsf{MATLANG}$ such that its expressions only take a single matrix, i.e., the adjacency matrix, as input. Some of our results generalise to directed graphs (with asymmetric adjacency matrices) or even arbitrary matrices. This is explored in an upcoming paper [34]. A full treatment of the general setting with multiple inputs is left as future work.

This paper is an extended version of the ICDT 2019 conference paper [33]. It extends that version by including all proofs in detail. Furthermore, the overall presentation and underlying proof techniques have been simplified. In addition, a new section (Section 8) has been added in which the difference between the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ and the $\odot_{v}$ operations is investigated.

2 Background

We denote the set of real numbers by $\mathbb{R}$ and the set of complex numbers by $\mathbb{C}$. The set of $m\times n$-matrices over the real (resp., complex) numbers is denoted by $\mathbb{R}^{m\times n}$ (resp., $\mathbb{C}^{m\times n}$). Column vectors are elements of $\mathbb{R}^{m\times 1}$ (or $\mathbb{C}^{m\times 1}$). Row vectors are elements of $\mathbb{R}^{1\times m}$ (or $\mathbb{C}^{1\times m}$). The entries of an $m\times n$-matrix $A$ are denoted by $A_{ij}$, for $i=1,\ldots,m$ and $j=1,\ldots,n$. The entries of a (column or row) vector $v$ are denoted by $v_{i}$, for $i=1,\ldots,m$. We often identify $\mathbb{R}^{1\times 1}$ with $\mathbb{R}$, and $\mathbb{C}^{1\times 1}$ with $\mathbb{C}$ and refer to these as scalars. Moreover, the $i$th row and $j$th column of a matrix $A\in\mathbb{R}^{m\times n}$ (or in $\mathbb{C}^{m\times n}$) are denoted by $A_{i*}$ and $A_{*j}$, respectively, for $i=1,\ldots,m$ and $j=1,\ldots,n$.

The following classes of matrices are of interest in this paper: square matrices (elements in $\mathbb{R}^{n\times n}$ or $\mathbb{C}^{n\times n}$), invertible matrices (square matrices $A$ for which there exists an inverse matrix $B$ such that $A\mskip 2.0mu{\cdot}\mskip 2.0muB=I=B\mskip 2.0mu{\cdot}\mskip 2.0muA$, where $\mskip 2.0mu{\cdot}\mskip 2.0mu$ denotes matrix multiplication, and $I$ is the identity matrix in $\mathbb{C}^{n\times n}$), symmetric matrices (such that $A_{ij}=A_{ji}$ for all $i$ and $j$), stochastic matrices ($A_{ij}\in\mathbb{R}$, $A_{ij}\geq 0$, $\sum_{j=1}^{n}A_{ij}=1$ for all $i$), doubly stochastic matrices ($A_{ij}\in\mathbb{R}$, $A_{ij}\geq 0$, $\sum_{j=1}^{n}A_{ij}=1$ and $\sum_{i=1}^{m}A_{ij}=1$ for all $i$ and $j$), quasi-stochastic matrices ($A_{ij}\in\mathbb{R}$, $\sum_{j=1}^{n}A_{ij}=1$ for all $i$), doubly quasi-stochastic matrices ($A_{ij}\in\mathbb{R}$, $\sum_{j=1}^{n}A_{ij}=1$ and $\sum_{i=1}^{m}A_{ij}=1$ for all $i$ and $j$), and orthogonal matrices (invertible real matrices whose inverse matrix is $O^{\mathsf{t}}$, where $O^{\mathsf{t}}$ denotes the transpose of $O$ obtained by switching rows and columns). The matrix $J\in\mathbb{R}^{n\times n}$ denotes the matrix consisting of all ones and $Z\in\mathbb{R}^{n\times n}$ denotes the zero matrix. We often do not specify the dimensions of matrices and vectors, as these will be clear from the context.

We consider undirected graphs without self-loops. Let $G=(V,E)$ be such a graph with vertices $V=\{1,\ldots,n\}$ and unordered edges $E\subseteq\{\{i,j\}\mid i,j\in V\}$. The order of $G$ is simply the number of vertices. Then, an adjacency matrix of a graph $G$ of order $n$, denoted by $A_{G}$, is an $n\times n$-matrix whose entries $(A_{G})_{ij}$ are set to $1$ if and only if $\{i,j\}\in E$, all other entries are set to [math]. Strictly speaking, an adjacency matrix requires an ordering on the vertices in $G$. In this paper, this ordering is irrelevant and we often speak about “the” adjacency matrix of a graph. For undirected graphs $G=(V,E)$, the adjacency matrix $A_{G}$ is a symmetric binary matrix with zeroes on its diagonal.

An eigenvalue of a matrix $A$ is a scalar $\lambda$ in $\mathbb{C}$ for which there is a non-zero vector $v$ satisfying $A\mskip 2.0mu{\cdot}\mskip 2.0muv=\lambda v$. Such a vector is called an eigenvector of $A$ for eigenvalue $\lambda$. The eigenspace of an eigenvalue is the vector space obtained as the span of a maximal set of linear independent eigenvectors for this eigenvalue. Here, the span of a set of vectors just refers to the set of all linear combinations of vectors in that set. A set of vectors is linear independent if no vector in that set can be written as a linear combination of other vectors. The dimension of an eigenspace is the number of linearly independent eigenvectors spanning that space. The spectrum of an undirected graph can be represented as $\mathsf{spec}(G)=\begin{pmatrix}\lambda_{1}&\lambda_{2}&\cdots&\lambda_{p}\\ m_{1}&m_{2}&\cdots&m_{p}\end{pmatrix}$, where $\lambda_{1}<\lambda_{2}<\cdots<\lambda_{p}$ are the distinct real eigenvalues of the adjacency matrix $A_{G}$ of $G$, and where $m_{1},m_{2},\ldots,m_{p}$ denote the dimensions of the corresponding eigenspaces. Two graphs are said to be co-spectral if they have the same spectrum.

We use $\mathsf{C}^{k}$ to denote the $k$-variable fragment of first-order logic with counting. More precisely, formulas in $\mathsf{C}^{k}$ are built up from a binary relation $R(x,y)$ (encoding the edge relation of a graph), disjunction, conjunction, negation and counting quantifiers $\exists^{\geq m}$ and use at most $k$ distinct variables [59]. A sentence in $\mathsf{C}^{k}$ is a formula without free variables. Two graphs $G$ and $H$ are equivalent with regards to $\mathsf{C}^{k}$, denoted by $G\equiv_{\mathsf{C}^{k}}H$, if $G\models\varphi$ if and only if $H\models\varphi$ for every sentence $\varphi$ in $\mathsf{C}^{k}$. Here, $\models$ denotes the standard notion of satisfaction of logical formulas (see e.g., [55, 59]).

3 Matrix query languages

As described in Brijder et al. [11], matrix query languages can be formalised as compositions of linear algebra operations. Intuitively, a linear algebra operation takes a number of matrices as input and returns another matrix. Examples of operations are matrix multiplication, conjugate transposition, computing the trace, just to name a few. By closing such operations under composition “matrix query languages” are formed. More specifically, for linear algebra operations $\mathsf{op}_{1},\ldots,\mathsf{op}_{k}$ the corresponding matrix query language is denoted by $\mathsf{ML}(\mathsf{op}_{1},\ldots,\mathsf{op}_{k})$ and consists of expressions formed by the following grammar:

[TABLE]

where $X$ denotes a matrix variable from an infinite set of variables, which serves to indicate the inputs to expressions, and $p_{i}$ denotes the number of inputs required by operation $\mathsf{op}_{i}$. As mentioned in the introduction, we focus on the case when only a fixed single matrix variable, denoted by $X$, is allowed in expressions. We denote expressions by $e(X)$ to make this explicit.

The semantics of an expression $e(X)$ in $\mathsf{ML}(\mathsf{op}_{1},\ldots,\mathsf{op}_{k})$ is defined inductively, relative to an assignment $\nu$ of $X$ to a matrix $\nu(X)\in\mathbb{C}^{m\times n}$, for some dimensions $m$ and $n$. We denote by e\bigl{(}\nu(X)\bigr{)} the result of evaluating $e(X)$ on $\nu(X)$. We define, as expected,

[TABLE]

for linear algebra operation $\mathsf{op}_{i}$.

In this paper we regard $\mathsf{MATLANG}$ as the matrix query language built-up from the atomic operations listed in Table 1 and in which only a single matrix variable $X$ is used.

In the table we also show the semantics of the atomic operations. We note that restrictions on the dimensions are in place to ensure that operations are well-defined. Using a simple type system one can formalise a notion of well-formed expressions which guarantees that the semantics of such expressions is well-defined. We refer to Brijder et al. [11] for details. We only consider well-formed expressions from here on. As one can observe from Table 1, $\mathsf{MATLANG}$ is parameterised by a set $\Omega$ of pointwise functions (see the last operation in Table 1). More specifically, $\Omega=\bigcup_{p>0}\Omega_{p}$, where $\Omega_{p}$ consists of some functions $f:\mathbb{C}^{p}\to\mathbb{C}$. The choice of $\Omega$ does not impact our results. Hence, we can take $\Omega$ to consist of all possible pointwise functions.

Remark 3.1**.**

The list of operations in Table 1 differs slightly from the list presented in Brijder et al. [11]: We explicitly mention the trace operation ($\mathsf{tr}$) and pointwise function applications for: scalar multiplication ($\times$), addition ($+$), pointwise product of vectors ($\odot_{v}$) and pointwise product of matrices, also called the Schur-Hadamard product ($\odot$). We remark that all of these operations can be expressed in the language proposed in Brijder et al. [11]. Conversely, any single matrix variable expression in the language of Brijder et al. [11] can be expressed in our $\mathsf{MATLANG}$ language.

Remark 3.2**.**

The choice of operations included in $\mathsf{MATLANG}$ is motivated by operations supported in linear algebra packages such as MAPLE, MATLAB, MATHEMATICA, R, and others [11]. In $\mathsf{MATLANG}$, we only include what we believe to be atomic operations, from which more complex operations can be derived. Many other, more complex, operations could of course be added. Hence $\mathsf{MATLANG}$ is just a starting point. We mention that extensions of $\mathsf{MATLANG}$ with an $\mathsf{inverse}$ operation (taking the inverse of a matrix if it exists) and an $\mathsf{eigen}$ operation (returning eigenvalues and eigenvectors) are consider Brijder et al. [11]. The precise impact of these operations on the expressive power is yet to be understood.

In the following, when $\mathcal{L}$ is a subset of the operations from Table 1, we denote by $\mathsf{ML}(\cal L)$ the fragment of $\mathsf{MATLANG}$ in which only the operations in ${\cal L}$ are supported.

4 Expressive power of matrix query languages

As mentioned in the introduction, we are interested in the expressive power of matrix query languages. In analogy with indistinguishability notions used in logic, we consider sentences in our matrix query languages. Let ${\cal L}$ be a subset of the operations supported in $\mathsf{MATLANG}$. We define an expression $e(X)$ in $\mathsf{ML}(\cal L)$ to be a sentence if $e(\nu(X))$ returns a $1\times 1$-matrix (i.e., a scalar) for any assignment $\nu$ of the matrix variable $X$ in $e(X)$. We note that the type system of $\mathsf{MATLANG}$ allows to easily check whether an expression in $\mathsf{ML}(\cal L)$ is a sentence (see Brijder et al. [11] for more details). Having defined sentences, a notion of equivalence naturally follows.

Definition 4.1**.**

Two matrices $A$ and $B$ in $\mathbb{C}^{m\times n}$ are said to be $\mathsf{ML}(\cal L)$-equivalent, denoted by $A\equiv_{\mathsf{ML}(\cal L)}\!B$, if and only if $e(A)=e(B)$ for all sentences $e(X)$ in $\mathsf{ML}(\cal L)$.

In other words, equivalent matrices cannot be distinguished by sentences in the matrix query language fragment under consideration. One could imagine defining equivalence with regards to arbitrary expressions, i.e., expressions in $\mathsf{MATLANG}$ that are not necessarily sentences. Such a notion would be too strong, however. Indeed, requiring that $e(A)=e(B)$ for arbitrary expressions $e(X)$ would imply that $A=B$ (just consider $e(X):=X)$) and then the story ends.

We aim to characterise equivalence of matrices for various matrix query languages. We will, however, not treat this problem in full generality and instead only consider equivalence of adjacency matrices of undirected graphs. The reason for this limitation is two-fold. First, we can benefit from existing results from graph theory; second, undirected graphs have symmetric adjacency matrices and symmetric matrices have desirable linear algebra properties. For example, every symmetric matrix is diagonalisable. We rely on the properties of symmetric matrices in the proofs of our results.

Definition 4.1, when applied to adjacency matrices, naturally results in the following notion of equivalence of graphs.

Definition 4.2**.**

Two graphs $G$ and $H$ of the same order are said to be $\mathsf{ML}(\cal L)$-equivalent, denoted by $G\equiv_{\mathsf{ML}(\cal L)}H$, if and only if their adjacency matrices are $\mathsf{ML}(\cal L)$-equivalent.

In the following sections we consider equivalence of graphs for various fragments $\mathsf{ML}(\cal L)$, starting from simple fragments only supporting a couple of linear algebra operations, up to the full $\mathsf{MATLANG}$ matrix query language.

5 Expressive power of the matrix query language $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$

We start, in Section 5.1, by considering the equivalence of graphs for $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$, i.e., the matrix query language in which only matrix multiplication and the trace operation are supported. We further introduce the notion of conjugacy of matrices, which will be used throughout the paper. In Section 5.2, we then explore which operations can be added to $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$ without increasing its distinguishing power.

5.1 $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$-equivalence

The matrix query language $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$ is a very restrictive fragment. Indeed, the only sentences that one can formulate in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$ are of the form (i) $\#\mathsf{cwalk}_{k}(X):=\mathsf{tr}(X^{k})$, where $X^{k}$ stands for the $k^{\text{th}}$ power of $X$, i.e., $X$ multiplied $k$ times with itself, and (ii) products of such sentences. To make the connection to graphs, we recall the following notions. A walk of length $k$ in a graph $G=(V,E)$ is a sequence $(v_{0},v_{1},\ldots,v_{k})$ of vertices of $G$ such that consecutive vertices are adjacent in $G$, i.e., $\{v_{i-1},v_{i}\}\in E$ for all $i=1,\ldots,k$. Furthermore, a closed walk is a walk that starts in and ends at the same vertex. Closed walks of length [math] correspond, as usual, to vertices in $G$. We note that, when evaluated on the adjacency matrix $A_{G}$ of $G$, $\#\mathsf{cwalk}_{k}(A_{G})$ is equal to the number of closed walks of length $k$ in $G$. Indeed, an entry $(A_{G}^{k})_{vw}$ of the $k^{\text{th}}$ power $A_{G}^{k}$ of adjacency matrix $A_{G}$ can be easily seen to correspond to the number of walks from $v$ to $w$ of length $k$ in $G$. Hence, $\#\mathsf{cwalk}_{k}(A_{G})=\mathsf{tr}(A_{G}^{k})=\sum_{v\in V}(A_{G}^{k})_{vv}$ indeed corresponds to the number of closed walks of length $k$ in $G$.

The following (folklore) characterisations are known to hold.

Proposition 5.1**.**

Let $G$ and $H$ be two graphs of the same order. The following statements are equivalent:

1. (1)

$G$* and $H$ have the same number of closed walks of length $k$, for all $k\geq 0$;* 2. (2)

$\mathsf{tr}(A_{G}^{k})=\mathsf{tr}(A_{H}^{k})$* for all $k\geq 0$;* 3. (3)

$G$* and $H$ are co-spectral; and* 4. (4)

there exists an orthogonal matrix $O$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$.

Proof.

For a proof of the equivalences (1) $\Leftrightarrow$ (2) $\Leftrightarrow$ (3) we refer to Proposition 1 in [25] (although these equivalences appeared in the literature many times before). The equivalence (3) $\Leftrightarrow$ (4) is also known (see e.g., Theorem 9-12 in [62]). ∎

Example 5.1**.**

The graphs $G_{1}$ (​  ) and $H_{1}$ (​  ) are the smallest pair (in terms of number of vertices) of non-isomorphic co-spectral graphs of the same order (see e.g., Figure 6.2 in [18]). From the previous proposition we then know that $G_{1}$ and $H_{1}$ have the same number of closed walks of any length. We note that the isolated vertex in $G_{1}$ ensures that $G_{1}$ and $H_{1}$ have the same number of vertices (and thus the same number of closed walks of length [math]).   ∎

As anticipated, sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$ can only extract information from adjacency matrices related to the number of closed walks in graphs. More precisely, we can add to Proposition 5.1 a fifth equivalent condition based on $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$-equivalence:

Proposition 5.2**.**

For two graphs $G$ and $H$ of the same order, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr})}H$ if and only if $G$ and $H$ have the same number of closed walks of any length.

Proof.

By definition, if $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr})}H$, then $e(A_{G})=e(A_{H})$ for any sentence $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$. This holds in particular for the sentences $\#\mathsf{cwalk}_{k}(X):=\mathsf{tr}(X^{k})$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$, for $k\geq 1$. Hence, $G$ and $H$ have indeed the same number of closed walks of length $k$, for $k\geq 1$. Furthermore, since $G$ and $H$ are of the same order and $A_{G}^{0}=A_{H}^{0}=I$ (by convention), $G$ and $H$ have also the same number of closed walks of length [math].

For the converse, if $G$ and $H$ have the same number of closed walks of any length, then the previous proposition tells that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ for some orthogonal matrix $O$. We next claim that when $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ holds for some orthogonal matrix $O$, then $e(A_{G})=e(A_{H})$ for any sentence $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$. In fact, this claim will follow from the more general Lemmas 5.1 and 5.2 below. We separate these lemmas from the current proof since we also need them later in the paper. ∎

We note that yet another interpretation of $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})}H$ can be given in terms of the homomorphism vectors mentioned in the introduction. That is, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})}H$ if and only if $\textsf{HOM}_{\cal F}(G)=\textsf{HOM}_{\cal F}(H)$ where ${\cal F}$ is the set of all cycles [27].

As mentioned in the proof of Proposition 5.2, we still need to show that if $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ holds for some orthogonal matrix $O$, then $e(A_{G})=e(A_{H})$ for any sentence $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$.

In more generality, when $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muT=T\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ holds for a (not necessarily invertible) matrix $T$, we say that $A_{G}$ and $A_{H}$ are $T$-conjugate. We remark that conjugation is not necessarily a symmetric relation, i.e., $A_{G}$ and $A_{H}$ can be $T$-conjugate whereas $A_{H}$ and $A_{G}$ may not be $T$-conjugate for the same matrix $T$. We also define the notion of $T$-conjugation for vectors and scalars, as is shown next.

Definition 5.2**.**

Let $n>1$. Let $T$ be a matrix in $\mathbb{C}^{n\times n}$. Two matrices $A$ and $B$ in $\mathbb{C}^{n\times n}$ are called $T$-conjugate if $A\mskip 2.0mu{\cdot}\mskip 2.0muT=T\mskip 2.0mu{\cdot}\mskip 2.0muB$. Two column vectors $A$ and $B$ in $\mathbb{C}^{n\times 1}$ are $T$-conjugate if $A=T\mskip 2.0mu{\cdot}\mskip 2.0muB$. Similarly, two row vectors $A$ and $B$ in $\mathbb{C}^{1\times n}$ are $T$-conjugate if $A\mskip 2.0mu{\cdot}\mskip 2.0muT=B$. Finally, if $A$ and $B$ are scalars in $\mathbb{C}$ (or elements in $\mathbb{C}^{1\times 1}$), then $A$ and $B$ are $T$-conjugate if $A=B$ (i.e., $T$-conjugation of scalars is simply equality).

In $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$ we allow for matrix multiplication and the trace operation. We first show that $T$-conjugation is preserved by matrix multiplication.

Lemma 5.1**.**

Let $\mathsf{ML}(\cal L)$ be any matrix query language fragment and let $G$ and $H$ be two graphs of the same order. Consider expressions $e_{1}(X)$ and $e_{2}(X)$ in $\mathsf{ML}(\cal L)$. If $e_{i}(A_{G})$ and $e_{i}(A_{H})$ are $T$-conjugate, for $i=1,2$, for some matrix $T$, then $e_{1}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{G})$ is also $T$-conjugate to $e_{1}(A_{H})\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{H})$ (provided, of course, that the multiplication is well-defined).

sketch.

The proof consists of a simple case analysis depending on the dimensions of $e_{1}(A_{G})$ and $e_{2}(A_{G})$ (or equivalently, the dimensions of $e_{1}(A_{H})$ and $e_{2}(A_{H})$) and by using the definition of $T$-conjugation. We refer for the full proof to the appendix. ∎

When considering the trace operation, we observe that $T$-conjugation is preserved by the trace operation, provide that $T$ is an invertible matrix.

Lemma 5.2**.**

Let $\mathsf{ML}(\cal L)$ be any matrix query language fragment and let $G$ and $H$ be two graphs of the same order. Let $e_{1}(X)$ be an expression in $\mathsf{ML}(\cal L)$. If $e_{1}(A_{G})$ and $e_{1}(A_{H})$ are $T$-conjugate for an invertible matrix $T$, then $\mathsf{tr}(e_{1}(A_{G}))$ and $\mathsf{tr}(e_{1}(A_{H}))$ are also $T$-conjugate.

Proof.

Let $e(X):=\mathsf{tr}(e_{1}(X))$. By assumption, $e_{1}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0muT=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{H})$ for an invertible matrix $T$ in case that $e_{1}(A_{G})$ is an $n\times n$-matrix with $n>1$, and $e_{1}(A_{G})=e_{1}(A_{H})$ in case that $e_{1}(A_{G})$ is a sentence. In the latter case, clearly also $e(A_{G})=\mathsf{tr}(e_{1}(A_{G}))=\mathsf{tr}(e_{1}(A_{H}))=e(A_{H})$. In the former case, we use the property that $\mathsf{tr}(T^{-1}\mskip 2.0mu{\cdot}\mskip 2.0muA\mskip 2.0mu{\cdot}\mskip 2.0muT)=\mathsf{tr}(A)$ for any matrix $A$ and invertible matrix $T$ (see e.g., Chapter 10 in [5] for a proof of this property). Hence, we have that $e(A_{G})=\mathsf{tr}(e_{1}(A_{G}))=\mathsf{tr}(T^{-1}\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0muT)=\mathsf{tr}(T^{-1}\mskip 2.0mu{\cdot}\mskip 2.0muT\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{H}))\allowbreak=\allowbreak\mathsf{tr}(I\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{H}))=\allowbreak\mathsf{tr}(e_{1}(A_{H}))=e(A_{H})$ holds, as desired. ∎

We remark that Lemmas 5.1 and 5.2 hold for any fragment $\mathsf{ML}(\cal L)$.

The claim at the end of the proof of Proposition 5.2, i.e., that $O$-conjugation of $A_{G}$ and $A_{H}$ indeed implies that $e(A_{G})=e(A_{H})$ for any sentence $e(X)\in\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$, now easily follows by induction on the structure of expressions, Indeed, since orthogonal matrices are invertible, Lemmas 5.1 and 5.2 imply that when $e_{1}(A_{G})$ and $e_{1}(A_{H})$, and $e_{2}(A_{G})$ and $e_{2}(A_{H})$ are $O$-conjugate for an orthogonal matrix $O$, then also $e_{1}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{G})$ and $e_{1}(A_{H})\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{H})$ are $O$-conjugate, and $\mathsf{tr}(e_{1}(A_{G}))$ and $\mathsf{tr}(e_{1}(A_{H}))$ are $O$-conjugate (i.e., equal). Hence, when $A_{G}$ and $A_{H}$ are $O$-conjugate, $e(A_{G})$ and $e(A_{H})$ are $O$-conjugate for any sentence $e(X)\in\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$. That is, $e(A_{G})=e(A_{H})$ for any sentence in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$.

5.2 Adding operations to $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$ without increasing its distinguishing power

We next investigate how much more $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$ can be extended whilst preserving the characterisation given in Proposition 5.2. Some more general observations will be made in this context, which will be used for other fragments later in the paper as well.

First, we consider the extension with scalar multiplication ($\times$) and addition ($+$).

Lemma 5.3**.**

Let $\mathsf{ML}(\cal L)$ be any matrix query language fragment. Let $e_{1}(X)$ and $e_{2}(X)$ be two expressions in $\mathsf{ML}(\cal L)$ and consider two graphs $G$ and $H$ of the same order. Then, if $e_{1}(A_{G})$ and $e_{1}(A_{H})$, and $e_{2}(A_{G})$ and $e_{2}(A_{H})$ are $T$-conjugate for some matrix $T$, then also $e_{1}(A_{G})+e_{2}(A_{G})$ and $e_{1}(A_{H})+e_{2}(A_{H})$ are $T$-conjugate, and $a\times e_{1}(A_{G})$ and $a\times e_{1}(A_{H})$ are $T$-conjugate for any scalar $a\in\mathbb{C}$.

Proof.

This is an immediate consequence of the definition of $T$-conjugation and that matrix multiplication is a bilinear operation, i.e., $(a\times A+b\times B)\mskip 2.0mu{\cdot}\mskip 2.0mu(c\times C+d\times D)=(a\times c)\times(A\mskip 2.0mu{\cdot}\mskip 2.0muC)+(a\times d)\times(A\mskip 2.0mu{\cdot}\mskip 2.0muD)+(b\times c)\times(B\mskip 2.0mu{\cdot}\mskip 2.0muC)+(b\times d)\times(B\mskip 2.0mu{\cdot}\mskip 2.0muD)$, for scalars $a$, $b$, $c$, $d\in\mathbb{C}$ and matrices or vectors $A,B,C$ and $D$. ∎

We next consider complex conjugate transposition (∗).

Lemma 5.4**.**

Let $\mathsf{ML}(\cal L)$ be any matrix query language fragment. Let $e(X)$ be an expression in $\mathsf{ML}(\cal L)$ and consider two graphs $G$ and $H$ of the same order. Then, if $e(A_{G})$ and $e(A_{H})$ are $T$-conjugate, and $e(A_{H})$ and $e(A_{G})$ are $T^{*}$-conjugate for some matrix $T$, then also $(e(A_{G}))^{*}$ and $(e(A_{H}))^{*}$ are $T$-conjugate, and $(e(A_{H}))^{*}$ and $(e(A_{G}))^{*}$ are $T^{*}$-conjugate.

Proof.

We distinguish between a number of cases, depending on the dimensions of $e(A_{G})$ (and hence also of $e(A_{H})$). Suppose that $e(A_{G})$ returns an $n\times n$-matrix for $n>1$. Then, by assumption $e(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0muT=T\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{H})$ and $e(A_{H})\mskip 2.0mu{\cdot}\mskip 2.0muT^{*}=T^{*}\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{G})$. It then follows, using that the operation ∗ is an involution ($(A^{*})^{*}=A$) and $(A\mskip 2.0mu{\cdot}\mskip 2.0muB)^{*}=B^{*}\mskip 2.0mu{\cdot}\mskip 2.0muA^{*}$, that

[TABLE]

and similarly,

[TABLE]

Furthermore, when $e(A_{G})$ is an $n\times 1$-vector for $n>1$, we have by assumption that $e(A_{G})=T\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{H})$ and $e(A_{H})=T^{*}\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{G})$. Hence, $(e(A_{G}))^{*}\mskip 2.0mu{\cdot}\mskip 2.0muT=(T^{*}\!\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{G}))^{*}=(e(A_{H}))^{*}$ and $(e(A_{H}))^{*}\mskip 2.0mu{\cdot}\mskip 2.0muT^{*}=(T\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{H}))^{*}=(e(A_{G}))^{*}$. Similarly, when $e(A_{G})$ is a $1\times n$-vector for $n>1$, one can verify that $((e(A_{G}))^{*}=T\mskip 2.0mu{\cdot}\mskip 2.0mu(e(A_{H}))^{*}$ and $(e(A_{H}))^{*}=T^{*}\mskip 2.0mu{\cdot}\mskip 2.0mu(e(A_{G}))^{*}$. Finally, if $e(A_{G})$ is a sentence then clearly $(e(A_{G}))^{*}=(e(A_{H}))^{*}$. ∎

We next consider pointwise function applications. Later in the paper we show that pointwise function applications on vectors or matrices do add expressive power. This is particularly true for pointwise multiplication of vectors ($\odot_{v}$) and of matrices ($\odot$). By contrast, when function applications are only allowed on scalars they do not add any expressive power. More specifically, let $\Omega$ be an arbitrary set of pointwise functions and let $f:\mathbb{C}^{k}\to\mathbb{C}$ be a function in $\Omega$. We denote by $\operatorname{\mathsf{apply}}_{\mathsf{s}}[f](e_{1},\ldots,\allowbreak e_{k})$ the application of $f$ on $e_{1}(X),\ldots,e_{k}(X)$ when each $e_{i}(X)$ is a sentence.

Lemma 5.5**.**

Let $\mathsf{ML}(\cal L)$ be any matrix query language fragment. Consider two graphs $G$ and $H$ of the same order and sentences $e_{1}(X),e_{2}(X),\ldots,e_{k}(X)$ in $\mathsf{ML}(\cal L)$. Let $f:\mathbb{C}^{k}\to\mathbb{C}$ be a function in $\Omega$. Suppose that for each $i=1,\ldots,k$, $e_{i}(A_{G})=e_{i}(A_{H})$ (i.e., they are $T$-conjugate for an arbitrary matrix $T$). Then also $\operatorname{\mathsf{apply}}_{\mathsf{s}}[f](e_{1}(A_{G}),\ldots,\allowbreak e_{k}(A_{G}))=\operatorname{\mathsf{apply}}_{\mathsf{s}}[f](e_{1}(A_{H}),\ldots,\allowbreak e_{k}(A_{H}))$ (i.e., they are $T$-conjugate as well).

Proof.

This is straightforward to verify since the result of a function $f:\mathbb{C}^{k}\to\mathbb{C}$ is fully determined by its input values. ∎

Given these lemmas, we can infer that the characterisation given in Proposition 5.2 remains to hold for $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},+,\times,{}^{*},\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],\allowbreak f\in\Omega)$-equivalence.

Corollary 5.3**.**

For two graphs $G$ and $H$ of the same order, we have that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr})}H$ if and only if $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},+,\allowbreak\times,\allowbreak\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)}H$.  ∎

Proof.

We only need to show that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr})}H$ implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},+,\times,\allowbreak\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)}H$. By Proposition 5.2, there exists an orthogonal matrix $O$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$. Furthermore, we have that $O^{*}\mskip 2.0mu{\cdot}\mskip 2.0muA_{G}=(A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO)^{*}=(O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H})^{*}=A_{H}\mskip 2.0mu{\cdot}\mskip 2.0muO^{*}$ since $A_{G}$ and $A_{H}$ are symmetric real matrices. Hence, $A_{H}$ and $A_{G}$ are $O^{*}$-conjugate. We also, importantly, observe that $O^{*}$ is an orthogonal matrix as well. Lemmas 5.1 and 5.2 then imply that $e(A_{G})$ and $e(A_{H})$ are $O$-conjugate, and $e(A_{H})$ and $e(A_{G})$ are $O^{*}$-conjugate for any expression $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$. Furthermore, Lemmas 5.3, 5.4 and 5.5 imply that addition, scalar multiplication, complex conjugate transposition and pointwise function applications on scalars preserve $O$ and $O^{*}$-conjugation. This in turn implies that $e(A_{G})=e(A_{H})$ for any sentence $e(X)\in\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},+,\times,\allowbreak\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)$. ∎

As a consequence, the graphs $G_{1}$ (​  ) and $H_{1}$ (​  ) from Example 5.1 cannot be distinguished by sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)$. As we will see later, including any other operation from Table 1, such as $\mathbb{1}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$, $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ or pointwise function applications on vectors or matrices, allows us to distinguish $G_{1}$ and $H_{1}$.

6 The impact of the $\mathbb{1}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ and ∗ operations

We next consider two fragments that support complex conjugate transposition ∗ and the operation $\mathbb{1}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$, which returns the all-ones column vector $\mathbb{1}$111We use $\mathbb{1}$ to denote the all-ones vector (of appropriate dimension) and use $\mathbb{1}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ (with brackets) for the corresponding one-vector operation.. More specifically, we consider the fragments $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$ and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1})$.

The presence of $\mathbb{1}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ allows to extract, in combination with ∗, other information from graphs than just the number of closed walks. Indeed, consider the sentence

[TABLE]

in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$ and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1})$. When applied on the adjacency matrix $A_{G}$ of a graph $G$, $\#\mathsf{walk}_{k}(A_{G})$ returns the number of (not necessarily closed) walks in $G$ of length $k$. In relation to the previous section, co-spectral graphs have the same number of closed walks of any length, yet do not necessarily have the same number of walks of any length. Similarly, graphs with the same number of walks of any length are not necessarily co-spectral. We illustrate this by the following example.

Example 6.1**.**

It can be verified that the co-spectral graphs $G_{1}$ (​  ) and $H_{1}$ (​  ) of Example 5.1 have $16$ versus $20$ walks of length $2$, respectively. As a consequence, $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$ and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1})$ can distinguish $G_{1}$ from $H_{1}$ by means of the sentence $\#\mathsf{walk}_{2}(X)$. By contrast, the graphs $G_{2}$ (​  ) and $H_{2}$ (​ ) are not co-spectral, yet have the same number of walks of any length. It is easy to see that $G_{2}$ and $H_{2}$ are not co-spectral (apart from verifying that their spectra are different): $H_{2}$ has $12$ closed walks of length $3$ (because of the triangles), whereas $G_{2}$ has no closed walks of length $3$. As a consequence, $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$ (and thus also $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1})$) can distinguish $G_{2}$ and $H_{2}$. We argue below that $G_{2}$ and $H_{2}$ have the same number of walks of any length and show that $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$ cannot distinguish $G_{2}$ and $H_{2}$.   ∎

The previous example illustrates the key difference between the fragments $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$ and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1})$. The former can only detect differences in the number of walks of certain lengths, the latter can detect differences in both the number of walks and the number of closed walks of certain lengths.

Graphs sharing the same number of walks of any length have been investigated before in spectral graph theory [19, 20, 21, 42, 65]. To state a spectral characterisation, the so-called main spectrum of a graph needs to be considered. The main spectrum of a graph is the set of eigenvalues whose eigenspace is not orthogonal to the $\mathbb{1}$ vector. More formally, consider an eigenvalue $\lambda$ and its corresponding eigenspace, represented by a matrix $V$ whose columns are eigenvectors of $\lambda$ that span the eigenspace of $\lambda$. Then, the main angle $\beta_{\lambda}$ of $\lambda$’s eigenspace is $\frac{1}{\sqrt{n}}\|V^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}\|_{2}$, where $\|\mskip 2.0mu{\cdot}\mskip 2.0mu\|_{2}$ is the Euclidean norm. The main eigenvalues are now simply those eigenvalues with a non-zero main angle. Furthermore, two graphs are said to be co-main if they have the same set of main eigenvalues and corresponding main angles. Intuitively, the importance of the orthogonal projection on $\mathbb{1}$ stems from the observation that $\#\mathsf{walk}_{k}(A_{G})$ can be expressed as $\sum_{i}\lambda_{i}^{k}\beta_{\lambda_{i}}^{2}$ where the $\lambda_{i}$’s are the distinct eigenvalues of $A_{G}$.222Underlying this observation is that $A_{G}$ is a symmetric matrix and hence diagonalisable. Clearly, only those eigenvalues $\lambda_{i}$ for which $\beta_{\lambda_{i}}$ is non-zero matter when computing $\#\mathsf{walk}_{k}(A_{G})$. This results in the following characterisation.

Proposition 6.1** (Theorem 1.3.5 in Cvetković et al. [21]).**

Two graphs $G$ and $H$ of the same order are co-main if and only if they have the same number of walks of length $k$, for every $k\geq 0$.   ∎

Furthermore, the following proposition follows implicitly from the proof of Theorem 3 in van Dam et al. [72]. This proposition is also explicitly proved more recently in Theorem 1.2 in Dell et al. [27] in the context of distinguishing graphs by means of homomorphism vectors $\textsf{HOM}_{\cal F}(G)$ and $\textsf{HOM}_{\cal F}(H)$ where ${\cal F}$ consists of all paths.

Proposition 6.2**.**

Two graphs $G$ and $H$ of the same order have the same number of walks of length $k$, for every $k\geq 0$, if and only if there is a doubly quasi-stochastic matrix $Q$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muQ=Q\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$.   ∎

Example 6.2** (Continuation of Example 6.1).**

Consider the subgraph $G_{3}$ (​  ) of $G_{2}$ and the subgraph $H_{3}$ (​  ) of $H_{2}$. It is readily verified that there exists a doubly quasi-stochastic matrix $Q$ such that $A_{G_{3}}\mskip 2.0mu{\cdot}\mskip 2.0muQ=Q\mskip 2.0mu{\cdot}\mskip 2.0muA_{H_{3}}$. Indeed, $A_{G_{3}}\mskip 2.0mu{\cdot}\mskip 2.0muQ$ is equal to

[TABLE]

which is equal to $Q\mskip 2.0mu{\cdot}\mskip 2.0muA_{H_{3}}$. Hence by Proposition 6.2, $G_{3}$ and $H_{3}$ have the same number of walks on any length.   ∎

Just as for the fragment $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr})$ (Proposition 5.2), it turns out that sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$ can only extract information from adjacency matrices related to the number of walks in graphs. More precisely, we have the following proposition.

Proposition 6.3**.**

Let $G$ and $H$ be two graphs of the same order. Then, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})}H$ if and only if $G$ and $H$ have the same number of walks of any length.

Proof.

It is straightforward to show that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})}H$ implies that $G$ and $H$ must have the same number of walks of any length. This follows from the same argument as given in the proof of Proposition 5.2. For the converse, we use the characterisation given in Proposition 6.2. That is, if $G$ and $H$ have the same number of walks of any length, then there exists a doubly quasi-stochastic matrix $Q$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muQ=Q\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$. In other words, $A_{G}$ and $A_{H}$ are $Q$-conjugate. We now show that when $A_{G}$ and $A_{H}$ are $Q$-conjugate, for a doubly quasi-stochastic matrix $Q$, then $e(A_{G})=e(A_{H})$ for all sentences $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$. We here rely on a more general result (Lemma 6.1 below), which states that $T$-conjugation is preserved by the operation $\mathbb{1}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ provided that $T$ is quasi-stochastic. We again separate this lemma from the current proof because we need it also later in the paper. This suffices to conclude that expressions in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$ preserve $Q$-conjugation for a doubly quasi-stochastic matrix $Q$. Indeed, to deal with complex conjugate transposition, we note that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muQ=Q\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ implies that $A_{H}\mskip 2.0mu{\cdot}\mskip 2.0muQ^{*}=(Q\mskip 2.0mu{\cdot}\mskip 2.0muA_{H})^{*}=(A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muQ)^{*}=Q^{*}\mskip 2.0mu{\cdot}\mskip 2.0muA_{G}$ since $A_{G}$ and $A_{H}$ are symmetric real matrices. Hence, $A_{H}$ and $A_{G}$ are $Q^{*}$-conjugate. Furthermore, since $Q$ is a real matrix and quasi doubly-stochastic, also $Q^{*}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=\mathbb{1}$ holds. That is, $Q^{*}$ is a (doubly) quasi-stochastic matrix as well. Hence, Lemmas 5.1 and 6.1 imply that $Q$-conjugation and $Q^{*}$-conjugation are preserved by matrix multiplication and the one-vector operation. Combined with Lemma 5.4, we may conclude that $Q$-conjugation and $Q^{*}$-conjugation is also preserved by complex conjugate transposition. Hence, by induction on the structure of expressions, $e(A_{G})=e(A_{H})$ for any sentence $e(X)\in\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$. ∎

We now show that $T$-conjugation is preserved under the one-vector operation for any quasi-stochastic matrix $T$. In fact, since the result of $\mathbb{1}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ is only dependent on the dimensions of the input, we have do not even need the $T$-conjugation assumption on the inputs.

Lemma 6.1**.**

Let $\mathsf{ML}(\cal L)$ be any matrix query language fragment and consider two graphs $G$ and $H$ of the same order. Let $e_{1}(X)$ be an expression in $\mathsf{ML}(\cal L)$. Then, $\mathbb{1}(e_{1}(A_{G}))$ and $\mathbb{1}(e_{1}(A_{H}))$ are $T$-conjugate for any quasi-stochastic matrix $T$.

Proof.

The proof is straightforward. Let $e(X):=\mathbb{1}(e_{1}(X))$. We distinguish between the following cases, depending on the dimensions of $e_{1}(A_{G})$. If $e_{1}(A_{G})$ is an $n\times n$-matrix or $n\times 1$-vector, for $n>1$, then $e(A_{G})=e(A_{H})=\mathbb{1}$ and $e(A_{G})=\mathbb{1}=T\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=T\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{H})$. Furthermore, if $e_{1}(A_{G})$ is a $1\times n$-vector or sentence, then $e(A_{G})=e(A_{H})=1$ and thus these agree and are $T$-conjugate. ∎

We next turn our attention to $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1})$. We know from Propositions 5.1 and 5.2 that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1})}H$ implies that $G$ and $H$ are co-spectral. Combined with Proposition 6.1 and the fact that the sentence $\#\mathsf{walk}_{k}(X)$ counts the number of walks of length $k$, we have that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1})}H$ implies that $G$ and $H$ are co-spectral and co-main. The following is known about such graphs.

Proposition 6.4** (Corollary to Theorem 2 in Johnson and Newman [49]).**

Two co-spectral graphs $G$ and $H$ of the same order are co-main if and only if there exists a doubly quasi-stochastic orthogonal matrix $O$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$.   ∎

In other words, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1})}H$ implies the existence of a doubly quasi-stochastic orthogonal matrix $O$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$. We further observe that $A_{H}\mskip 2.0mu{\cdot}\mskip 2.0muO^{*}=O^{*}\mskip 2.0mu{\cdot}\mskip 2.0muA_{G}$ and that $O^{*}$ is a doubly quasi-stochastic orthogonal matrix as well. We can now use Lemmas 5.1, 5.2 and 6.1 to show the converse. Indeed, these lemmas combined tell us that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ implies that $e(A_{G})=e(A_{H})$ for any sentence $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1})$. As a consequence, we have the following proposition.

Proposition 6.5**.**

For two graphs $G$ and $H$ of the same order, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1})}H$ if and only if $G$ and $H$ have the same number of closed walks of any length, and the same number of walks of any length, if and only if $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ for a doubly quasi-stochastic orthogonal matrix $O$.   ∎

We can also phrase $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1})$-equivalence in terms of homomorphism vectors. That is, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1})}H$ if and only if $\textsf{HOM}_{\cal F}(G)=\textsf{HOM}_{\cal F}(H)$, where ${\cal F}$ now consists of all cycles and paths. This complements the results in Dell et al. [27].

As a note aside, an alternative characterisation to Proposition 6.4 (Theorem 3 in van Dam et al. [72]) is that $G$ and $H$ are co-spectral and co-main if and only if both $G$ and $H$ and their complement graphs $\bar{G}$ and $\bar{H}$ are co-spectral. Here, the complement graph $\bar{G}$ of $G$ is the graph with adjacency matrix given by $J-A_{G}-I$, and similarly for $\bar{H}$.

Example 6.3** (Continuation of Example 6.1).**

Consider the subgraph $G_{4}$ (​  ) of $G_{2}$ and the subgraph $H_{4}$ (​  ) of $H_{2}$. These are known to be the smallest non-isomorphic co-spectral graphs with co-spectral complements (see e.g., Figure 4 in [41]). From the previous remark it follows that $G_{4}$ and $H_{4}$ have the same number of walks of any length and the same number of closed walks of any length. These graphs are thus indistinguishable by sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$ and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1})$. Combined with our observation in Example 6.2 that also $G_{3}$ and $H_{3}$ have the same number of walks, we conclude that the disjoint unions $G_{2}=G_{3}\cup G_{4}$ (​  ) and $H_{2}=H_{3}\cup H_{4}$ (​ ) have the same number of walks of any length, as anticipated in Example 6.1.   ∎

Clearly, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1})}H$ implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,^{*},\mathbb{1})}H$. We already mentioned in Example 6.1 that the graphs $G_{2}$ (​ ) and $H_{2}$ (​ ) show that the converse does not hold.

We conclude again by observing that addition, scalar multiplication and pointwise function application on scalars can be added to $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,^{*},\mathbb{1})$ and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1})$ at no increase in expressiveness.

Corollary 6.4**.**

Let $G$ and $H$ be two graphs of the same order. Then,

1. (1)

$G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)}H$* if and only if $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,^{*},\mathbb{1})}H$; and* 2. (2)

$G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)}H$* if and only if $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1})}H$.*

Proof.

(1) We only need to show that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,^{*},\mathbb{1})}H$ implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},+,\times,\allowbreak\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],\allowbreak f\in\Omega)}H$. We have that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,^{*},\mathbb{1})}H$ implies $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muQ=Q\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ for a doubly quasi-stochastic matrix $Q$ (Proposition 6.3). Furthermore, in the proof of Proposition 6.3 we have shown that $A_{H}\mskip 2.0mu{\cdot}\mskip 2.0muQ^{*}=Q^{*}\mskip 2.0mu{\cdot}\mskip 2.0muA_{G}$ where $Q^{*}$ is again a doubly quasi-stochastic matrix. Lemmas 5.1, 5.3, 5.4, 5.5 and 6.1 imply that $Q$-conjugation and $Q^{*}$-conjugation are preserved by all operations in the fragment $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)$.

(2) We only need to show that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1})}H$ implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},\,{}^{*},\mathbb{1},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)}H$. We have that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1})}H$ implies $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ for a doubly quasi-stochastic orthogonal matrix $O$ (Proposition 6.5). We observed earlier that $A_{H}\mskip 2.0mu{\cdot}\mskip 2.0muO^{*}=O^{*}\mskip 2.0mu{\cdot}\mskip 2.0muA_{G}$ and that $O^{*}$ is a doubly quasi-stochastic orthogonal matrix as well. Lemmas 5.1, 5.2, 5.3, 5.4, 5.5 and 6.1, imply that $O$-conjugation and $O^{*}$-conjugation are preserved by all operations in the fragment $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,{}^{*},\mathsf{tr},\mathbb{1},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)$.

In both cases, we can therefore conclude, by induction on the structure of expressions, that for any sentence $e(X)$, $e(A_{G})$ and $e(A_{H})$ are conjugate and hence, $e(A_{G})=e(A_{H})$. ∎

As we will see later, including any other operation from Table 1, such as $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ or pointwise function applications on vectors or matrices, allows us to distinguish $G_{4}$ and $H_{4}$. We recall from Example 6.3 that these graphs cannot be distinguished by sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$ and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1})$.

7 The impact of the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ operation

We next consider the operation $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ which takes as a column vector as input and returns the diagonal matrix with the input vector on its diagonal. 333The $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ operation is also defined for $1\times 1$-matrices (scalars) in which case it just returns that scalar. From the fragments considered so far, $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$ and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1})$ are the only fragments in which vectors can be defined and for which the inclusion of $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ has an impact. Therefore, in this section we consider equivalence with regards to $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$ and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$. This section is organised as follows. First, we illustrate what information can be obtained from graphs using the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ operation (Section 7.1). We then show in Section 7.2 that one can compute so-called equitable partitions of graphs. From this, we can infer that equivalence of graphs with regards to the fragments under consideration implies that the graphs have a common equitable partition. We use this observation in Sections 7.3 and 7.4 to characterise $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1})$- and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1})$-equivalence, respectively. Finally, in Section 7.5 we show that we can add pointwise function applications on vectors without increasing the distinguishing power of the fragments.

7.1 Example of the impact of the presence of $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$

Using $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ we can extract new information from graphs, as is illustrated in the following example.

Example 7.1**.**

Consider graphs $G_{4}$ (​  ) and $H_{4}$ (​​ ​). In $G_{4}$ we have vertices of degrees [math] and $2$, and in $H_{4}$ we have vertices of degrees $1$, $2$ and $3$. We will count the number of vertices of degree $3$. Given that we know that $3$ is an upper bound on the degrees of vertices in $G_{4}$ and $H_{4}$, we consider the sentence $\#\mathsf{3degr}(X)$ given by

[TABLE]

in which we, for convenience, allow addition and scalar multiplications. Each of the subexpressions $\operatorname{\mathsf{diag}}(X\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}(X)-d\times\mathbb{1}(X))$, for $d=0,1$ and $2$, sets the diagonal entry corresponding to vertex $v$ to [math] when $v$ has degree $d$. By taking the product of these diagonal matrices, entries that are set to [math] will remain zero in the resulting diagonal matrix. This implies that, after taking these products, the only non-zero diagonal entries are those corresponding to vertices of degree different from [math], $1$ or $2$. In other words, only for vertices of degree $3$ the diagonal entries carry a non-zero value, i.e., the value $6=(3-0)(3-1)(3-2)$. By appropriately rescaling by the factor $\frac{1}{6}$, the diagonal entries for the degree three vertices are set to $1$, and then summed up. Hence, $\#\mathsf{3degr}(X)$ indeed counts the number vertices of degree three when evaluated on adjacency matrices of graphs with vertices of maximal degree $3$. Since $\#\mathsf{3degr}(A_{G_{4}})=[0]\neq[1]=\#\mathsf{3degr}(A_{H_{4}})$ we can distinguish $G_{4}$ and $H_{4}$. We can obtain similar expressions for $\#d\!\mathsf{degr}(X)$ for arbitrary $d$, provided that we know the maximal degree of vertices in the graph. The way that these expressions are constructed is similar to the so-called Schur-Wielandt Principle indicating how to extract entries from a matrix that hold specific values by means of pointwise multiplication of matrices (see e.g., Proposition 1.4 in [61]). Here, we do not have pointwise matrix multiplication available but since we extract information from vectors, pointwise multiplication of vectors is simulated by normal matrix multiplication of diagonal matrices with the vectors on their diagonals.   ∎

The use of the diagonal matrices and their products as in our example sentence $\#\mathsf{3degr}(X)$ can also be generalised to obtain information about so-called iterated degrees of vertices in graphs, e.g., to identify and/or count vertices that have a number of neighbours each of which have neighbours of specific degrees, and so on. Such iterated degree information is closely related to equitable partitions and fractional isomorphisms of graphs (see e.g., Chapter 6 in [66]). We phrase our results in terms of equitable partitions instead of iterated degree sequences.

7.2 Equitable partitions

Formally, an equitable partition ${\cal V}=\{V_{1},\ldots,V_{\ell}\}$ of $G$ is a partition of the vertex set $V$ of $G$ such that for all $i,j=1,\ldots,\ell$ and $v,v^{\prime}\in V_{i}$, $\mathsf{deg}(v,V_{j})=\mathsf{deg}(v^{\prime},V_{j})$. Here, $\mathsf{deg}(v,V_{j})$ is the number of vertices in $V_{j}$ that are adjacent to $v$. In other words, an equitable partition is such that the graph is regular within each part, i.e., all vertices in a part have the same degree, and is bi-regular between any two different parts, i.e., the number of edges between any two vertices in two different parts is constant. A graph always has a trivial equitable partition: simply treat each vertex as a part by its own. More interesting is the coarsest equitable partition of a graph, i.e., the unique equitable partition for which any other equitable partition of the graph is a refinement thereof [66].

The conditions underlying equitable partitions can be equivalently stated in terms of adjacency matrices and indicator vectors describing the partitions. We first introduce the notion of indicator vector. Let $G=(V,E)$ be a graph of order $n$ with $V=\{1,\ldots,n\}$. Let $V^{\prime}\subseteq V$. We denote the indicator vector of $V^{\prime}$ as the column vector $\mathbb{1}_{V^{\prime}}$ in $\mathbb{R}^{n\times 1}$ and defined such that $(\mathbb{1}_{V^{\prime}})_{v}=1$ if $v\in V^{\prime}$ and $(\mathbb{1}_{V^{\prime}})_{v}=0$ otherwise. Then, given a partition ${\cal V}=\{V_{1},\ldots,V_{\ell}\}$ of $V$ we represent ${\cal V}$ by the $\ell$ indicator vectors $\mathbb{1}_{V_{i}},\ldots,\mathbb{1}_{V_{\ell}}$. We observe that $\mathbb{1}=\sum_{i=1}^{\ell}\mathbb{1}_{V_{i}}$ due to ${\cal V}$ being a partition. We can now express that ${\cal V}$ is an equitable partition in linear algebra terms. More precisely, ${\cal V}$ is an equitable partition of $G$ if and only if for all $i,j=1,\ldots,\ell$,

[TABLE]

for some (arbitrary) vertex $v\in V_{i}$.

We next use the standard terminology for relating two graphs in terms of their equitable partitions [66]. More specifically, two graphs $G$ and $H$ are said to have a common equitable partition if there exists an equitable partition ${\cal V}=\{V_{1},\ldots,V_{\ell}\}$ of $G$ and an equitable partition ${\cal W}=\{W_{1},\ldots,W_{\ell}\}$ of $H$ such that (a) the sizes of the parts agree, i.e., $|V_{i}|=|W_{i}|$ for each $i=1,\ldots,\ell$, and (b) $\mathsf{deg}(v,V_{j})=\mathsf{deg}(w,W_{j})$ for any $v\in V_{i}$ and $w\in W_{i}$ and any $i,j=1,\ldots,\ell$. When ${\cal V}$ and ${\cal W}$ satisfy these conditions, we say that these partitions witness that $G$ and $H$ have a common equitable partition. Similarly, two graphs are said to have a common coarsest equitable partition if the partitions ${\cal V}$ of $G$ and ${\cal W}$ of $H$ mentioned above are the coarsest equitable partitions of $G$ and $H$, respectively. Proposition 7.1 below characterises when two graphs do have a common equitable partition. Furthermore, when two graphs have a common equitable partition they also have a common coarsest equitable partition (see e.g., Theorem 6.5.1 in [66]).

Equitable partitions naturally arise as the result of the colour refinement procedure [8, 38, 73], also known as the $1$-dimensional Weisfeiler-Lehman (1WL) algorithm, used as a subroutine in graph isomorphism solvers. Furthermore, there is a close connection to the study of fractional isomorphisms of graphs [66, 69], as already mentioned in the introduction. We recall: two graphs $G$ and $H$ are said to be fractional isomorphic if there exists a doubly stochastic matrix $S$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muS=S\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$. Furthermore, a logical characterisation of graphs with a common equitable partition exists, as is stated next.

Proposition 7.1** (Theorem 1 in Tinhofer [69], Section 4.8 in Immerman and Lander [47]).**

Let $G$ and $H$ be two graphs of the same order. Then, $G$ and $H$ are fractional isomorphic if and only if $G$ and $H$ have a common equitable partition if and only if $G\equiv_{\mathsf{C}^{2}}H$.   ∎

Example 7.2**.**

The matrix linking the adjacency matrices of $G_{3}$ (​  ) and $H_{3}$ (​  ) in Example 6.2 is in fact a doubly stochastic matrix (all its entries are either [math] or $\frac{1}{2}$). Hence, $G_{3}$ and $H_{3}$ have a common equitable partition by Proposition 7.1. One can alternatively verify that the partitions of $G_{3}$ and $H_{3}$ consisting of a single part containing all the vertices of $G_{3}$ and $H_{3}$, respectively, witness that $G_{3}$ and $H_{3}$ have a common equitable partition. By contrast, graphs $G_{2}$ (​  ) and $H_{2}$ (​ ) do not have a common equitable partition. Indeed, fractional isomorphic graphs must have the same multi-set of degrees, i.e., the same multi-set consisting of the degrees of vertices (Proposition 6.2.6 in [66]), which does not hold for $G_{2}$ and $H_{2}$. Indeed, we note that there is an isolated vertex in $G_{2}$ but not in $H_{2}$. For the same reason, $G_{1}$ (​  ) and $H_{1}$ (​  ), and $G_{4}$ (​  ) and $H_{4}$ (​ ) are not fractional isomorphic.   ∎

To relate equitable partitions to $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$- and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-equivalence, we show that the presence of $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ allows us to formulate a number of expressions, denoted by $\mathsf{eqpart}_{i}(X)$, for $i=1,\ldots,\ell$, that together extract the coarsest equitable partition from a given graph. By evaluating these expressions on $A_{G}$ and $A_{H}$, one can use additional sentences to detect whether these partitions witness that $G$ and $H$ have a common equitable partition. In this subsection, ${\cal L}$ can be either $\{\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}}\}$ or $\{\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}}\}$.

Proposition 7.2**.**

Let $G$ and $H$ be two graphs of the same order. Then, $G\equiv_{\mathsf{ML}({\cal L})}H$ implies that $G$ and $H$ have a common equitable partition.

Proof.

The proof is quite lengthy so we first describe its structure.

• (a)

We first argue that we can use addition and scalar multiplication at no increase in distinguishing power. This will simplify the construction of the expressions later on. We denote by ${\cal L}^{+}$ the extension of ${\cal L}$ with $+$ and $\times$.

• (b)

We then show how to construct a number of expressions, denoted by $\mathsf{eqpart}_{i}(X)$, for $i=1,\ldots,\ell$, in $\mathsf{ML}({\cal L}^{+})$. The key property of these expressions is that when they are evaluated on the adjacency matrix $A_{G}$ of $G$, $\mathsf{eqpart}_{i}(A_{G})$, for $i=1,\ldots,\ell$, correspond to indicator vectors representing an equitable partition of $G$.

• (c)

The construction of the expressions $\mathsf{eqpart}_{i}(X)$, for $i=1,\ldots,\ell$, depend on $A_{G}$. As such, it is not guaranteed that $\mathsf{eqpart}_{i}(A_{H})$, for $i=1,\ldots,\ell$, correspond to indicator vectors representing an equitable partition of $H$. We show, however, that when $G\equiv_{\mathsf{ML}({{\cal L}^{+}})}H$ holds, then $\mathsf{eqpart}_{i}(A_{H})$, for $i=1,\ldots,\ell$, indeed correspond to indicator vectors representing an equitable partition of $H$. To show this, we construct a number of sentences in $\mathsf{ML}({\cal L}^{+})$.

• (d)

Finally, we observe that the partitions represented by $\mathsf{eqpart}_{i}(A_{G})$ and $\mathsf{eqpart}_{i}(A_{H})$, for $i=1,\ldots,\ell$, witness that $G$ and $H$ have a common equitable partition.

Hence, all combined, this suffices to conclude that $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$ implies that $G$ and $H$ have a common equitable partition. Given (a), the same conclusion holds for $G\equiv_{\mathsf{ML}({\cal L})}H$.

(a) Showing that $G\equiv_{\mathsf{ML}({\cal L})}H$ if and only if $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$. As mentioned earlier, it will be convenient to use addition and scalar multiplication in our expressions. Clearly, $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$ implies $G\equiv_{\mathsf{ML}(\cal L)}H$. Hence, it suffices to show that $G\equiv_{\mathsf{ML}({\cal L})}H$ implies $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$.444We remark that we cannot rely yet on the conjugation-preservation Lemma 5.3 to show that $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$ if and only if $G\equiv_{\mathsf{ML}(\cal L)}H$. Indeed, at this point we do not know yet for what kind of matrices $T$, $T$-conjugation is preserved by the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$-operation. This will only be settled in Lemma 7.1 later in this section. To see this, we observe that any expression in $\mathsf{ML}({\cal L}^{+})$ can be written as a linear combination of expressions in $\mathsf{ML}(\cal L)$. For completeness, we verify this in the appendix. Consider now a sentence $e(X)$ in $\mathsf{ML}({\cal L}^{+})$. Hence, $e(X)$ can be written as $\sum_{i=1}^{p}a_{i}\times e_{i}(X)$ with $a_{i}\in\mathbb{C}$ and sentences $e_{i}(X)\in\mathsf{ML}(\cal L)$, for $i=1,\ldots,p$. By assumption, $e_{i}(A_{G})=e_{i}(A_{H})$, for $i=1,\ldots,p$, Hence also $e(A_{G})=\sum_{i=1}^{p}a_{i}\times e_{i}(A_{G})=\sum_{i=1}^{p}a_{i}\times e_{i}(A_{H})=e(A_{H})$. As a consequence, $G\equiv_{\mathsf{ML}({\cal L})}H$ indeed implies $G\equiv_{\mathsf{ML}({{\cal L}^{+}})}H$.

(b) Computing the coarsest equitable partition of $G$ by expression in $\mathsf{ML}({\cal L}^{+})$. We next show that we can compute the indicator vectors of an equitable partition of $G$ by means of expressions in $\mathsf{ML}({\cal L}^{+})$. To see this, we implement the algorithm GDCR for finding this partition [51]. We recall this algorithm (in a slightly different form than presented in Kersting et al. [51]) in Algorithm 1. In a nutshell, the algorithm takes as input $A_{G}$, the adjacency matrix of $G$, and returns a matrix whose columns hold indicator vectors that represent an equitable partition of $G$.

The algorithm starts, on line 1, by creating a partition consisting of a single part containing all vertices, represented by the indicator vector $\mathbb{1}$, and stored in vector $B^{(0)}$. Then, in the $i^{\text{th}}$ step, the current partition is represented by $\ell_{i-1}$ indicator vectors $\mathbb{1}_{V_{1}^{(i-1)}},\ldots,\mathbb{1}_{V_{\ell_{i-1}}^{(i-1)}}$ which constitute the columns of matrix $B^{(i-1)}$. The refinement of this partition is then computed in two steps. First, the matrix $M^{(i)}:=A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muB^{(i-1)}$ (line 4) is computed; second, each $\mathbb{1}_{V_{j}^{(i-1)}}$ is refined by putting vertices $v$ and $w$ in the same part if and only if they have the same rows in $M^{(i)}$, i.e., when $M_{v*}^{(i)}=M_{w*}^{(i)}$ holds (line 5). The corresponding partition ${\cal V}^{(i)}$ is then represented by, say $\ell_{i}$, indicator vectors and stored as the columns of $B^{(i)}$ (line 6). This is repeated until no further refinement of the partition is obtained. At most $n$ iterations are needed. The correctness of the algorithm is established in [51]. That is, the resulting indicator vectors represent an equitable partition of $G$. In fact, they represent the coarsest equitable partition of $G$.

We next detail how a run of the algorithm on adjacency matrix $A_{G}$ can be simulated using expressions in $\mathsf{ML}({\cal L}^{+})$. The initialisation step is easy: We compute $B^{(0)}$ by means of the expression $b^{(0)}(X):=\mathbb{1}(X)$. Clearly, $B^{(0)}=b^{(0)}(A_{G})$. Next, suppose by induction that we have $\ell_{i-1}$ expressions $b^{(i-1)}_{1}(X),\ldots,b^{(i-1)}_{\ell_{i-1}}(X)$ such that when these expressions are evaluated on $A_{G}$, they return the indicator vectors stored in the columns of $B^{(i-1)}$. That is, $\mathbb{1}_{V_{j}^{(i-1)}}=b^{(i-1)}_{j}(A_{G})$ for all $j=1,\ldots,\ell_{i-1}$. We next show how the $i^{\text{th}}$ iteration is simulated.

We first compute the $\ell_{i-1}$ vectors stored in the columns of $M^{(i)}$ (line 4). We compute these column vectors one at a time. To this aim, we consider expressions

[TABLE]

Clearly, $m^{(i)}_{j}(A_{G})=M^{(i)}_{*j}$, as desired.

A bit more challenging is the computation of the refined partition in ${\cal V}^{(i)}$ (line 5) since we need to inspect all columns $M^{(i)}_{*j}$ and identify rows on which all these columns agree, as explained above. It is here that the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ operation plays a crucial role. Moreover, to compute this refined partition we need to know all values occurring in $M^{(i)}$. The expressions below depend on these values and hence on the input adjacency matrix $A_{G}$.

Let $D^{(i)}_{j}$ be the set of values occurring in the column vector $M^{(i)}_{*j}$, for $j=1,\ldots,\ell_{i-1}$. We compute, by means of an $\mathsf{ML}({\cal L}^{+})$ expression, an indicator vector which identifies the rows in $M^{(i)}_{*j}$ that hold a specific value $c\in D^{(i)}_{j}$. This expression is similar to the one used in Example 7.1 to extract vertices of degree 3 from the degree vector. More precisely, we consider expressions

[TABLE]

for the current iteration $i$, column $j$ in $M^{(i)}$, and value $c\in D^{(i)}_{j}$. The correctness of these expressions follows from a similar explanation as given in Example 7.1. Given these expressions, one can now easily obtain an indicator vector identifying all rows in $M^{(i)}$ that hold a specific value combination $(c_{1},\ldots,c_{\ell_{i-1}})$ in their columns, where each $c_{j}\in D^{(i)}_{j}$, as follows:

[TABLE]

That is, we simply take the boolean conjunction of all indicator vectors $\mathbb{1}_{=c_{j}}^{(i),j}(X)$, for $j=1,\ldots,\ell_{i-1}$. We note that $\mathbb{1}_{=(c_{1},\ldots,c_{\ell_{i-1}})}^{(i)}(A_{G})$ may return the zero vector, i.e., when $(c_{1},\ldots,c_{\ell_{i-1}})$ does not occur as a row in $M^{(i)}$. We are only interested in value combinations that do occur. Suppose that there are $\ell_{i}$ distinct value combinations $(c_{1},\ldots,c_{\ell_{i-1}})$ for which $\mathbb{1}_{=(c_{1},\ldots,c_{\ell_{i-1}})}^{(i)}(A_{G})$ returns a non-zero indicator vector. We denote by $b^{(i)}_{1}(X),\ldots,\allowbreak b^{(i)}_{\ell_{i}}(X)$ the corresponding expressions of the form $\mathbb{1}_{=(c_{1},\ldots,c_{\ell_{i-1}})}^{(i)}(X)$. It should be clear that $b^{(i)}_{1}(A_{G}),\ldots,\allowbreak b^{(i)}_{\ell_{i}}(A_{G})$ are indicator vectors corresponding to the refined partition ${\cal V}^{(i)}$ as stored in $B^{(i)}$. This concludes the simulation of the $i^{\text{th}}$ iteration of the algorithm.

Finally, after the $n^{\text{th}}$ iteration we define

[TABLE]

for $i=1,\ldots,\ell_{n}$. In the following, we denote $\ell_{n}$ by $\ell$. We remark once more that all expressions defined above depend on the input $A_{G}$, as their definitions rely on the values occurring in the matrices $M^{(i)}$ computed along the way.

(c) Checking that $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$ implies that $\mathsf{eqpart}_{i}(A_{H})$, for $i=1,\ldots,\ell$, also represent an equitable partition of $H$. Recall that we want to show that if $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$ holds, then $G$ and $H$ have a common equitable partition. As a first step we verify that $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$ implies that the vectors $\mathsf{eqpart}_{i}(A_{H})$, for $i=1,\ldots,\ell$, correspond to an equitable partition of $H$. The challenge is to check all this by means of sentences in $\mathsf{ML}({\cal L}^{+})$. We will use the following sentences.

For each $i=1,\ldots,\ell$, we first check whether $\mathsf{eqpart}_{i}(A_{H})$ is also a binary vector. We note that, by construction of the expression $\mathsf{eqpart}_{i}(X)$, $\mathsf{eqpart}_{i}(A_{H})$ returns a real vector. To check whether every entry in $\mathsf{eqpart}_{i}(A_{H})$ is either [math] or $1$, we show that all of its entries must satisfy the equation $x(x-1)=0$. To this aim, we consider the $\mathsf{ML}({\cal L}^{+})$ sentence

[TABLE]

We claim that if $X$ is assigned a diagonal real matrix, say $\Delta$, then $\mathsf{binary\_diag}(\Delta)=[0]$ if and only if $\Delta$ is a binary diagonal matrix.

Indeed, if $\Delta$ is a binary diagonal matrix, then $\Delta\mskip 2.0mu{\cdot}\mskip 2.0mu\Delta=\Delta$, $\Delta\mskip 2.0mu{\cdot}\mskip 2.0mu\Delta-\Delta=Z$, where $Z$ is the zero matrix, and hence $\mathsf{binary\_diag}(\Delta)=\mathbb{1}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0muZ\mskip 2.0mu{\cdot}\mskip 2.0muZ\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=[0]$. Conversely, suppose that $\mathsf{binary\_diag}(\Delta)=[0]$. We observe that $(\Delta\mskip 2.0mu{\cdot}\mskip 2.0mu\Delta-\Delta)\mskip 2.0mu{\cdot}\mskip 2.0mu(\Delta\mskip 2.0mu{\cdot}\mskip 2.0mu\Delta-\Delta)$ is a diagonal matrix with squared real numbers on its diagonal. Hence, $\mathsf{binary\_diag}(\Delta)=[0]$ implies that the sum of the (squared real) diagonal elements in $\Delta\mskip 2.0mu{\cdot}\mskip 2.0mu\Delta-\Delta$ is [math]. This in turn implies that every element on the diagonal in $\Delta\mskip 2.0mu{\cdot}\mskip 2.0mu\Delta-\Delta$ must be zero. Hence, every element on $\Delta$’s diagonal must satisfy the equation $x^{2}-x=0$, implying that either $x=0$ or $x=1$. As a consequence, $\Delta$ is a binary diagonal matrix.

We now observe that $\mathsf{binary\_diag}(\operatorname{\mathsf{diag}}(\mathsf{eqpart}_{i}(A_{G})))=[0]$ since $\mathsf{eqpart}_{i}(A_{G})$ returns an indicator vector. Then, $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$ implies that the equality

[TABLE]

must hold, for all $i=1,\ldots,\ell$. Hence, the matrices $\operatorname{\mathsf{diag}}(\mathsf{eqpart}_{i}(A_{H}))$ are indeed binary and so are its diagonal elements described by $\mathsf{eqpart}_{i}(A_{H})$, as desired. 2. 2.

We next verify that all indicator vectors $\mathsf{eqpart}_{i}(A_{H})$ combined represent a partition of the vertex set of $H$. To verify this partition condition, we simply consider the sentence \mathbb{1}(X)^{*}\mskip 2.0mu{\cdot}\mskip 2.0mu\bigl{(}\sum_{i=1}^{\ell}\mathsf{equit}_{i}(X)\bigr{)}. Since $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$ implies

[TABLE]

it must be the case that the indicator vectors $\mathsf{eqpart}_{i}(A_{H})$, for $i=1,\ldots,\ell$, form a partition. 3. 3.

We finally verify that the partition of $H$ represented by the indicator vectors $\mathsf{equit}_{i}(A_{H})$, for $i=1,\ldots,\ell$, is an equitable partition of $H$. Let ${\cal V}=\{V_{1},\ldots,V_{\ell}\}$ be the equitable partition of $G$, represented by the indicator vectors $\mathsf{eqpart}_{i}(A_{G})$, for $i=1,\ldots,\ell$. Similarly, let ${\cal W}=\{W_{1},\ldots,W_{\ell}\}$ be the partition of $H$, represented by the indicator vectors $\mathsf{eqpart}_{i}(A_{H})$, for $i=1,\ldots,\ell$. We show that ${\cal W}$ is an equitable partition of $H$. As already mentioned at the beginning of this section, we can rephrase “being equitable” in linear algebra terms. In particular, we know that for any $i,j=1,\ldots,\ell$,

[TABLE]

returns the zero vector, where $v$ is an arbitrary vertex in $V_{i}$, the part corresponding to the indicator vector $\mathsf{eqpart}_{i}(A_{G})$. We want to check whether the same condition holds for $A_{H}$. We therefore consider the expressions $\mathsf{equi\_test}_{ij}(X)$, for $i,j=1,\ldots,\ell$ given by

[TABLE]

and check whether, when evaluated on $A_{H}$, the obtained diagonal matrix is the zero matrix. This would imply that for all $i,j=1,\ldots,\ell$,

[TABLE]

also returns the zero vector. As a consequence, ${\cal W}$ is an equitable partition of $H$. It rests us only to show that we can check, by means of sentences, whether a diagonal matrix is the zero matrix. We use the sentence

[TABLE]

for this purpose. A similar argument as for the expression $\mathsf{binary\_diag}(X)$ shows that the $\mathsf{zerotest\_diag}(X)$ expression returns $[0]$ on diagonal real matrices if and only if the diagonal matrix is the zero matrix. We here again use that a sum of squares equals zero if and only if each summand is zero. Since $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$, we have that

[TABLE]

for all $i,j=1,\ldots,\ell$, as desired.

(d) Verifying that $G$ and $H$ have a common equitable partition. To conclude the proof we need to show that the equitable partitions ${\cal V}$ of $G$ and ${\cal W}$ of $H$, as defined above, witness that $G$ and $H$ have a common equitable partition. In other words, we must have that (i) $|V_{i}|=|W_{i}|$ for every $i=1,\ldots,\ell$; and (ii) for any $i,j=1,\ldots,\ell$, $\mathsf{deg}(v,V_{j})=\mathsf{deg}(w,W_{j})$ for any $v\in V_{i}$ and any $w\in W_{i}$. We observe that the expressions $\mathsf{equi\_test}_{ij}(X)$ used above already verify condition (ii). We next show that, for each $i=1,\ldots,\ell$, the indicator vector $\mathsf{equit}_{i}(A_{G})$ contains the same number of $1$’s as $\mathsf{eqpart}_{i}(A_{G})$. To show this, it suffices to consider the sentences $(\mathbb{1}(X))^{*}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathsf{eqpart}_{i}(X)$, for $i=1,\ldots,\ell$. Clearly, $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$ implies that $\mathbb{1}^{*}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathsf{eqpart}_{i}(A_{G})=\mathbb{1}^{*}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathsf{eqpart}_{i}(A_{H})$, for $i=1,\ldots,\ell$. Hence, $\mathsf{eqpart}_{i}(A_{H})$ and $\mathsf{eqpart}_{i}(A_{G})$ contain the same number of ones. So, we may conclude that $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$ implies indeed that $G$ and $H$ have a common equitable partition. Given that $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$ if and only if $G\equiv_{\mathsf{ML}({\cal L})}H$, also $G\equiv_{\mathsf{ML}({\cal L})}H$ implies that $G$ and $H$ have a common equitable partition, as desired. ∎

7.3 Characterisation of $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-equivalence

We next consider $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-equivalence. We have just shown that $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-equivalent graphs have a common equitable partition. The converse also holds, as will be shown below (Proposition 7.3).

We first introduce a notion of compatibility that will be used to define the proper notion of conjugation for $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-equivalence. Let $G$ and $H$ be two graphs of order $n$. Let ${\cal V}=\{V_{1},\ldots,V_{\ell}\}$ and ${\cal W}=\{W_{1},\ldots,W_{\ell}\}$ be partitions of $G$ and $H$, respectively. Furthermore, let $\mathbb{1}_{V_{i}}$ and $\mathbb{1}_{W_{i}}$, for $i=1,\ldots,\ell$, denote the indicator vectors corresponding to $V_{i}\in{\cal V}$ and $W_{i}\in{\cal W}$, respectively. Consider a matrix $T$ in $\mathbb{C}^{n\times n}$. Then, $T$ is said to be compatible with the partitions ${\cal V}$ of $G$ and ${\cal W}$ of $H$, if for $i=1,\ldots,\ell$,

[TABLE]

Intuitively, this condition implies that $T$ has a block structure determined by the partitions ${\cal V}$ and ${\cal W}$ and only has non-zero blocks for blocks corresponding to the same parts $V_{i}$ and $W_{i}$ in these partitions.

Proposition 7.3**.**

Let $G$ and $H$ be two graphs of the same order. If $G$ and $H$ have a common equitable partition, then $e(A_{G})=e(A_{H})$ for every sentence $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$.

Proof.

By assumption, $G$ and $H$ have a common equitable partition. Let ${\cal V}=\{V_{1},\ldots,\allowbreak V_{\ell}\}$ and ${\cal W}=\{W_{1},\ldots,W_{\ell}\}$ be partitions of $G$ and $H$, respectively, that witness this fact. As before, we denote by $\mathbb{1}_{V_{i}}$ and $\mathbb{1}_{W_{i}}$, for $i=1,\ldots,\ell$, the corresponding indicator vectors. We know from Proposition 7.1 that there exists a doubly stochastic matrix $S$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muS=S\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$. As previously observed, also $A_{H}\mskip 2.0mu{\cdot}\mskip 2.0muS^{*}=S^{*}\mskip 2.0mu{\cdot}\mskip 2.0muA_{G}$ holds, where $S^{*}$ is again doubly stochastic. Then, Lemmas 5.1, 5.4 and 6.1 imply that $S$-conjugation and $S^{*}$-conjugation are preserved by matrix multiplication, complex conjugate transposition, and the one-vector operation. To conclude that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ holds, we verify that the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ operation also preserves $S$- and $S^{*}$-conjugation. We rely on a more general result (Lemma 7.1 below), which states that $T$-conjugation, for a matrix $T$, is preserved by the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ operation provided that $T$ is doubly quasi-stochastic and compatible with equitable partitions of $G$ and $H$ that witness that $G$ and $H$ have a common equitable partition. We again separate this lemma from the current proof because we need it later in the paper. When considering the doubly stochastic matrix $S$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muS=S\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ holds, the matrix $S$ can be assumed to be compatible with ${\cal V}$ and $\cal W$. To see this, we recall from the proof of Theorem 6.5.1 in [66] that we can take $S$ to be such that for $i\neq j$, $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0muS\mskip 2.0mu{\cdot}\mskip 2.0mu\operatorname{\mathsf{diag}}(\mathbb{1}_{W_{j}})$ is the $|V_{i}|\times|W_{j}|$ zero matrix, and for $i=j$, $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0muS\mskip 2.0mu{\cdot}\mskip 2.0mu\operatorname{\mathsf{diag}}(\mathbb{1}_{W_{i}})$ is the square $|V_{i}|\times|W_{i}|$ matrix in which all entries are equal to $\frac{1}{|V_{i}|}$. Hence, $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0muS=S\mskip 2.0mu{\cdot}\mskip 2.0mu\operatorname{\mathsf{diag}}(\mathbb{1}_{W_{i}})$ for all $i,j=1,\ldots,\ell$.

As a consequence, if $e_{1}(A_{G})$ and $e_{1}(A_{H})$ are $S$-conjugate, then Lemma 7.1 implies that $\operatorname{\mathsf{diag}}(e_{1}(A_{G}))$ and $\operatorname{\mathsf{diag}}(e_{1}(A_{H}))$ are $S$-conjugate. We also note that $\operatorname{\mathsf{diag}}(\mathbb{1}_{W_{i}})\mskip 2.0mu{\cdot}\mskip 2.0muS^{*}=S^{*}\mskip 2.0mu{\cdot}\mskip 2.0mu\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})$. So, $S^{*}$ is compatible with ${\cal W}$ and ${\cal V}$. An inductive argument then shows that $e(A_{G})$ and $e(A_{H})$ are $S$-conjugate (and thus equal) for any sentence $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$, as desired. ∎

Let ${\cal V}$ and ${\cal W}$ be equitable partitions of $G$ and $H$, respectively, that witness that $G$ and $H$ have a common equitable partition. We next show that $T$-conjugation, by means of doubly quasi-stochastic matrices $T$ that are compatible with ${\cal V}$ and ${\cal W}$, is indeed preserved by the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ operation. Showing this requires a bit more work than our previous conjugation-preservation results.

More precisely, a key property underlying this preservation property is the following: Any vector that can be obtained by evaluating expressions in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$ on an adjacency matrix of a graph, can be written in a canonical way in terms of the indicator vectors representing an equitable partitions of that graph. We state this requirement for general matrix query languages, as follows.

Let $\mathsf{ML}(\cal L)$ be a matrix query language. We say that $\mathsf{ML}(\cal L)$-vectors are constant on equitable partitions if, for any expression $e(X)\in\mathsf{ML}(\cal L)$, any graph $G$, and any equitable partition ${\cal V}=\{V_{1},\ldots,V_{\ell}\}$ of $G$, when $e(A_{G})$ is an $n\times 1$-vector then there exists scalars $a_{i}\in\mathbb{C}$, for $i=1,\ldots,\ell$, such that

[TABLE]

holds. Here, $\mathbb{1}_{V_{1}},\ldots,\mathbb{1}_{V_{\ell}}$ represent the equitable partition ${\cal V}$ of $G$.

Intuitively, this condition is important for the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ operation since it takes a vector as input and the linear combination (1) allows one to only reason about (linear combinations of) diagonal matrices obtained by the indicator vectors of the equitable partitions. Compatibility in turn implies conjugation preservation for such (indicator vector-based) diagonal matrices, which can then be lifted, due to linearity, to conjugation of arbitrary diagonal matrices. We make this argument more formal in the proof of the following Lemma.

Lemma 7.1**.**

Let $\mathsf{ML}(\cal L)$ be a matrix query language fragment such that $\mathsf{ML}(\cal L)$-vectors are constant on equitable partitions. Let $G$ and $H$ be two graphs of the same order which have a common equitable partition. Furthermore, let ${\cal V}$ and ${\cal W}$ be equitable partitions of $G$ and $H$, respectively, that witness that $G$ and $H$ have a common equitable partition, and let $T$ be a doubly quasi-stochastic matrix which is compatible with ${\cal V}$ and ${\cal W}$. Finally, let $e(X)$ be an expression in $\mathsf{ML}(\cal L)$. Then, if $e(A_{G})$ and $e(A_{H})$ are $T$-conjugate, then also $\operatorname{\mathsf{diag}}(e(A_{G}))$ and $\operatorname{\mathsf{diag}}(e(A_{H}))$ are $T$-conjugate.

Proof.

Let $e(X)$ be an expression in $\mathsf{ML}(\cal L)$. Consider now $e^{\prime}(X):=\operatorname{\mathsf{diag}}(e(X))$. We distinguish between two cases, depending on the dimensions of $e(A_{G})$. First, if $e(A_{G})$ is a sentence then we know by induction that $e(A_{G})=e(A_{H})$. Hence,

[TABLE]

Next, if $e(A_{G})$ is a column vector, then we know that $e(A_{G})=T\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{H})$ and furthermore, since $\mathsf{ML}({\cal L})$-vectors are constant on equitable partitions, that $e(A_{G})=\sum_{i=1}^{\ell}a_{i}\times\mathbb{1}_{V_{i}}$ and $e(A_{H})=\sum_{i=1}^{\ell}b_{i}\times\mathbb{1}_{W_{i}}$ for some scalars $a_{i}$ and $b_{i}$ in $\mathbb{C}$, for $i=1,\ldots,\ell$. Here, $\mathbb{1}_{V_{i}}$ and $\mathbb{1}_{W_{i}}$, for $i=1,\ldots,\ell$, are the indicator vectors representing the equitable partitions ${\cal V}=\{V_{1},\ldots,V_{\ell}\}$ of $G$ and ${\cal W}=\{W_{1},\ldots,W_{\ell}\}$ of $H$, respectively. We first show that $a_{i}=b_{i}$, for $i=1,\ldots,\ell$. Indeed, since $T\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=\mathbb{1}$ and $T$ is compatible with ${\cal V}$ and ${\cal W}$, we have that

[TABLE]

As a consequence, using that $\mathbb{1}_{V_{i}}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{V_{j}}^{\phantom{a}}$ is [math] if $i\neq j$ and $|V_{i}|$ if $i=j$, we obtain

[TABLE]

for all $i=1,\ldots,\ell$. Since $|V_{i}|=|W_{i}|\neq 0$, we indeed have that $a_{i}=b_{i}$ for all $i=1,\ldots,\ell$.

We may now conclude that

[TABLE]

Hence $e^{\prime}(A_{G})$ and $e^{\prime}(A_{H})$ are indeed $T$-conjugate. ∎

In the context of Proposition 7.3, i.e., to show that the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ operation preserves $S$-conjugation (and $S^{*}$-conjugation), we need to verify that $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-vectors are constant on equitable partitions. We verify this, in the appendix, by induction on the structure of expressions in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$. In fact, we more generally show the following.

Proposition 7.4**.**

$\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)$-vectors are constant on equitable partitions.  ∎

All combined, we obtain the following characterisations.

Theorem 7.3**.**

Let $G$ and $H$ be two graphs of the same order. Then, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ if and only if there is doubly stochastic matrix $S$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muS=S\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ if and only if $G\equiv_{\mathsf{C}^{2}}H$ if and only if $G$ and $H$ have a common equitable partition.  ∎

Proof.

This is a direct consequence of Propositions 7.1, 7.2, 7.3 and 7.4. ∎

We further complement Theorem 7.3 by observing that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ if and only if $\textsf{HOM}_{\cal F}(G)=\textsf{HOM}_{\cal F}(H)$ where ${\cal F}$ consists of all trees [27].

As a consequence, following Example 7.2, sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,^{*},\mathbb{1},\operatorname{\mathsf{diag}})$ can distinguish $G_{1}$ (​  ) and $H_{1}$ (​  ), $G_{2}$ (​  ) and $H_{2}$ (​ ), $G_{4}$ (​  ) and $H_{4}$ (​  ), because all these pairs of graphs do not have a common equitable partition. By contrast, $G_{3}$ (​  ) and $H_{3}$ (​  ) cannot be distinguished by sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,^{*},\mathbb{1},\operatorname{\mathsf{diag}})$.

We remark that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ if and only if $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)}H$. This is again a direct consequence of the fact that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ implies that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muS=S\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ and $A_{H}\mskip 2.0mu{\cdot}\mskip 2.0muS^{*}=S^{*}\mskip 2.0mu{\cdot}\mskip 2.0muA_{G}$, for a doubly stochastic matrix $S$, and that all operations in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\allowbreak\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)$ preserve $S$-conjugation and $S^{*}$-conjugation.

7.4 Characterisation of

$\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-equivalence

We next consider $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-equivalence. We already know a couple of implications when $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ holds. For example, there must exist an orthogonal matrix $O$ such that $O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=\mathbb{1}$ and $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ (Propositions 6.4 and 6.5). Furthermore, we know that $G$ and $H$ must have a common equitable partition and hence, there exists a doubly stochastic matrix $S$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muS=S\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ (Proposition 7.3). It is tempting to conjecture that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ if and only if there exists an orthogonal doubly stochastic matrix $O$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$. This does not hold, however. Indeed, invertible doubly stochastic matrices are necessarily permutation matrices (see e.g., Theorem 2.1 in [29]). Then, $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ would imply that $G$ and $H$ are isomorphic, contradicting that our fragments cannot go beyond $\mathsf{C}^{3}$-equivalence [11]. Instead, we have the following characterisation.

Theorem 7.4**.**

Let $G$ and $H$ be two graphs of the same order. Then the following hold: $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ if and only if $G$ and $H$ have a common equitable partition and $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ for some doubly quasi-stochastic orthogonal matrix $O$ which is compatible with some equitable partitions ${\cal V}$ of $G$ and ${\cal W}$ of $H$, that witness that $G$ and $H$ have a common equitable partition.

In the proof of Theorem 7.4 we will rely on Specht’s Theorem (see e.g., [48]), which we recall first. Let ${\cal A}=A_{1},\ldots,A_{p}$ and ${\cal B}=B_{1},\ldots,B_{p}$ be two sequences of complex matrices that are closed under complex conjugate transposition. That is, if $A_{i}$ occurs in ${\cal A}$ then so does $A_{i}^{*}$; the same must hold for ${\cal B}$. The sequences ${\cal A}$ and ${\cal B}$ are called simultaneously unitary equivalent if there exists a unitary matrix $U$ such that $A_{i}\mskip 2.0mu{\cdot}\mskip 2.0muU=U\mskip 2.0mu{\cdot}\mskip 2.0muB_{i}$, for $i=1,\ldots,p$. Specht’s Theorem provides a means of checking simultaneous unitary equivalence in terms of trace identities. Indeed, Specht’s Theorem states that ${\cal A}$ and ${\cal B}$ are simultaneously unitary equivalent if and only if

[TABLE]

for all words $w(x_{1},\ldots,x_{p})$ over the alphabet $\{x_{1},\ldots,x_{p}\}$. In expression $w(A_{1},\ldots,A_{p})$ we instantiated $x_{i}$ with $A_{i}$ and interpret concatenation in the word $w$ as matrix multiplication; we interpret $w(B_{1},\ldots,B_{p})$ in a similar way. Specht’s Theorem also holds when ${\cal A}$ and ${\cal B}$ consist of real matrices and unitary matrices are replaced by orthogonal matrices [48]. The required condition is that ${\cal A}$ and ${\cal B}$ are closed under transposition. We next show Theorem 7.4.

Proof.

To show that the existence of a matrix $O$, as stated in the Theorem, implies the equivalence $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$, we argue as before. More precisely, we show that $O$-conjugation is preserved by the operations in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$. This is, however, a direct consequence of Lemmas 5.1, 5.2, 6.1 and 7.1. We remark that Proposition 7.4 guarantees that Lemma 7.1 can be applied. Indeed, Proposition 7.4 implies that $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-vectors are constant on equitable partitions. We may thus conclude that all expressions in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$ preserve $O$-conjugation. Hence, $e(A_{G})=e(A_{H})$ for any sentence $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$.

For the converse direction, we need to show that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ implies that there exists an orthogonal matrix $O$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$, and where $O$ satisfies the conditions mentioned in the statement of the Theorem.

The existence of the orthogonal matrix $O$ is shown using Specht’s Theorem, which we just recalled. This is done as follows: We first rephrase the conditions required for $O$, i.e., that it is a doubly quasi-stochastic matrix which is compatible with some equitable partitions ${\cal V}$ and ${\cal W}$ of $G$ and $H$, respectively, in terms of such trace identities. Then we show that these trace identities can be expressed by sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$.

We note that our approach is inspired by Thüne [68]. In that work, simultaneous unitary equivalence of graphs is studied with respect to their so-called [math], $1$ and $2$-dimensional Weisfeiler-Lehman closures. Here, we consider simultaneous orthogonal equivalence with respect to all possible matrices that can be obtained using $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-expressions.

We start by defining the sequences ${\cal A}$ and ${\cal B}$ on which we will apply Specht’s Theorem. Consider the sequences of real symmetric matrices: ${\cal A}:=A_{G},J,\allowbreak\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{1}}),\ldots,\allowbreak\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{\ell}})$ and ${\cal B}:=A_{H},J,\operatorname{\mathsf{diag}}(\mathbb{1}_{W_{1}}),\ldots,\operatorname{\mathsf{diag}}(\mathbb{1}_{W_{\ell}})$, where $\mathbb{1}_{V_{i}}$ and $\mathbb{1}_{W_{i}}$, for $i=1,\ldots,\ell$, denote the indicator vectors corresponding to the coarsest equitable partitions ${\cal V}=\{V_{1},\ldots,V_{\ell}\}$ and ${\cal W}=\{W_{1},\ldots,W_{\ell}\}$ of $G$ and $H$, respectively. We observe that ${\cal A}$ and ${\cal B}$ are closed under transposition. By the real counterpart of Specht’s Theorem we can check whether there exists an orthogonal matrix $O$ such that

[TABLE]

hold, for $i=1,\ldots,\ell$, in terms of trace identities. It is clear that conditions (2) and (4) express that $A_{G}$ and $A_{H}$ must be $O$-conjugate and that $O$ must be compatible with ${\cal V}$ and ${\cal W}$. The orthogonality of $O$ is implied by Specht’s Theorem. Condition (3) ensures that we can choose $O$ such that $O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=\mathbb{1}$ holds. To see this, we next modify the proof of Lemma 4 in Thüne [68], stated for unitary matrices, so that it holds for orthogonal matrices.

We first observe that $\mathbb{1}$ is an eigenvector of $O$. Indeed, $J\mskip 2.0mu{\cdot}\mskip 2.0muO\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=\mathbb{1}\mskip 2.0mu{\cdot}\mskip 2.0mu(\mathbb{1}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0muO\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1})=\alpha\times\mathbb{1}$ with $\alpha=\mathbb{1}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0muO\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}$ and $J\mskip 2.0mu{\cdot}\mskip 2.0muO\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=O\mskip 2.0mu{\cdot}\mskip 2.0muJ\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=(\mathbb{1}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1})\times O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}$. In other words, $O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=\frac{\alpha}{n}\times\mathbb{1}$ since $\mathbb{1}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=n$. Furthermore, because $\mathbb{1}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0muO^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}$ is a scalar, $\mathbb{1}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0muO^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=(\mathbb{1}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0muO^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1})^{\mathsf{t}}=\mathbb{1}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0muO\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=\alpha$. We next show that $\alpha=\pm n$. Indeed, since $O$ is an orthogonal matrix

[TABLE]

and thus $\alpha^{2}=n^{2}$ or $\alpha=\pm n$. Hence, $O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=\pm\mathbb{1}$. When $O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=\mathbb{1}$, $O$ is already doubly quasi-stochastic. In case when $O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=-\mathbb{1}$, we simply replace $O$ by $(-1)\times O$ to obtain that $O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=\mathbb{1}$. This rescaling does not impact the validity of conditions (2) and (4). Hence, $O$ can indeed be assumed to be doubly quasi-stochastic.

It remains to show that the trace identities implying the existence of an orthogonal matrix $O$ satisfying conditions (2), (3) and (4) can be expressed in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$. For every word $w(x,j,b_{1},\allowbreak\ldots,b_{\ell})$ we consider the sentence

[TABLE]

in which variables $x,j,b_{1},\ldots,b_{\ell}$ are assigned to matrix variable $X$, expression $\mathbb{1}(X)\mskip 2.0mu{\cdot}\mskip 2.0mu(\mathbb{1}(X))^{*}$ (which evaluates to $J$), and expressions $\operatorname{\mathsf{diag}}(\mathsf{eqpart}_{i}(X))$, for $i=1,\ldots,\ell$, respectively. Here, the expressions $\mathsf{eqpart}_{i}(X)$ correspond to the expressions extracting the indicator vectors of the (coarsest) equitable partition of a graph, as defined in the proof of Proposition 7.2. We recall from that proof that $\mathsf{eqpart}_{i}(X)$ are expressible in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times)$. As a consequence, the sentences $e_{w}(X)$ belong to $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times)$. We have seen, however, in the proof of Proposition 7.2, that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times)}H$. Hence, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ implies that $e_{w}(A_{G})=e_{w}(A_{H})$ for every word $w$. ∎

We note that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$. The converse does not hold, as is illustrated next.

Example 7.5**.**

Consider $G_{3}$ (​  ) and $H_{3}$ (​  ). These graphs are fractional isomorphic (and thus $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-equivalent) but are not co-spectral. Hence, $G_{3}\not\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H_{3}$ since $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-equivalence implies co-spectrality. On the other hand, $G_{5}$ (​ ) and $H_{5}$ (​ ) are co-spectral regular graphs (Figure 2 in van Dam et al. [71]), with co-spectral complements. The equitable partitions ${\cal V}$ of $G_{5}$ and ${\cal W}$ of $H_{5}$, consisting of a single part containing all vertices, witness that $G_{5}$ and $H_{5}$ have a common equitable partition. We thus know from before that there exists an orthogonal matrix $O$ such that $A_{G_{5}}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H_{5}}$ and $O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=\mathbb{1}$ (this follows from $G_{5}$ and $H_{5}$ being co-spectral and co-main). Moreover, the compatibility requirement with ${\cal V}$ and ${\cal W}$ is vacuously satisfied. Indeed, these partitions are represented by the indicator vector $\mathbb{1}$ and we note that $\operatorname{\mathsf{diag}}(\mathbb{1})\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0mu\operatorname{\mathsf{diag}}(\mathbb{1})$ always holds. Hence, $G_{5}$ and $H_{5}$ cannot be distinguished by $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$ by Theorem 7.4.   ∎

It would be tempting to conjecture that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ if and only if $G$ and $H$ are co-spectral and have a common equitable partition. It is clearly a necessary condition. We see in the next Section, however, that there exists graphs that are co-spectral and have a common equitable partition, yet are not $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-equivalent.

We remark that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ if and only if $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu.\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)}H$. This is again a direct consequence of the fact that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ implies that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$, $A_{H}\mskip 2.0mu{\cdot}\mskip 2.0muO^{*}=O^{*}\mskip 2.0mu{\cdot}\mskip 2.0muA_{G}$, for an orthogonal doubly quasi-stochastic matrix $O$ which is compatible with equitable partitions of $G$ and $H$, that witness that $G$ and $H$ have a common equitable partition. Furthermore, for such matrices $O$, we observe that all operations in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\allowbreak\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)$ preserve $O$-conjugation and $O^{*}$-conjugation.

7.5 Pointwise function applications on vectors

A crucial ingredient for obtaining characterisations of equivalence in the presence of the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ operation is that vectors are constant on equitable partitions (Proposition 7.4 and Lemma 7.1). In this way, vectors obtained by evaluating expressions on equivalent $A_{G}$ and $A_{H}$ are “almost” the same, up to the use of indicator vectors. More precisely, when $e(A_{G})=\sum_{i=1}^{\ell}a_{i}\times\mathbb{1}_{V_{i}}$ then $e(A_{H})=\sum_{i=1}^{\ell}a_{i}\times\mathbb{1}_{W_{i}}$, and vice versa.

We next show that this tight relationship between vectors computed from $A_{G}$ and $A_{H}$ allows us to extend the matrix query languages considered in this section with pointwise function applications on vectors. More precisely, we denote by $\operatorname{\mathsf{apply}}_{\mathsf{v}}[f]$, for $f\in\Omega$, that we only allow function applications of the form $e(X):=\operatorname{\mathsf{apply}}_{\mathsf{v}}[f](e_{1}(X),\ldots,\allowbreak e_{p}(X))$ where each $e_{i}(X)$ returns a vector when evaluated on a matrix.

Proposition 7.5**.**

Let $G$ and $H$ be two graphs of the same order.

• (1)

$G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$* if and only if $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],\operatorname{\mathsf{apply}}_{\mathsf{v}}[f],f\in\Omega)}H$.*

• (2)

$G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$* if and only if $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],\operatorname{\mathsf{apply}}_{\mathsf{v}}[f],f\in\Omega)}H$.*

Proof.

In view of the previous results, it suffices to show that (1) $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\allowbreak\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],\allowbreak f\in\Omega)$-equivalence implies $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],\allowbreak\operatorname{\mathsf{apply}}_{\mathsf{v}}[f],f\in\Omega)$-equivalence; and that (2) $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\allowbreak\,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)$-equivalence implies $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\allowbreak\,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],\operatorname{\mathsf{apply}}_{\mathsf{v}}[f],f\in\Omega)$-equivalence. Both implication follow if we can show that $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\allowbreak\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],\operatorname{\mathsf{apply}}_{\mathsf{v}}[f],f\in\Omega)$-vectors are constant on equitable partitions and that $\operatorname{\mathsf{apply}}_{\mathsf{v}}[f]$, for $f\in\Omega$, preserves conjugation by quasi doubly-stochastic matrices that are compatible with equitable partitions of $G$ and $H$ that witness that $G$ and $H$ have a common equitable partition.

For conciseness, let ${\cal L}^{\dagger}$ denote $\{\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f]\operatorname{\mathsf{apply}}_{\mathsf{v}}[f],f\in\Omega\}$, i.e., ${\cal L}^{\dagger}$ consists of all operations considered so far. Proposition 7.4 (being constant on partitions) trivially generalizes to $\mathsf{ML}({\cal L}^{\dagger})$-vectors. Indeed, it suffices to consider the case

[TABLE]

where $e_{1}(X),\ldots,e_{p}(X)$ are expressions in $\mathsf{ML}({\cal L}^{\dagger})$ such that each $e_{i}(A_{G})$ returns a vector. We may assume by induction that for $i=1,\ldots,p$, $e_{i}(A_{G})=\sum_{j=1}^{\ell}a_{j}^{(i)}\times\mathbb{1}_{V_{i}}$ for scalars $a_{j}^{(i)}\in\mathbb{C}$, for $j=1,\ldots,\ell$, and where $\mathbb{1}_{V_{i}}$, for $i=1,\ldots,\ell$, are indicator vectors representing an equitable partition of $G$. Since the sets of entries in the indicator vectors holding value $1$ are disjoint for any two different indicator vectors, we have that

[TABLE]

So, indeed, $\mathsf{ML}({\cal L}^{\dagger})$-vectors are constant on equitable partitions.

Let ${\cal V}=\{V_{1},\ldots,V_{\ell}\}$ and ${\cal W}=\{W_{1},\ldots,W_{\ell}\}$ be equitable partitions of $G$ and $H$, respectively. Suppose that these partitions witness that $G$ and $H$ have a common equitable partition. That $T$-conjugation, for a quasi doubly-stochastic matrices $T$ that is compatible with ${\cal V}$ and ${\cal W}$, is also preserved by pointwise function applications on vectors now easily follows. Indeed, consider $e(X):=\operatorname{\mathsf{apply}}_{\mathsf{v}}[f](e_{1}(X),\ldots,e_{p}(X))$. By assumption, $e_{i}(A_{G})=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{i}(A_{H})$ for all $i=1,\ldots,p$. Furthermore, $e_{i}(A_{G})=\sum_{j=1}^{\ell}a_{j}^{(i)}\times\mathbb{1}_{V_{i}}$ and $e_{i}(A_{H})=\sum_{j=1}^{\ell}b_{j}^{(i)}\times\mathbb{1}_{W_{i}}$. The indicator vectors $\mathbb{1}_{V_{i}}$ and $\mathbb{1}_{W_{i}}$, for $i=1,\ldots,\ell$, represent the partitions ${\cal V}$ and ${\cal W}$, as before. We have seen in the proof of Lemma 7.1 that $T$-conjugation of these vectors implies $a_{j}^{(i)}=b_{j}^{(i)}$ for $j=1,\ldots,\ell$ and $i=1,\ldots,p$. As a consequence, $e(A_{G})$ is equal to

[TABLE]

which is equal to $T\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{H})$, as desired. ∎

Going back to the graphs $G_{5}$ (​ ) and $H_{5}$ (​ ) in Example 7.5, these cannot even be distinguished by sentences in the large fragments in Proposition 7.5. In Section 9, we show that by allowing pointwise function applications on matrices, and in particular the Schur-Hadamard product, we can distinguish these two graphs.

8 The impact of pointwise multiplication on vectors

In the preceding section the main use of the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ operation related to the construction of the coarsest equitable partition (see e.g., the proof of Proposition 7.2) and more specifically, to the ability to pointwise multiply two vectors (see e.g., Example 7.1). In this section we investigate how $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-equivalence and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$-equivalence change if we replace the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ operation with the operation, denoted by $\odot_{v}$, which pointwise multiplies vectors.

We first remark that $e_{1}(X)\odot_{v}e_{2}(X)$ can be expressed as $\operatorname{\mathsf{diag}}(e_{1}(X))\mskip 2.0mu{\cdot}\mskip 2.0mu\operatorname{\mathsf{diag}}(e_{2}(X))\mskip 2.0mu{\cdot}\mskip 2.0mu\allowbreak\mathbb{1}(X)$. So surely, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\odot_{v})}H$. We show below that the converse also holds. For this fragment, it thus does not matter whether we include $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ or $\odot_{v}$. Similarly, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})}H$. The converse, however, does not hold as we will show below555It was incorrectly stated in the conference version [33] that $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ and $\odot_{v}$ are interchangeable.. Finally, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})}H$ trivially implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\odot_{v})}H$. The results in this section show that the converse does not hold.

Example 8.1**.**

Consider the graphs $G_{6}$ (​​ ) and $H_{6}$ (​ )666I am indebted to David E. Roberson for providing these two graphs [64].. On can verify that these graphs are co-spectral and have a common equitable partition (and thus also have co-spectral complements). Using the diagonal operation we can construct the Laplacian of a graph by simply considering expression $L(X):=(\operatorname{\mathsf{diag}}(X\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}(X))-X$. It is now easy to detect that $G_{6}$ and $H_{6}$ have Laplacians that are not co-spectral. Indeed, consider the $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times)$ expression $e_{L,k}(X):=\mathsf{tr}(L(X)^{k})$. Then, we can check that $e_{L,3}(A_{G_{6}})=1602\neq 1618=e_{L,3}(A_{H_{6}})$. The relation between co-spectrality and traces of powers of matrices (cfr. Proposition 5.1) holds more generally for symmetric matrices (this follows easily from the real version of Specht’s Theorem used in the proof of Theorem 7.4). Hence, we can infer that the Laplacians of $G_{6}$ and $H_{6}$ are not co-spectral. Another way of verifying this is that $G_{6}$ and $H_{6}$ have a different number of spanning trees ($192$ versus $160$) and Kirchhoff’s matrix tree theorem (see e.g., Proposition 1.3.4 in [14]) implies that graphs with co-spectral Laplacians must have the same number of spanning trees. Hence, $G_{6}$ and $H_{6}$ can be distinguished by $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},+,\times)$ (and hence also by sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$ as we have seen before). Nevertheless, we will see that $G_{6}$ and $H_{6}$ cannot be distinguished by sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})$. More generally, we show that two graphs are $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})$-equivalent if and only if they are co-spectral and have a common equitable partition (see Proposition 8.3 below).  ∎

As mentioned earlier, when considering $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$ one can equivalently use $\odot_{v}$ instead of $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$.

Proposition 8.1**.**

Let $G$ and $H$ be two graphs of the same order. Then, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\odot_{v})}H$ if and only if $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$.

Proof.

We already observed at the beginning of this section that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\odot_{v})}H$. It remains to show the reverse implication. This can be proved by a simple inductive argument, transforming sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot_{v})$ into sentences without $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$. With each sentence $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot_{v})$ we associate, in a natural way, the number of occurrences, denoted by $k$, of the $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ operation in this sentence. We will show, by induction on $k$, that any sentence $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot_{v})$ is equivalent to a sentence $\tilde{e}(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\odot_{v})$. Here, equivalence of sentences means that $e(A)=\tilde{e}(A)$ for any input matrix $A$. Consider a sentence $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot_{v})$. Clearly, if $k=0$, then $e(X)$ is already a sentence in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\odot_{v})$. Assume now that any sentence $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot_{v})$ with $k$ occurrences is equivalent to a sentence $\tilde{e}(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\odot_{v})$. Consider a sentence $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot_{v})$ having $k+1$ occurrences of $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$. Since $e(X)$ is a sentence, we can write $e(X)$ as $e_{1}(X)\mskip 2.0mu{\cdot}\mskip 2.0mu\operatorname{\mathsf{diag}}(e_{2}(X))\mskip 2.0mu{\cdot}\mskip 2.0mue_{3}(X)$, where $e_{1}(X)$, $e_{2}(X)$ and $e_{3}(X)$ are expressions in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot_{v})$. We will next eliminate the occurrence of $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$ in $\operatorname{\mathsf{diag}}(e_{2}(X))$. We distinguish between the following two cases, depending on the dimension of $e_{2}(A)$, were $A$ is an $n\times n$-matrix. Suppose first that $e_{2}(A)$ is a scalar. Clearly, in this case $\operatorname{\mathsf{diag}}(e_{2}(X))$ is equivalent to $e_{2}(X)$ and hence $e(X)$ is equivalent to the expression $e^{\prime}(X):=e_{1}(X)\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(X)\mskip 2.0mu{\cdot}\mskip 2.0mue_{3}(X)$. Suppose next that $e_{2}(A)$ evaluates to an $n\times 1$ column vector, with $n>1$. Then, necessarily, $e_{1}(A)$ evaluates to a $1\times n$ row vector and $e_{3}(A)$ evaluates to an $n\times 1$ column vector. This implies that $e(X)$ is equivalent to $e^{\prime}(X):=e_{1}(X)\mskip 2.0mu{\cdot}\mskip 2.0mu(e_{2}(X)\odot_{v}e_{3}(X))$. We remark that in both cases, $e^{\prime}(X)$ is an expression in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot_{v})$ consisting of $k$ occurrences of $\operatorname{\mathsf{diag}}(\mskip 2.0mu{\cdot}\mskip 2.0mu)$. By the induction hypothesis, $e^{\prime}(X)$ (and hence also $e(X)$) is indeed equivalent to a sentence $\tilde{e}(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\odot_{v})$, as desired. Hence, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\odot_{v})}H$ implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$.

∎

The above proof fails when the trace operation is present. The reason is that we can have sentences like $e_{L,k}(X)$ in Example 8.1, which cannot be decomposed as $e_{1}(X)\mskip 2.0mu{\cdot}\mskip 2.0mu\operatorname{\mathsf{diag}}(e_{2}(X))\mskip 2.0mu{\cdot}\mskip 2.0mue_{3}(X)$. The previous proof crucially relies on the fact that all sentences can be written in that form.

We next consider $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})$. To analyse the distinguishability of graphs by sentences in this fragment, we follow the same approach as for $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})$.

Corollary 8.2**.**

Let $G$ and $H$ be two graphs of the same order. Then, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})}H$ implies that $G$ and $H$ have a common equitable partition.

Proof.

This is a direct consequence of Propositions 7.2 and 8.1. Indeed, we know already that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ implies that $G$ and $H$ have a common equitable partition and we have just shown that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}})}H$ if and only if $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\odot_{v})}H$. It now suffices to observe that $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\odot_{v})$ is a smaller fragment than $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})$.∎

Furthermore, we can add pointwise vector multiplication to the list of operations in Proposition 7.4. We defer the proof to the appendix.

Proposition 8.2**.**

$\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot_{v},\operatorname{\mathsf{diag}},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)$-vectors are constant on equitable partitions. ∎

It remains to identify an appropriate class of matrices for which conjugation is preserved by pointwise vector multiplication. Let $G$ and $H$ be two graphs that have a common equitable partition. As before, let ${\cal V}=\{V_{1},\ldots,V_{\ell}\}$ and ${\cal W}=\{W_{1},\ldots,W_{\ell}\}$ be equitable partitions of $G$ and $H$, respectively. The corresponding indicator vectors are denoted by $\mathbb{1}_{V_{i}}$ and $\mathbb{1}_{W_{i}}$, for $i=1,\ldots,\ell$, respectively. We say that a matrix $T$ preserves the partitions ${\cal V}$ and ${\cal W}$ if $\mathbb{1}_{V_{i}}=T\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$ for all $i=1,\ldots,\ell$. We note that this condition is weaker than the compatibility notion used before (see the proof of Lemma 7.1 where we verified the preservation of partitions for matrices that are compatible with these partitions).

Lemma 8.1**.**

Let $\mathsf{ML}(\cal L)$ be a matrix query language such that $\mathsf{ML}(\cal L)$-vectors are constant on equitable partitions. Let $G$ and $H$ be two graphs of the same order and let ${\cal V}$ and ${\cal W}$ be equitable partitions of $G$ and $H$, respectively. Assume that ${\cal V}$ and ${\cal W}$ witness that $G$ and $H$ have a common equitable partition. Furthermore, let $T$ be a matrix which preserves ${\cal V}$ and ${\cal W}$. Finally, Let $e_{1}(X)$ and $e_{2}(X)$ be expressions in $\mathsf{ML}(\cal L)$ which evaluate to vectors. Then, if $e_{1}(A_{G})$ and $e_{1}(A_{H})$ are $T$-conjugate, and $e_{2}(A_{G})$ and $e_{2}(A_{H})$ are $T$-conjugate, then also $e_{1}(A_{G})\odot_{v}e_{2}(A_{G})$ and $e_{1}(A_{H})\odot_{v}e_{2}(A_{H})$ are $T$-conjugate.

Proof.

The proof is similar to the proof of Lemma 7.1. Consider $e(X):=e_{1}(X)\odot_{v}e_{2}(X)$. We distinguish between two cases, depending on the dimensions of $e(A_{G})$. First, if $e(A_{G})$ is a sentence then we know by induction that $e_{1}(A_{G})=e_{1}(A_{H})$ and $e_{2}(A_{G})=e_{2}(A_{H})$. Hence,

[TABLE]

Next, if $e_{1}(A_{G})$ and $e_{2}(A_{G})$ are (column) vectors, then we know that $e_{1}(A_{G})=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{H})$ and $e_{2}(A_{G})=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{H})$. We argued in the proof of Lemma 7.1 that when $\mathbb{1}_{V_{i}}=T\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$ holds for $i=1,\ldots,\ell$, then since vectors are constant on equitable partitions, $e_{1}(A_{G})=\sum_{i=1}^{\ell}a_{i}\times\mathbb{1}_{V_{i}}=\sum_{i=1}^{\ell}a_{i}\times(T\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}})=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{H})$ and $e_{2}(A_{G})=\sum_{i=1}^{\ell}b_{i}\times\mathbb{1}_{V_{i}}=\sum_{i=1}^{\ell}b_{i}\times(T\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}})=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{H})$. We may now conclude that

[TABLE]

Hence, $e(A_{G})$ and $e(A_{H})$ are indeed $T$-conjugate. ∎

We can now state a characterisation of $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})$-equivalence.

Theorem 8.3**.**

Let $G$ and $H$ be two graphs of the same order. Let ${\cal V}$ and ${\cal W}$ be equitable partitions of $G$ and $H$, respectively, that witness that $G$ and $H$ have a common equitable partition. Then, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})}H$ if and only if there exists an orthogonal matrix $O$ which preserves ${\cal V}$ and ${\cal W}$ and such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$.

Proof.

To show that the existence of a matrix $O$, as stated in the Theorem, implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})}H$, we argue as before. More precisely, we show that $O$-conjugation and $O^{*}$-conjugation is preserved by the operations in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})$. This is, however, a direct consequence of Lemmas 5.1, 5.2, 5.4, 6.1 and 8.1. We remark that Proposition 8.2 guarantees that Lemma 8.1 can be applied. Indeed, Proposition 8.2 implies that $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})$-vectors are constant on equitable partitions. Furthermore, since $\mathbb{1}_{V_{i}}=O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$, for all $i=1,\ldots,\ell$, and $\mathbb{1}=\sum_{i=1}^{\ell}\mathbb{1}_{V_{i}}=\sum_{i=1}^{\ell}\mathbb{1}_{W_{i}}$, we have that $\mathbb{1}=O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}$. Hence, $O$ is doubly quasi-stochastic and Lemma 6.1 applies. Moreover, if $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ then $A_{H}\mskip 2.0mu{\cdot}\mskip 2.0muO^{*}=O^{*}\mskip 2.0mu{\cdot}\mskip 2.0muA_{G}$ due to the orthogonality of $O$, and $O^{*}$ preserves ${\cal W}$ and ${\cal V}$. Hence, Lemma 5.4 applies. We may thus conclude that all expressions in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})$ preserve $O$- and $O^{*}$-conjugation. Hence, $e(A_{G})=e(A_{H})$ for any sentence $e(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})$.

For the converse direction, we need to show that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\,\mathsf{tr},\mathbb{1},\odot_{v})}H$ implies that there exists an orthogonal matrix $O$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$, and where $O$ preserves the partitions ${\cal V}$ and ${\cal W}$. This can be shown, just like in the proof of Theorem 7.4, by means of trace conditions. In particular, we impose trace conditions such that $O$ satisfies $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ and $(\mathbb{1}_{V_{i}}\mskip 2.0mu{\cdot}\mskip 2.0mu(\mathbb{1}_{V_{i}})^{*})\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0mu(\mathbb{1}_{W_{i}}\mskip 2.0mu{\cdot}\mskip 2.0mu(\mathbb{1}_{W_{i}})^{*})$, for $i=1,\ldots,\ell$. These conditions replace conditions (3) and (4) in the proof of Theorem 7.4. We show in the appendix that this indeed implies that $O$ can be chosen such that it preserves ${\cal V}$ and ${\cal W}$. The argument is based on a generalisation of Lemma 4 in Thüne [68]. As in the proof of Theorem 7.4, the trace conditions $e_{w}(X)$ which ensure the existence of an orthogonal matrix $O$ such that $(\mathbb{1}_{V_{i}}\mskip 2.0mu{\cdot}\mskip 2.0mu(\mathbb{1}_{V_{i}})^{*})\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0mu(\mathbb{1}_{W_{i}}\mskip 2.0mu{\cdot}\mskip 2.0mu(\mathbb{1}_{W_{i}})^{*})$ holds for $i=1,\ldots,\ell$, rely on the expressions $\mathsf{eqpart}_{i}(X)$ (from the proof of Proposition 7.2). These expressions use addition and scalar multiplication. It is now easily verified (see appendix) that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})}H$ implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v},+,\times)}H$ and hence, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})}H$ implies that $e_{w}(A_{G})=e_{w}(A_{H})$. ∎

As it turns out, $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})$-equivalence precisely captures co-spectral and fractional isomorphic graphs.

Proposition 8.3**.**

Let $G$ and $H$ be graphs of the same order. Then, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})}H$ if and only if $G$ and $H$ are co-spectral and have a common equitable partition.

We also observe that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},^{*},\mathbb{1},\odot_{v})}H$ if and only if $\textsf{HOM}_{\cal F}(G)=\textsf{HOM}_{\cal F}(H)$ where ${\cal F}$ consists of all trees and cycles. This follows from the results in Dell et al. [27].

Proof.

If $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})}H$, then $G$ and $H$ must have a common equitable partition by Corollary 8.2. Furthermore, we know from Propositions 5.1 and 5.2, that $G$ and $H$ must also be co-spectral. For the converse, we explicitly construct an orthogonal matrix $O$ such that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ and $O$ preserves equitable partitions of $G$ and $H$ that witness that $G$ and $H$ have a common equitable partition. Then, Theorem 8.3 implies that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})}H$ holds.

We next construct the matrix $O$. Let $G$ and $H$ be two graphs of order $n$ and assume that ${\cal V}=\{V_{1},\ldots,V_{\ell}\}$ and ${\cal W}=\{W_{1},\ldots,W_{\ell}\}$ are equitable partitions of $G$ and $H$, respectively, that witness that $G$ and $H$ have a common equitable partition. As before, we denote by $\mathbb{1}_{V_{1}},\ldots,\mathbb{1}_{V_{\ell}}$ the indicator vectors corresponding to ${\cal V}$ and by $\mathbb{1}_{W_{1}},\ldots,\mathbb{1}_{W_{\ell}}$ the indicator vectors corresponding to ${\cal W}$.

We first observe the following. It is known that, for indicator vectors representing an equitable partition, the subspace $U_{G}=\textsf{span}(\mathbb{1}_{V_{1}},\ldots,\mathbb{1}_{V_{\ell}})$ of $\mathbb{C}^{n}$ is an $A_{G}$-invariant subspace (see e.g., Lemma 5.2 in [16]). In other words, for any $v\in U_{G}$, $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muv\in U_{G}$. Furthermore, since $A_{G}$ is a symmetric matrix, also the orthogonal complement subspace $U_{G}^{\bot}$ is $A_{G}$-invariant (see e.g., Theorem 36 in [50]). Here, $U_{G}^{\bot}$ consists of all vectors $v^{\prime}$ in $\mathbb{C}^{n}$ that are orthogonal to any vector $v\in U_{G}$, i.e., such $v^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0muv^{\prime}=0$ holds. Let us interpret $A_{G}$ as the linear operator $T_{G}:\mathbb{C}^{n}\to\mathbb{C}^{n}:v\mapsto A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muv$. This is a diagonalisable operator (because $A_{G}$ is symmetric) and it is known that the restrictions $T_{G}|_{U_{G}}$ and $T_{G}|_{U_{G}^{\bot}}$ are also diagonalisable operators (because of the invariance of these two subspaces (see e.g., Corollary 15.9 in [36])). This implies, as this is merely a restatement of a linear operator to be diagonalisable, that there exists eigenvectors $v_{1},\ldots,v_{\ell},v_{1}^{\prime},\ldots,v_{n-\ell}^{\prime}$ of $A_{G}$ such that $U_{G}=\textsf{span}(v_{1},\ldots,v_{\ell})$ and $U_{G}^{\bot}=\textsf{span}(v_{1}^{\prime},\ldots,v_{n-\ell}^{\prime})$. Furthermore, if we denote by $P_{G}$ the matrix with columns $\mathbb{1}_{V_{1}},\ldots,\mathbb{1}_{V_{\ell}}$, then $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muP_{G}=P_{G}\mskip 2.0mu{\cdot}\mskip 2.0muC$ with $C$ the $\ell\times\ell$-matrix such that $C_{ij}=\deg(v,V_{j})$ for some (arbitrary) vertex $v\in V_{i}$ (see e.g., Lemma 6.1 in [16]). Also $C_{ij}$ is diagonalisable (this follows from the fact that the characteristic polynomial of $C$ divides that of $A_{G}$ (see e.g., Theorem 6.2 in [16]) and hence there exists $\ell$ linearly independent eigenvectors $c_{1},\ldots,c_{\ell}$ of $C$777We here use that we work in the algebraically closed field $\mathbb{C}$ for which being diagonalisable coincides with the minimal polynomial being a product of monic linear factors of the form $(x-\lambda)$. So, since the characteristic polynomial of $C$ divides that of $A_{G}$, also the minimal polynomial of $C$ divides that of $A_{G}$. Since $A_{G}$ is diagonalisable, its minimal polynomial is a product of monic linear factors. Hence, also the minimal polynomial of $C$ has this form and $C$ is diagonalisable as well.. It is known that $v_{i}=P_{G}\mskip 2.0mu{\cdot}\mskip 2.0muc_{i}$, for $i=1,\ldots,\ell$, are independent eigenvectors of $A_{G}$. More precisely, if $C\mskip 2.0mu{\cdot}\mskip 2.0muc_{i}=\lambda_{i}\times c_{i}$ then $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0mu(P_{G}\mskip 2.0mu{\cdot}\mskip 2.0muc_{i})=\lambda_{i}\times(P_{G}\mskip 2.0mu{\cdot}\mskip 2.0muc_{i})$. Moreover, $P_{G}\mskip 2.0mu{\cdot}\mskip 2.0muc_{i}\in U_{G}$, for $i=1,\ldots,\ell$. We may thus assume that $U_{G}$ is spanned by $P_{G}\mskip 2.0mu{\cdot}\mskip 2.0muc_{1},\ldots,P_{G}\mskip 2.0mu{\cdot}\mskip 2.0muc_{\ell}$.

The reasoning above also holds for $A_{H}$, i.e., there are eigenvectors $w_{1},\ldots,w_{\ell},w_{1}^{\prime},\allowbreak\ldots,w_{n-\ell}^{\prime}$ of $A_{H}$ such that $U_{H}=\textsf{span}(w_{1},\ldots,w_{\ell})$ and $U_{H}^{\bot}=\textsf{span}(w_{1}^{\prime},\ldots,w_{n-\ell}^{\prime})$.

Important to observe here is that ${\cal V}$ and ${\cal W}$ witness that $G$ and $H$ have a common equitable partition. As a consequence, $A_{H}\mskip 2.0mu{\cdot}\mskip 2.0muP_{H}=P_{H}\mskip 2.0mu{\cdot}\mskip 2.0muC$, where $P_{H}$ is now the matrix with columns $\mathbb{1}_{W_{1}},\ldots,\mathbb{1}_{W_{\ell}}$ and $C$ is the same $\ell\times\ell$-matrix as used above. We may thus assume that $U_{H}$ is spanned by $P_{H}\mskip 2.0mu{\cdot}\mskip 2.0muc_{1},\ldots,P_{H}\mskip 2.0mu{\cdot}\mskip 2.0muc_{\ell}$ and furthermore, $P_{G}\mskip 2.0mu{\cdot}\mskip 2.0muc_{i}$ and $P_{H}\mskip 2.0mu{\cdot}\mskip 2.0muc_{i}$ are eigenvectors of $A_{G}$ and $A_{H}$, respectively, both belonging to the same eigenvalue $\lambda_{i}$ of $C$.

We next use that $G$ and $H$ are co-spectral. The argument above, combined with co-spectrality, implies that the (multi-set) of eigenvalues corresponding to the eigenvectors spanning $U_{G}$ and $U_{H}$ are the same. This implies in turn, by co-spectrality, that we may also assume that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muv_{i}^{\prime}=\lambda_{i}\times v_{i}^{\prime}$ and $A_{H}\mskip 2.0mu{\cdot}\mskip 2.0muw_{i}^{\prime}=\lambda_{i}\times w_{i}^{\prime}$, for $i=1,\ldots,n-\ell$, for some eigenvalues $\lambda_{i}$ of $A_{G}$ (and $A_{H}$). We recall that $U_{G}$ and $U_{H}$ are also spanned by $\mathbb{1}_{V_{i}},\ldots,\mathbb{1}_{V_{\ell}}$ and $\mathbb{1}_{W_{1}},\ldots,\mathbb{1}_{W_{\ell}}$, respectively. This implies, that the eigenvectors spanning $U_{G}^{\bot}$ and $U_{H}^{\bot}$ are necessarily orthogonal to these indicator vectors.

We define $O$ as the matrix $O_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO_{H}^{\mathsf{t}}$, where $O_{G}$ is the orthogonal matrix whose columns consist of the vectors $\frac{1}{\sqrt{n_{1}}}\mathbb{1}_{V_{1}},\ldots,\frac{1}{\sqrt{n_{\ell}}}\mathbb{1}_{V_{\ell}},v_{1}^{\prime},\ldots,v_{n-\ell}^{\prime}$ and $O_{H}$ is the orthogonal matrix whose columns consist of the vectors $\frac{1}{\sqrt{n_{1}}}\mathbb{1}_{W_{1}},\ldots,\frac{1}{\sqrt{n_{\ell}}}\mathbb{1}_{W_{\ell}},w_{1}^{\prime},\ldots,w_{n-\ell}^{\prime}$, where $n_{i}=|V_{i}|=|W_{i}|$ and where we assume the eigenvectors $v_{i}^{\prime}$ and $w_{i}^{\prime}$ to be normalised. As a consequence, $O$ is clearly an orthogonal matrix and thus $O\mskip 2.0mu{\cdot}\mskip 2.0muO^{\mathsf{t}}=I=O^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0muO$ holds. In view of the construction of the eigenvectors, we have the following more explicit expression for $O$:

[TABLE]

We verify the required conditions. To begin with, we note that $O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}=\mathbb{1}_{V_{i}}$, for $i=1,\ldots,\ell$. Indeed, this follows from the fact that $\mathbb{1}^{\mathsf{t}}_{W_{j}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$ is zero when $i\neq j$ and is $|W_{i}|=n_{i}$ when $i=j$. Moreover, $(w_{j}^{\prime})^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}=0$ because of $w_{j}^{\prime}\in U_{H}^{\bot}$, for all $j=1,\ldots,n-\ell$. Hence, $O$ indeed preserves ${\cal V}$ and ${\cal W}$. It remains to verify that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$. We verify this for both terms in the above expression for $O$. Since $v_{i}^{\prime}$ and $w_{i}^{\prime}$ are eigenvectors of $A_{G}$ and $A_{H}$, respectively, belonging to the same eigenvalue $\lambda_{i}$, we have for the second term:

[TABLE]

For the first term in the expression for $O$, we consider the matrices

[TABLE]

for some (arbitrary) vertices $v_{i}\in V_{i}$ and $w_{i}\in W_{i}$. We here used that the indicator vectors represent equitable partitions. We now look at the entries in the matrices $B_{G}$ and $B_{H}$ and observe that $J=\sum_{i,j=1}^{\ell}\mathbb{1}_{V_{j}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}^{\mathsf{t}}_{W_{i}}$. Hence, for each $p,q\in\{1,\ldots,n\}$ we can define $f(p)$ and $g(q)$ as the unique indexes of indicator vectors $\mathbb{1}_{V_{f(p)}}$ and $\mathbb{1}_{W_{g(q)}}$ such that they hold value $1$ at position $p$ and $q$, respectively. Then,

[TABLE]

because the indicator vectors correspond to equitable partitions that witness that $G$ and $H$ have a common equitable partition. Hence, we may indeed conclude that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$. ∎

Example 8.4**.**

We already mentioned that the graphs $G_{6}$ (​​ ) and $H_{6}$ (​ ) are co-spectral and have a common equitable partition. Proposition 8.3 implies that $G_{6}\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})}H_{6}$, as anticipated.  ∎

We mention that we can extend $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})$ with $+$, $\times$, and pointwise function applications on scalars and vectors, without increasing the distinguishing power of the fragments. This can be shown in precisely the same way as before by showing that $O$- and $O^{*}$-conjugation is preserved by these operations when $O$ is an orthogonal matrix which preserves equitable partitions (that witness that the graphs have a common equitable partition).

Finally, we separate distinguishability by $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})$ and $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\odot_{v})$. It suffices to consider the graphs $G_{3}$ (​  ) and $H_{3}$ (​  ) which are fractionally isomorphic but not co-spectral. Proposition 8.1 implies that these are indistinguishable by $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\odot_{v})$. Proposition 8.3 implies that they can be distinguished by $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v})$.

9 The impact of pointwise functions on matrices

The final operation that we consider is pointwise function applications on matrices. In particular, we start by considering the Schur-Hadamard product, which we denote by the binary operation $\odot$, i.e., $(A\odot B)_{ij}=A_{ij}B_{ij}$ for matrices $A$ and $B$. Our results will imply that once two graphs are equivalent with regards to sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot)$, then they will be equivalent with regards to sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot,\operatorname{\mathsf{apply}}[f],f\in\Omega)$ for any pointwise function application $\operatorname{\mathsf{apply}}[f]$, be it on scalars, vectors or matrices. That is, the graphs will be $\mathsf{MATLANG}$-equivalent.

This section is structured in a similar way as Section 7. More precisely, we first illustrate the additional power that the Schur-Hadamard product provides by means of an example in Section 9.1. Then, in Section 9.2 we show that we can compute a so-called stable edge partition of a graph by means of expressions in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot,+,\times)$. Such an edge partition can be regarded as a generalisation of the notion of equitable partition. Stable edge partitions can be obtained as the result of the edge colouring or $2$-dimensional Weisfeiler-Lehman (2WL) algorithm. In Section 9.2, we also show that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot)}H$ implies that $G$ and $H$ are indistinguishable by the 2WL-algorithm. It is known from the seminal paper by Cai, Fürer and Immerman [15], that this is equivalent to $\mathsf{C}^{3}$-equivalence. Based on this, in Section 9.3 we prove the main result of this section, i.e., that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot)}H$ if and only if $G\equiv_{\mathsf{MATLANG}}H$ if and only if $G\equiv_{\mathsf{C}^{3}}H$.

We remark that from the work by Brijder et al. [11] it implicitly follows that $\mathsf{C}^{3}$-equivalence implies $\mathsf{MATLANG}$-equivalence. Our results thus show that converse implication also holds. That is, $\mathsf{MATLANG}$-equivalence coincides with $\mathsf{C}^{3}$-equivalence.

9.1 Example of the impact of the presence of Schur-Hadamard product

We start with an example showing what extra information can be computed from graphs when the Schur-Hadamard product is present.

Example 9.1**.**

We recall that in expression $\#\mathsf{3degr}(X)$ in Example 7.1, products of diagonal matrices resulted in the ability to zoom in on vertices that carry specific degree information. When diagonal matrices are concerned, the product of matrices coincides with pointwise multiplication of the vectors on the diagonals. Allowing pointwise multiplication on matrices has the same effect, but now on edges in graphs. As an example, suppose that we want to count the number of “triangle walks” in $G$, i.e., walks $(v_{0},\ldots,v_{k})$ of length $k$ in $G$ such that each edge $\{v_{i-1},v_{i}\}$ in the walk is part of a triangle. This can be done by expression

[TABLE]

where $f_{>0}(x)=1$ if $x\neq 0$ and $f_{>0}(x)=0$ otherwise888The use of $\operatorname{\mathsf{apply}}[f_{>0}](\mskip 2.0mu{\cdot}\mskip 2.0mu)$ is just for convenience. Its application inside sentences can be simulated with operations in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\,\mathsf{tr},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot)$ when evaluated on given adjacency matrices.. Indeed, when evaluated on adjacency matrix $A_{G}$, $A_{G}^{2}\odot A_{G}$ extracts from $A_{G}^{2}$ only those entries corresponding to paths $(u,v,w)$ of length $2$ such that $(u,w)$ is an edge as well, i.e., it identifies edges involved in triangles in $G$. Then, $\operatorname{\mathsf{apply}}[f_{>0}](A_{G}^{2}\odot A_{G})$ sets all non-zero entries to $1$. By considering the $k^{\text{th}}$ power of this matrix and summing up all its entries, the number of triangle paths of length $k$ is obtained. It can be verified that for graphs $G_{5}$ (​ ) and $H_{5}$ (​ ), $\#\Delta\mathsf{paths}_{2}(A_{G_{5}})=[160]\neq[132]=\#\Delta\mathsf{paths}_{2}(A_{H_{5}})$ and hence, they can be distinguished when the Schur-Hadamard product is available. Recall that all previous fragments could not distinguish between these two graphs.   ∎

As mentioned earlier, we will use the Schur-Hadamard product to compute stable edge partitions of graphs, obtained as the result of the edge colouring or $2$-dimensional Weisfeiler-Lehman (2WL) algorithm [8, 15, 60, 73]. Such partitions can be seen as a generalization of equitable partitions, but now partitioning all pairs of vertices, rather than single vertices. We detail these notions next.

9.2 Stable edge partitions

We first recall the notion of stable edge partition and define when two graphs are indistinguishable by the 2WL algorithm. Then, similarly to the proof of Proposition 7.2, we show that when two graphs are indistinguishable by sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot)$, then they are indistinguishable by the 2WL algorithm (Proposition 9.1 below).

As already mentioned, the stable edge partition of a graph $G=(V,E)$ arises as the result of applying the 2WL-algorithm [8, 15, 60, 73] on $G$. In Algorithm 2 we provide the pseudo-code of the algorithm 2-Stab, taken from Bastert [8], which implements the 2WL-algorithm. In a nutshell, the algorithm starts by assigning every vertex pair a colour, and then revises colourings iteratively based on some structural information. When no revision of the colouring occurs, the colouring has stabilized, the algorithm stops and returns the stable colouring. Colourings naturally induce partitions of $V\times V$, by simply grouping together vertex pairs with the same colour. The stable edge partition of $G$ is the partition induced by the stable colouring returned by 2-Stab. The algorithm 2-Stab needs at most $n^{2}$ iterations when evaluated on a graph of order $n$.

More precisely, in this context, a colouring $\chi$ assigns a colour to each vertex pair in $V\times V$, i.e., if we denote by $C$ a set of colours, it is a function $\chi:V\times V\to C$. The partition of $V\times V$ induced by $\chi$ is denoted by $\Pi_{\chi}(G)$ and will be represented by indicator matrices, one for each colour $c\in C$. Assume that $V=\{1,\ldots,n\}$. An indicator matrix for a subset $Y\subseteq V\times V$ is a matrix $E_{Y}$ in $\mathbb{R}^{n\times n}$ such that $(E_{Y})_{v_{1}v_{2}}=1$ if $(v_{1},v_{2})\in Y$ and $(E_{Y})_{v_{1}v_{2}}=0$ otherwise. For a colour $c\in C$, we denote by $E_{c}$ the indicator matrix for $\{(v_{1},v_{2})\in V\times V\mid\chi(v_{1},v_{2})=c\}$. Hence, $\Pi_{\chi}(G)$ is represented by the indicator matrices $E_{c}$, for $c\in C$.

Algorithm 2-Stab starts (on lines 1 and 2) with an initial colouring $\chi_{0}:V\times V\to\{0,1,2\}$ encoding adjacency, non-adjacency and loop information. More precisely, for vertices $v,w\in V$, $\chi_{0}(v,v)=2$, $\chi_{0}(v,w)=1$ if $(v,w)\in E$, and $\chi_{0}(v,w)=0$ for $v\neq w$ and $(v,w)\not\in E$. Then, 2-Stab adjusts the current colouring in each iteration, as follows.

Suppose that the current colouring is $\chi:V\times V\to C$. Given this colouring, for each pair of vertices $v_{1},v_{2}\in V$, the so-called structure list $\mathsf{L}^{2}(v_{1},v_{2})$ is computed (lines 4 and 5). To define these lists, the structure constants are needed, which are defined as

[TABLE]

for colours $c$ and $d$ in $C$ and vertices $v_{1}$ and $v_{2}$ in $V$. These numbers count the number of triangles999With a triangle one simply means a triple $(v_{1},v_{2})$, $(v_{1},v_{3})$ and $(v_{2},v_{3})$ of vertex pairs, none of which has to be an edge in $G$., based on $(v_{1},v_{2})$ whose other two pairs of vertices $(v_{1},v_{3})$ and $(v_{3},v_{2})$ have prescribed colours $c$ and $d$, respectively. Then, in a structure list we simply gather all these numbers for a specific vertex pair. That is,

[TABLE]

Based on this information, 2-Stab will assign new colours to pairs of vertices (lines 6–8). More precisely, $C$ is replaced by a minimal set of colours $C^{\prime}$ such that each unique $\mathsf{L}^{2}(v_{1},v_{2})$ corresponds precisely to a single colour $c^{\prime}$ in $C^{\prime}$. Hence, the new colouring $\chi^{\prime}:V\times V\to C^{\prime}$ will assign $(v_{1}^{\prime},v_{2}^{\prime})$ the colour $c^{\prime}$, corresponding to $\mathsf{L}^{2}(v_{1},v_{2})$, when $\mathsf{L}^{2}(v_{1},v_{2})=\mathsf{L}^{2}(v_{1}^{\prime},v_{2}^{\prime})$. It is easily verified that the partition $\Pi_{\chi^{\prime}}(G)$ is a refinement of $\Pi_{\chi}(G)$, which in turn is a refinement of $\Pi_{\chi_{0}}(G)$.

Algorithm 2-Stab now replaces $\chi$ by $\chi^{\prime}$ and $C$ by $C^{\prime}$ (lines 9 and 10), and repeats this process until the number of colours remains fixed (line 11). In other words, the corresponding partition is not further refined. The algorithm then returns this final (stable) colouring.

The stable edge partition of $G$, denoted by $\Pi(G)$, is now the partition induced by the stable colouring. It is known that $\Pi(G)$ is the unique coarsest partition of $V\times V$ which refines $\Pi_{\chi_{0}}(G)$ and corresponding to a colouring satisfying the stability condition stated on lines 7 and 8 in Algorithm 2.

Two graphs $G=(V,E)$ and $H=(W,F)$ of the same order are now said to be indistinguishable by the 2WL algorithm, denoted by $G\equiv_{\mathsf{2WL}}H$, if the stable edge partitions $\Pi(G)$ and $\Pi(H)$ of $G$ and $H$, respectively, are (i) of the form $\Pi(G)=\{E_{c_{1}},\ldots,E_{c_{\ell}}\}$ and $\Pi(H)=\{F_{c_{1}},\ldots,F_{c_{\ell}}\}$, that is, the parts in the partitions correspond to the same colours; and (ii) the corresponding parts in these partitions have the same size, that is, $E_{c_{i}}$ and $F_{c_{i}}$ have the same number of entries carrying the value $1$, for $i=1,\ldots,\ell$.

In the seminal paper by Cai, Fürer and Immerman [15], the connection with logical indistinguishability was made.

Theorem 9.2**.**

Let $G$ and $H$ be two graphs of the same order. Then, $G\equiv_{\mathsf{2WL}}H$ if and only if $G\equiv_{\mathsf{C}^{3}}H$.   ∎

We next show that $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot)$-equivalence implies indistinguishability by the 2WL algorithm.

Proposition 9.1**.**

Let $G$ and $H$ be graphs of the same order. Then, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot)}H$ implies that $G\equiv_{\mathsf{2WL}}H$.

Proof.

The overall proof is similar (both in terms of structure as strategy) to the proof of Proposition 7.2, but using indicator matrices (representing the edge partitions) instead of indicator vectors (which represented the vertex partitions), and by relying on the algorithm 2-Stab to compute the stable edge partition of a graph instead of algorithm GDCR (which computed the coarsest equitable partition). First, to simplify the construction of the expressions later on, we allow for addition and scalar multiplication. It can be verified (see appendix) that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot)}H$ implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot,+,\times)}H$. We may thus indeed consider $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot,+,\times)$ from now on.

The proof consists of the following two steps:

• (a)

We first construct a number of expressions in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot,+,\times)$, denoted by $\mathsf{stabcol}_{c}(X)$, for $c\in C$ and $C$ a set of colours. The key property of these expressions is that when they are evaluated on the adjacency matrix $A_{G}$ of $G$, $\mathsf{stabcol}_{c}(A_{G})$, for $c\in C$, correspond to indicator matrices representing the stable edge partition of $G$.

• (b)

The construction of the expressions $\mathsf{stabcol}_{c}(X)$, for $c\in C$, depend on $A_{G}$. As such, it is not guaranteed that $\mathsf{stabcol}_{c}(A_{H})$, for $c\in C$, correspond to indicator matrices representing the stable edge partition of $H$. We show, however, when $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot,+,\times)}H$. holds, then $\mathsf{stabcol}_{c}(A_{H})$, for $c\in C$, indeed correspond to indicator matrices representing the stable edge partition of $H$. To show this, we construct a number of sentences in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot,+,\times)$. Along the way, based on the construction of the expressions $\mathsf{stabcol}_{c}(X)$, for $c\in C$, we show that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot,+,\times)}H$ implies that $G\equiv_{\mathsf{2WL}}H$ holds.

(a) Compute the stable edge partition of a graph. Given $G=(V,E)$, let $\Pi(G)=\{E_{c_{1}},\ldots,E_{c_{\ell}}\}$ be its stable edge partition. We show that we can construct expressions $\mathsf{stabcol}_{c_{i}}(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\allowbreak\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot,+,\times)$, such that $E_{c_{i}}=\mathsf{stabcol}_{c_{i}}(A_{G})$, for $i=1,\ldots,\ell$. The expressions are constructed by simulating the run of the algorithm 2-Stab on $A_{G}$.

The initialisation step of 2-Stab is easy to simulate in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot,+,\times)$. Indeed, we simply consider expressions $\mathsf{stabcol}_{2}^{(0)}(X):=\operatorname{\mathsf{diag}}(\mathbb{1}(X))$; $\mathsf{stabcol}_{1}^{(0)}(X):=X$; and finally, $\mathsf{stabcol}_{0}^{(0)}(X):=\mathbb{1}(X)\mskip 2.0mu{\cdot}\mskip 2.0mu(\mathbb{1}(X))^{*}-X-\operatorname{\mathsf{diag}}(\mathbb{1}(X))$. Then, the indicator matrices $\mathsf{stabcol}_{0}^{(0)}(A_{G})$, $\mathsf{stabcol}_{1}^{(0)}(A_{G})$, and $\mathsf{stabcol}_{2}^{(0)}(A_{G})$ represent the initial partition $\Pi_{\chi_{0}}(G)=\{E_{0},E_{1},E_{2}\}$ corresponding to the initial colouring $\chi_{0}$.

Suppose now that after iteration $i$, the current set of colours is $C$ and the colouring is $\chi:V\times V\to C$. Assume, by induction, that we have expressions $\mathsf{stabcol}_{c}^{(i)}(X)$ in $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},+,\times,\odot)$, one for each $c\in C$, such that $\mathsf{stabcol}_{c}^{(i)}(A_{G})$ is an indicator matrix representing the part in the edge partition $\Pi_{\chi}(G)$, induced by $\chi$, for colour $c$. Given these, we next construct expressions for the refined partition computed by 2-Stab in the next iteration.

First, for each pair of colours $(c,d)$ in $C$, we consider the expression

[TABLE]

On input $A_{G}$, it is readily verified that $P_{c,d}^{(i+1)}(A_{G})$ is a matrix whose entry corresponding to vertices $v_{1}$ and $v_{2}$ holds the value $p_{v_{1},v_{2}}^{c,d}$.

Let ${\cal P}_{c,d}^{(i+1)}$ be the set of numbers occurring in $P_{c,d}^{(i+1)}(A_{G})$. For each value $p$ in ${\cal P}_{c,d}^{(i+1)}$, we now extract an indicator matrix indicating the positions in $P_{c,d}^{(i+1)}(A_{G})$ that hold value $p$.

This can be done using an expression $\mathsf{ind}_{c,d,p}^{(i+1)}(X)$ which works in a similar way as $\#\mathsf{3deg}(X)$ in Example 7.1, but uses the Schur-Hadamard product instead of products of diagonal matrices. The following example illustrates the underlying idea (see also the Schur-Wielandt Principle [61] mentioned before).

Example 9.3**.**

Consider $P_{c,d}=\begin{pmatrix}2&0&3\\ 1&3&2\\ 0&2&3\end{pmatrix}$ with ${\cal P}_{c,d}=\{0,1,2,3\}$. Suppose that we want to find all entries holding value $3$. This can be computed, as follows:

[TABLE]

where $\frac{1}{6}=\frac{1}{3(3-1)(3-2)}$, just as in Example 7.1.  ∎

More generally, to identify positions that hold a specific value in $P_{c,d}^{(i+1)}(A_{G})$, we consider the expression $\mathsf{ind}^{(i+1)}_{c,d,p}(X)$ defined by

[TABLE]

It should be clear from Example 9.3 that $\mathsf{ind}^{(i+1)}_{c,d,p}(A_{G})$ indeed results in the desired indicator matrix. We note that the expression $\mathsf{ind}^{(i+1)}_{c,d,p}(X)$ depends on the values in ${\cal P}^{(i+1)}_{c,d}$ and hence also depends on $A_{G}$.

Let $C^{\prime}$ be the new set of colours assigned by 2-Stab$(G)$ during the current iteration. As mentioned earlier, each colour $c$ in $C^{\prime}$ is in correspondence with $\mathsf{L}^{2}(v_{1},v_{2})$ for some vertices $v_{1}$ and $v_{2}$. Let us pick a colour $c$ in $C^{\prime}$ and assume that it corresponds to

[TABLE]

We next use $\mathsf{ind}^{(i+1)}_{c,d,p}(X)$ and the Schur-Hadamard product to identify all vertex pairs that are assigned colour $c$, as follows:

[TABLE]

In other words, we use the Schur-Hadamard product to simulate the “conjunction” of the binary matrices representing the vertex pairs $(v_{1},v_{2})$ having non-zero structure constants $p_{v_{1},v_{2}}^{c_{i},d_{i}}$, for $i=1,\ldots,s$. It is now easily verified that, on input $A_{G}$, $\mathsf{stabcol}_{c}^{(i+1)}(A_{G})$ returns an indicator matrix in which the entries holding a $1$ correspond precisely to the pairs $(v_{1}^{\prime},v_{2}^{\prime})\in V\times V$ such that $\mathsf{L}^{2}(v_{1}^{\prime},v_{2}^{\prime})=\mathsf{L}^{2}(v_{1},v_{2})$ where $\mathsf{L}^{2}(v_{1},v_{2})$ corresponds to colour $c$. In other words, $\mathsf{stabcol}_{c}^{(i+1)}(A_{G})$ represents the refined edge partition corresponding to the part associated with colour $c$. We do this for every colour in $C^{\prime}$. Clearly, $\mathsf{stabcol}_{c}^{(i+1)}(A_{G})$, for $c\in C^{\prime}$, represent the refined partition $\Pi_{\chi^{\prime}}(G)$ corresponding to $\chi^{\prime}:V\times V\to C^{\prime}$.

We continue in this way until the colouring stabilises. i.e., no further colours are needed. We denote the final set of colours by $C$ and by $\mathsf{stabcol}_{c}(X)$, for $c\in C$, the $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\allowbreak\operatorname{\mathsf{diag}},\odot,+,\allowbreak\times)$ expressions computing the parts $E_{c}$ in $\Pi(G)$. The correctness of these expressions follows from the previous arguments and the correctness of the algorithm 2-Stab.

(b) Verifying that $G\equiv_{\mathsf{2WL}}H$.

Just as in the proof of Proposition 7.2, the expressions $\mathsf{stabcol}_{c}(X)$ depend on $A_{G}$ since we explicitly used the values occurring in $P^{(i)}_{c,d}(A_{G})$ and the colours assigned to vertex pairs during each iteration $i$ of the execution of 2-Stab on $G$. Let $\Pi(H)$ be stable edge partition of $H$. We next show that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot,+,\times)}H$ implies that $\Pi(H)$ consists of $\mathsf{stabcol}_{c}(A_{H})$, for $c\in C$. Furthermore, we show that the number of ones in $\mathsf{stabcol}_{c}(A_{G})$ and $\mathsf{stabcol}_{c}(A_{H})$ agree for all $c\in C$. Hence, $G$ and $H$ are indistinguishable by the 2WL algorithm.

The proof is by induction on the number of iterations of $\textsc{2-Stab}(G)$ and $\textsc{2-Stab}(H)$. We denote by $\chi_{G}^{(i)}:V\times V\to C^{(i)}_{G}$ and $\chi_{H}^{(i)}:W\times W\to C^{(i)}_{H}$ the colourings used in the $i^{\text{th}}$ iteration of $\textsc{2-Stab}(G)$ and $\textsc{2-Stab}(H)$, respectively. The induction hypothesis is that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot,+,\times)}H$ implies that $C^{(i)}_{G}=C^{(i)}_{H}=C^{(i)}$ and furthermore that for each $c\in C^{(i)}$, $\mathsf{stabcol}_{c}^{(i)}(A_{H})$ is an indicator matrix, and all $\mathsf{stabcol}_{c}^{(i)}(A_{H})$ together constitute the edge partition $\Pi_{\chi^{(i)}_{H}}(H)$. Moreover, we show that for each $c\in C^{(i)}$, $\mathsf{stabcol}_{c}^{(i)}(A_{G})$ and $\mathsf{stabcol}_{c}^{(i)}(A_{H})$ have the same number of ones. This clearly suffices, for if this holds, $\mathsf{stabcol}_{c}(A_{H})$, for $c\in C$, constitute $\Pi(H)$ and $\mathsf{stabcol}_{c}(A_{G})$ and $\mathsf{stabcol}_{c}(A_{H})$ have the same number of ones, for all $c\in C$.

We start by verifying the hypothesis for the base case, i.e., when $i=0$. Clearly, $\chi_{G}^{(0)}$ and $\chi_{H}^{(0)}$ use the same colours $C^{(0)}_{G}=C^{(0)}_{H}=C^{(0)}=\{0,1,2\}$. By definition of the expressions $\mathsf{stabcol}_{c}^{(0)}(X)$, all $\mathsf{stabcol}_{c}^{(0)}(A_{H})$ together represent $\Pi_{\chi^{(0)}_{H}}(H)$. Moreover, by considering the sentences

[TABLE]

for $c\in C^{(0)}$, $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot,+,\times)}H$ implies that $\mathsf{\#ones}^{(0)}_{c}(A_{G})=\mathsf{\#ones}^{(0)}_{c}(A_{H})$. Hence, we may conclude that $\mathsf{stabcol}_{c}^{(0)}(A_{G})$ and $\mathsf{stabcol}_{c}^{(0)}(A_{H})$ have the same number of ones, as desired.

Suppose, by induction, that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot,+,\times)}H$ implies that $\chi_{G}^{(i)}:V\times V\to C^{(i)}_{G}$ and $\chi_{H}^{(i)}:W\times W\to C^{(i)}_{H}$ with $C^{(i)}_{G}=C^{(i)}_{H}=C^{(i)}$. Furthermore, the current edge partition $\Pi_{\chi^{(i)}_{H}}(H)$ of $H$ is represented by $\mathsf{stabcol}_{c}^{(i)}(A_{H})$, for $c\in C^{(i)}$. Furthermore, for each $c\in C^{(i)}$, the number of ones in $\mathsf{stabcol}^{(i)}_{c}(A_{H})$ and $\mathsf{stabcol}^{(i)}_{c}(A_{G})$ agree.

As before, let ${\cal P}^{(i+1)}_{c,d}$ be the set of values occurring in $P_{c,d}^{(i+1)}(A_{G})$ and consider the expressions $\mathsf{ind}^{(i+1)}_{c,d,p}(X)$ for $c,d\in C^{(i)}$ and $p\in{\cal P}^{(i+1)}_{c,d}$. We show that $\mathsf{ind}^{(i+1)}_{c,d,p}(A_{H})$ is a binary matrix as well containing the same number of ones as $\mathsf{ind}^{(i+1)}_{c,d,p}(A_{G})$. This implies that each value $p\in{\cal P}^{(i+1)}_{c,d}$ occurs in $P_{c,d}^{(i+1)}(A_{H})$ and moreover, it occurs the same number of times as in $P_{c,d}^{(i+1)}(A_{G})$. Hence, the set of values occurring in $P^{(i+1)}_{c,d}(A_{H})$ is the same as those occurring in $P_{c,d}^{(i+1)}(A_{G})$.

To check that $\mathsf{ind}^{(i+1)}_{c,d,p}(A_{H})$ is a binary matrix, we use the sentence

[TABLE]

This sentence will return $[0]$, when given a real matrix as input, if and only if the input matrix is a binary matrix. We have seen a similar expression in the proof of Proposition 7.2. Indeed, for a binary matrix $B$, $B\odot B=B$ and hence $B\odot B-B=Z$, where $Z$ is the zero matrix. Since $Z\odot Z=Z$, $\mathsf{binary}(B)=\mathbb{1}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0muZ\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}=[0]$. For the converse, assume that $\mathsf{binary}(B)=[0]$. We observe that each entry in $(B\odot B-B)\odot(B\odot B-B)$ is non-negative value. Indeed, all entries are squares of real numbers. Hence, when $\mathsf{binary}(B)=[0]$, the sum of all these squared entries must be zero. This implies that $B\odot B-B=Z$. This in turn implies that $B$ can only contain [math] or $1$ as entries, since these are the only real values satisfying $x^{2}-x=0$. Hence, when $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot,+,\times)}H$ holds, then since all $\mathsf{ind}_{c,d,p}^{(i+1)}(A_{G})$, for $c,d\in C^{(i)}$ and $p\in{\cal P}^{(i+1)}_{c,d}$, are binary matrices,

[TABLE]

So indeed, $\mathsf{ind}^{(i+1)}_{c,d,p}(A_{H})$ is a binary matrix as well.

The new colours in $\textsc{2-Stab}(G)$ are assigned based on the structure lists $\mathsf{L}^{2}(v_{1},v_{2})$. We show that for every unique structure list $\mathsf{L}^{2}(v_{1},v_{2})$ there is a pair of vertices $w_{1},w_{2}$ in $W$ such that $\mathsf{L}^{2}(v_{1},v_{2})=\mathsf{L}^{2}(w_{1},w_{2})$. This implies that $\textsc{2-Stab}(H)$ will use the same colours for refining $\chi_{H}^{(i)}$ as $\textsc{2-Stab}(G)$ uses to refine $\chi_{G}^{(i)}$. Hence, the revised colourings $\chi_{G}^{(i+1)}:V\times V\to C_{G}^{(i+1)}$ and $\chi_{H}^{(i+1)}:W\times W\to C_{H}^{(i+1)}$ satisfy indeed that $C_{G}^{(i+1)}=C_{H}^{(i+1)}=C^{(i+1)}$.

Consider a structure list $\mathsf{L}^{2}(v_{1},v_{2})$ and assume that it corresponds to a new colour $c\in C_{G}^{(i+1)}$. We know that $\mathsf{stabcol}_{c}^{(i+1)}(A_{G})$ returns the indicator matrix indicating which vertex pairs in $V\times V$ have this structure list (colour $c$). The expression $\mathsf{stabcol}_{c}^{(i+1)}(X)$ consists of the Schur-Hadamard product of $\mathsf{ind}^{(i+1)}_{c,d,p}(X)$ for every $(c,d,p)$ in $\mathsf{L}^{2}(v_{1},v_{2})$. We have shown above that $\mathsf{ind}^{(i+1)}_{c,d,p}(A_{G})$ and $\mathsf{ind}^{(i+1)}_{c,d,p}(A_{H})$ contain the same number of ones, meaning that there are vertex pairs $(w_{1},w_{2})\in W\times W$ for which $p_{w_{1},w_{2}}^{c,d}=p=p_{v_{1},v_{2}}^{c,d}$. Furthermore, in a similar way as above, we can show that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathsf{tr},\mathbb{1},\odot,+,\times)}H$ implies that $\mathsf{stabcol}_{c}^{(i+1)}(A_{H})$ is a binary matrix which consists of the same number of ones as $\mathsf{stabcol}_{c}^{(i+1)}(A_{G})$. So, $\textsc{2-Stab}(H)$ needs the same set of colours $C_{G}^{(i+1)}$ as $\textsc{2-Stab}(G)$ in the refinement phase. Hence, we can take $C_{G}^{(i+1)}=C_{H}^{(i+1)}=C^{(i+1)}$.

By construction, $\mathsf{stabcol}_{c}^{(i+1)}(A_{H})$ and $\mathsf{stabcol}_{c^{\prime}}^{(i+1)}(A_{H})$ do not have a common entry holding value $1$, for each distinct pair of colours $c,c^{\prime}\in C^{(i+1)}$. We note that the number of entries holding value $1$ in all $\mathsf{stabcol}_{c}^{(i+1}(A_{H})$ combined sum up $n^{2}$. Indeed, we know that this holds for $\mathsf{stabcol}_{c}^{(i+1)}(A_{G})$ and we have just shown that $\mathsf{stabcol}_{c}^{(i+1))}(A_{H})$ consists of the same number of ones as $\mathsf{stabcol}_{c}^{(i+1)}(A_{G})$. Hence, $\mathsf{stabcol}_{c}^{(i+1)}(A_{H})$ also represent a partition of $W\times W$, i.e., the partition $\Pi_{\chi_{H}^{(i+1)}}(H)$ and the induction hypothesis is satisfied. ∎

9.3 Main result

We are now ready to show the main result of this section.

Theorem 9.4**.**

Let $G$ and $H$ be two graphs of the same order, then $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot)}H$ if and only if $G\equiv_{\mathsf{MATLANG}}H$ if and only if $G\equiv_{\mathsf{C}^{3}}H$.

Proof.

We show that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot)}H$ implies $G\equiv_{\mathsf{C}^{3}}H$, and that $G\equiv_{\mathsf{C}^{3}}H$ implies $G\equiv_{\mathsf{MATLANG}}H$. Since $\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot)$ is a smaller fragment than $\mathsf{MATLANG}$, $G\equiv_{\mathsf{MATLANG}}H$ clearly implies $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot)}H$, resulting in the theorem.

Let us assume that $G\equiv_{\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,\,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot)}H$ holds. Then, the previous proposition implies that $G\equiv_{\mathsf{2WL}}H$. Combined with Theorem 9.2, this implies that $G\equiv_{\mathsf{C}^{3}}H$. Next, we assume that $G\equiv_{\mathsf{C}^{3}}H$ holds. We show that this implies that $G\equiv_{\mathsf{MATLANG}}H$. We here rely on the connection between $\mathsf{MATLANG}$ and three-variable logics [11], which we first recall.

In Proposition 4.2 in Brijder et al. [11] it was shown that for every sentence $e(X)$ in $\mathsf{MATLANG}$ there exists an equivalent formula $\varphi_{e}(z)$ in the relational calculus with aggregates which uses only three “base variables”. We will not recall the syntax of this calculus formally (see [54] for a full definition) but only recall that in this calculus, we have base variables and numerical variables. Base variables can be bound to base columns of relations, and compared for equality. Numerical variables can be bound to numerical columns, and can be equated to function applications and aggregates. The free variable $z$ in $\varphi_{e}(z)$ is a numeric variable since a scalar is returned by $e(X)$.

We now make the connection between matrices, on which $\mathsf{MATLANG}$ expressions are evaluated, and such typed relations, on which calculus expressions are evaluated. More specifically, a matrix $A$ is encoded as a ternary relation $\textsf{Rel}(A)$ where two base columns are reserved for the indices of the matrix and the numerical column holds the value in each entry (vectors and scalars are represented analogously). It is now understood that the equivalence of $e(X)$ and $\varphi_{e}(z)$ means that $e(A_{G})$ and the evaluation of $\varphi_{e}(z)$ on $\textsf{Rel}(A_{G})$ results in the same scalar. Let $c=e(A_{G})\in\mathbb{C}$ and consider the calculus sentence $\psi_{e}:=\exists z\,\varphi_{e}(z)\land z=c$. Following the arguments in the proof of Proposition 4.4 in [11], which in turn rely on standard translation techniques (see e.g., [44, 54]), one can show that $\psi_{e}$ can be equivalently expressed by a sentence $\psi_{e}^{\prime}$ in $C_{\infty\omega}^{3}$ [59], i.e., in infinitary counting logic with three distinct (untyped) variables over binary relations. These binary relations encode graphs in a standard way by simply storing the edge relation. It is known that $G\equiv_{C_{\infty\omega}^{3}}H$ if and only if $G\equiv_{\mathsf{C}^{3}}H$ [43]. By assumption $G\equiv_{\mathsf{C}^{3}}H$ and hence $G\equiv_{C_{\infty\omega}^{3}}H$. This implies that $\psi_{e}^{\prime}(G)=\psi_{e}^{\prime}(H)$ since $\psi_{e}^{\prime}$ is a sentence in $C_{\infty\omega}^{3}$. Hence, also $\psi_{e}$ evaluates to true on both $\mathsf{Rel}(A_{G})$ and $\mathsf{Rel}(A_{H})$, and $\varphi_{e}(z)$ returns the value $c$ on both $\mathsf{Rel}(A_{G})$ and $\mathsf{Rel}(A_{H})$. As a consequence, also $e(A_{H})=c$ and $e(A_{G})=e(A_{H})$. Since this argument works for any $\mathsf{MATLANG}$ sentence $e(X)$, we have that $G\equiv_{\mathsf{MATLANG}}H$. ∎

The results by Dell et al. [27] also tell that $G\equiv_{\mathsf{MATLANG}}H$ if and only if $\textsf{HOM}_{\cal F}(G)=\textsf{HOM}_{\cal F}(H)$ where ${\cal F}$ consists of all graphs of tree-width at most two.

We conclude by providing an algebraic characterisation of $\mathsf{MATLANG}$-equivalence based on a result by Dawar et al. [25]. To state this result, we need the notion of coherent algebra (see e.g., [30]). The coherent algebra $\mathfrak{C}(A_{G})$ associated with $A_{G}$ is the smallest complex matrix algebra containing $A_{G}$, $I$, and $J$ and which is closed under the Schur-Hadamard product. The coherent algebra $\mathfrak{C}(A_{H})$ associated with $A_{H}$ is defined similarly. The algebras $\mathfrak{C}(A_{G})$ and $\mathfrak{C}(A_{H})$ are said to be algebraically isomorphic if there is bijection $\imath:\mathfrak{C}(A_{G})\to\mathfrak{C}(A_{H})$ which is an algebra morphism which in addition satisfies: $\imath(J)=J$, $\imath(A^{*})=(\imath(A))^{*}$ and $\imath(A\odot B)=\imath(A)\odot\imath(B)$, for all matrices $A,B\in\mathfrak{C}(A_{G})$.

Proposition 9.2** (Proposition 7 in Dawar et al. [25]).**

Let $G$ and $H$ be two graphs of the same order. Then, $G\equiv_{\mathsf{C}^{3}}H$ if and only if there exists an algebraic isomorphism $\imath:\mathfrak{C}(A_{G})\to\mathfrak{C}(A_{H})$ such that $\imath(A_{G})=\imath(A_{H})$. ∎

This correspondence can be made a bit more precise and in line with our previous characterisations.

Proposition 9.3**.**

Let $G$ and $H$ be two graphs of the same order, then $G\equiv_{\mathsf{MATLANG}}H$ if and only if there exists an orthogonal matrix $O$ such that $E_{c}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muF_{c}$, for $c\in C$, where $E_{c}$ and $F_{c}$, for $c\in C$, constitute the stable edge partitions $\Pi(G)$ and $\Pi(H)$ of $G$ and $H$, respectively. (Here, $C$ denotes the set of colours used by the colourings that induce the partitions).

Proof.

We know from Proposition 9.1 that $G\equiv_{\mathsf{MATLANG}}H$ implies that $G\equiv_{\mathsf{2WL}}H$. Moreover, we can compute $\Pi(G)$ and $\Pi(H)$ by means of the expressions $\mathsf{stabcol}_{c}(X)$ in $\mathsf{MATLANG}$. Let $C=\{c_{1},\ldots,c_{\ell}\}$ be the set of colours used in these partitions. Just as in the proof of Theorem 7.4, we consider sentences e_{w}(X):=\mathsf{tr}\bigl{(}w(\mathsf{stabcol}_{c_{1}}(X),\ldots,\allowbreak\mathsf{stabcol}_{c_{\ell}}(X))\bigr{)} for some word $w$ over $\ell$ variables. Then, $G\equiv_{\mathsf{MATLANG}}H$ implies that $e_{w}(A_{G})=e_{w}(A_{H})$ for any such word $w$, and thus by the real version of Specht’s Theorem, there exists an orthogonal matrix $O$ such that $\mathsf{stabcol}_{c}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathsf{stabcol}_{c}(A_{H})$ for all $c\in C$, as desired. In the application of Specht’s Theorem it is crucial that $\Pi(G)$ and $\Pi(H)$ are closed under transposition. This known to hold, i.e., for every part $E_{c}$ in $\Pi(G)$ there is a part $E_{c^{\prime}}$ such that $E_{c}^{\mathsf{t}}=E_{c^{\prime}}$. This property also holds for $\Pi(H)$ (see e.g., [8]).

For the converse, suppose that there exists an orthogonal matrix $O$ such that $E_{c}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muF_{c}$, for $c\in C$. We note that this implies that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ since $A_{G}=\sum_{c\in D}E_{c}$ and $A_{H}=\sum_{c\in D}F_{c}$ for some subset of colours $D$ of $C$. This follows the fact that the 2WL algorithm refines the initial colouring, in which edges are coloured differently than non-edges. So, a colour used for an edge in $G$ can only be used for an edge in $H$, and vice versa. Moreover, it is known that the binary matrices in $\Pi(G)$ and $\Pi(H)$ form a basis for $\mathfrak{C}(A_{G})$ and $\mathfrak{C}(A_{H})$, respectively. If we now consider $\imath:\mathfrak{C}(A_{G})\to\mathfrak{C}(A_{H}):A\mapsto O\mskip 2.0mu{\cdot}\mskip 2.0muA\mskip 2.0mu{\cdot}\mskip 2.0muO^{\mathsf{t}}$, then this is known to be an algebraic isomorphism between $\mathfrak{C}(A_{G})$ and $\mathfrak{C}(A_{H})$ [30]. Hence, by Proposition 9.2, $G\equiv_{\mathsf{C}^{3}}H$ and thus also $G\equiv_{\mathsf{MATLANG}}H$ by Theorem 9.4.∎

Remark 9.5**.**

The orthogonal matrix $O$ in the statement of Proposition 9.3 can be taken to be compatible with the coarsest equitable partitions of $G$ and $H$, that witness that $G$ and $H$ have a common equitable partition. This is in agreement with Theorem 7.4. This follows from the fact that there is a subset $K$ of colours such that $I=\sum_{c\in K}E_{c}=\sum_{c\in K}F_{c}$ [8]. Furthermore, the diagonal matrices $E_{c}$, for $c\in K$, correspond to $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{c}})$ for the coarsest equitable partition ${\cal V}=\{V_{c}\mid c\in K\}$ of $G$. Similarly, for $c\in K$, $F_{c}=\operatorname{\mathsf{diag}}(\mathbb{1}_{W_{c}})$ correspond to the coarsest equitable partition ${\cal W}=\{W_{c}\mid c\in K\}$ of $H$ [8].

Remark 9.6**.**

The proof of Proposition 9.3 relied on results by Brijder et al. [11] and Dawar et al. [25] in which connections with $\mathsf{C}^{3}$-equivalence were made. A direct proof of Proposition 9.3 is possible. Indeed, it suffices to show that $O$-conjugation, for an orthogonal matrix $O$ such that $E_{c}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muF_{c}$ holds for each colour $c\in C$, is preserved by all operations in $\mathsf{MATLANG}$, including arbitrary pointwise functions on matrices. We do not detail this further in this paper in order to keep the paper of reasonably length (the proof consists of a long case analysis in which all previous conjugation-preserving conditions need to be verified in the context of stable edge partitions). The crucial ingredient in all this is that one can verify that for any expression $e(X)$ in $\mathsf{MATLANG}$, such that $e(A_{G})$ returns a matrix, we can write $e(A_{G})=\sum_{c\in C}a_{c}\times E_{c}$ and $e(A_{H})=\sum_{c\in C}a_{c}\times F_{c}$. This is generalization $\mathsf{ML}(\cal L)$-vectors being constant on equitable partitions, but now for $\mathsf{ML}(\cal L)$-matrices being constant on stable edge partitions. The ability to rewrite $e(A_{G})$ (and $e(A_{H})$) in terms of the indicator matrices allows to show, e.g., that $O$-conjugation is preserved by the Schur-Hadamard product and, more generally, by any pointwise function application on matrices.

10 Conclusion

We have characterised $\mathsf{ML}(\cal L)$-equivalence for undirected graphs and identified what additional distinguishing power each of the operations in $\mathsf{MATLANG}$ has. Some of the results generalise to directed graphs (with asymmetric adjacency matrices) or even arbitrary matrices. This is explored in an upcoming paper [34]. The extension to the case when queries can have multiple inputs is wide open.

Of interest may also be to connect $\mathsf{ML}(\cal L)$-equivalence to fragments of first-order logic (without counting). A possible line of attack could be to work over the boolean semiring instead of over the complex numbers (see Grohe and Otto [39] for a similar approach). More general semirings could open the way for modelling and querying labeled graphs using matrix query languages (see also [13]).

Another question is which additional linear algebra operations should be added to the matrix language $\mathsf{MATLANG}$ such that $\mathsf{C}^{k}$-equivalence can be captured, for $k\geq 4$. We refer to [4, 39, 57] characterisation of $\mathsf{C}^{k}$-equivalence in terms of solutions of linear systems of equations, which may serve as inspiration. Finally, connections between $\mathsf{MATLANG}$ and rank logics, as studied in the context of the descriptive complexity of linear algebra [23, 22, 24, 37, 40, 45], are worth exploring.

**Acknowledgement.**The author is grateful to Joeri Rammelaere (and his Python skills) for computing the numerical quantities of the example graphs used in this paper.

Proof of Lemma 5.1

Lemma 5.1**.**

Let $\mathsf{ML}(\cal L)$ be any matrix query language fragment and let $G$ and $H$ be graphs of the same order. Consider expressions $e_{1}(X)$ and $e_{2}(X)$ in $\mathsf{ML}(\cal L)$. If $e_{i}(A_{G})$ and $e_{i}(A_{H})$ are $T$-similar, for $i=1,2$, for an arbitrary matrix $T$, then $e_{1}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{G})$ is also $T$-similar to $e_{1}(A_{H})\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{H})$.

Proof.

To show this lemma, we distinguish between the following cases, depending on the dimensions of $e_{1}(A_{G})$ and $e_{2}(A_{G})$ (or equivalently, the dimensions of $e_{1}(A_{H})$ and $e_{2}(A_{H})$). Let $e(X):=e_{1}(X)\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(X)$. Let $n$ be the order of $G$ (and $H$).

• •

($\mathbf{n\times n,n\times n}$): $e_{1}(A_{G})$ and $e_{2}(A_{G})$ are of dimension $n\times n$. By assumption, $e_{1}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0muT=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{H})$ and $e_{2}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0muT=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{H})$. Hence,

[TABLE]

• •

($\mathbf{n\times n,n\times 1}$): $e_{1}(A_{G})$ is of dimension $n\times n$ and $e_{2}(A_{G})$ is of dimension $n\times 1$. By assumption, $e_{1}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0muT=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{H})$ and $e_{2}(A_{G})=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{H})$. Hence,

[TABLE]

• •

($\mathbf{n\times 1,1\times n}$): $e_{1}(A_{G})$ is of dimension $n\times 1$ and $e_{2}(A_{G})$ is of dimension $1\times n$. By assumption, $e_{1}(A_{G})=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{H})$ and $e_{2}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0muT=e_{2}(A_{H})$. Hence,

[TABLE]

• •

($\mathbf{n\times 1,1\times 1}$): $e_{1}(A_{G})$ is of dimension $n\times 1$ and $e_{2}(A_{G})$ is of dimension $1\times 1$. By assumption, $e_{1}(A_{G})=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{H})$ and $e_{2}(A_{G})=e_{2}(A_{H})$. Hence,

[TABLE]

• •

($\mathbf{1\times n,n\times n}$): $e_{1}(A_{G})$ is of dimension $1\times n$ and $e_{2}(A_{G})$ is of dimension $n\times n$. By assumption, $e_{1}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0muT=e_{1}(A_{H})$ and $e_{2}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0muT=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{H})$. Hence,

[TABLE]

• •

($\mathbf{1\times n,n\times 1}$): $e_{1}(A_{G})$ is of dimension $1\times n$ and $e_{2}(A_{G})$ is of dimension $n\times 1$. By assumption, $e_{1}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0muT=e_{1}(A_{H})$ and $e_{2}(A_{G})=T\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{H})$. Hence,

[TABLE]

• •

($\mathbf{1\times 1,1\times n}$): $e_{1}(A_{G})$ is of dimension $1\times 1$ and $e_{2}(A_{G})$ is of dimension $1\times n$. By assumption, $e_{1}(A_{G})=e_{1}(A_{H})$ and $e_{2}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0muT=e_{2}(A_{H})$. Hence,

[TABLE]

• •

($\mathbf{1\times 1,1\times 1}$): $e_{1}(A)$ and $e_{2}(A)$ are of dimension $1\times 1$. By assumption, $e_{1}(A_{G})=e_{1}(A_{H})$ and $e_{2}(A_{G})=e_{2}(A_{H})$. Hence, $e(A_{G})=e_{1}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{G})=e_{1}(A_{H})\mskip 2.0mu{\cdot}\mskip 2.0mu\allowbreak e_{2}(A_{H})=e(A_{H})$.

This concludes the proof. ∎

Continuation of the proofs of Proposition 7.2 and Theorems 8.3 and 9.1

In all three proofs we relied on the presence of addition and scalar multiplication to compute either equitable partitions or stable edge partitions. Since addition and scalar multiplication are used in the proofs before the proper conjugacy notion was identified, we cannot simply rely on Lemma 5.3. We therefore show that when ${\cal L}$ is either $\{\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathbb{1},\operatorname{\mathsf{diag}}\}$, $\{\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\operatorname{\mathsf{diag}}\}$ (for Proposition 7.2), $\{\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\odot_{v}\}$ (Theorem 8.3) or $\{\mskip 2.0mu{\cdot}\mskip 2.0mu,\mathsf{tr},{}^{*},\mathbb{1},\operatorname{\mathsf{diag}},\odot\}$ (Theorem 9.1), that $G\equiv_{\mathsf{ML}(\cal L)}H$ implies $G\equiv_{\mathsf{ML}({\cal L}^{+})}H$, where ${\cal L}^{+}={\cal L}\cup\{+,\times\}$.

We show this by verifying that any expression $e(X)$ in $\mathsf{ML}({\cal L}^{+})$ can be equivalently written as a linear combination of expressions in $\mathsf{ML}(\cal L)$. We denote equivalence by $\equiv$ and $e(X)\equiv e^{\prime}(X)$ means that $e(A)=e^{\prime}(A)$ for all matrices $A$. We verify our claim by induction on the structure of expressions.

(base case) Let $e(X):=X$. This already has the desired form.

(matrix multiplication) Let $e(X):=e_{1}(X)\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(X)$. By induction we have that $e_{1}(X)\equiv\sum_{i=1}^{p}a_{i}\times e_{i}^{1}(X)$ and $e_{2}(X)\equiv\sum_{i=1}^{q}b_{i}\times e_{i}^{2}(X)$. Hence, $e(X)\equiv\sum_{i=1}^{p}\sum_{j=1}^{q}(a_{i}\times b_{i})\times(e_{i}^{1}(X)\mskip 2.0mu{\cdot}\mskip 2.0mue_{j}^{2}(X)$).

(complex conjugate transposition) Let $e(X):=(e_{1}(X))^{*}$. By induction, $e_{1}(X)\equiv\sum_{i=1}^{p}a_{i}\times e_{i}^{1}(X)$. Then, $e(X)\equiv\sum_{i=1}^{p}\bar{a}_{i}\times(e_{i}(X))^{*}$.

(trace) Let $e(X):=\mathsf{tr}(e_{1}(X))$. By induction, $e_{1}(X)\equiv\sum_{i=1}^{p}a_{i}\times e_{i}^{1}(X)$. Then, $e(X)\equiv\sum_{i=1}^{p}a_{i}\times\mathsf{tr}(e_{i}(X))$.

(ones-vector) Let $e(X):=\mathbb{1}(e_{1}(X))$. By induction, $e_{1}(X)\equiv\sum_{i=1}^{p}a_{i}\times e_{i}^{1}(X)$. Then, $e(X)\equiv\mathbb{1}(e_{1}^{1}(X))$.

(diag) Let $e(X):=\operatorname{\mathsf{diag}}(e_{1}(X))$. By induction, $e_{1}(X)\equiv\sum_{i=1}^{p}a_{i}\times e_{i}^{1}(X)$. Then, $e(X)\equiv\sum_{i=1}^{p}a_{i}\times\operatorname{\mathsf{diag}}(e_{i}^{1}(X))$.

(pointwise vector product) Let $e(X):=e_{1}(X)\odot_{v}e_{2}(X)$. By induction we have that $e_{1}(X)\equiv\sum_{i=1}^{p}a_{i}\times e_{i}^{1}(X)$ and $e_{2}(X)\equiv\sum_{i=1}^{q}b_{i}\times e_{i}^{2}(X)$. Hence, $e(X)\equiv\sum_{i=1}^{p}\sum_{j=1}^{q}(a_{i}\times b_{i})\times(e_{i}^{1}(X)\odot_{v}e_{j}^{2}(X)$).

(Schur-Hadamard) (pointwise vector product) Let $e(X):=e_{1}(X)\odot e_{2}(X)$. By induction we have that $e_{1}(X)\equiv\sum_{i=1}^{p}a_{i}\times e_{i}^{1}(X)$ and $e_{2}(X)\equiv\sum_{i=1}^{q}b_{i}\times e_{i}^{2}(X)$. Hence, $e(X)\equiv\sum_{i=1}^{p}\sum_{j=1}^{q}(a_{i}\times b_{i})\times(e_{i}^{1}(X)\odot e_{j}^{2}(X)$).

This concludes the proof.∎

Proof of Proposition 7.4

Proposition 7.4**.**

$\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathsf{tr},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)$-vectors are constant on equitable partitions.

Proof.

Let ${\cal L}^{\#}$ denote $\{\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathsf{tr},\mathbb{1},\operatorname{\mathsf{diag}},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega\}$. Consider a graph $G$ of order $n$ with equitable partition ${\cal V}=\{V_{1},\ldots,V_{\ell}\}$. As before, let $\mathbb{1}_{V_{1}},\ldots,\mathbb{1}_{V_{\ell}}$ be the corresponding indicator vectors. We will show that for any expression $e(X)\in\mathsf{ML}({\cal L}^{\#})$ such that $e(A_{G})$ is an $n\times 1$-vector, $e(A_{G})$ can be uniquely written in the form $\sum_{i=1}^{\ell}a_{i}\times\mathbb{1}_{V_{i}}$ for scalars $a_{i}\in\mathbb{C}$.

We show, by induction on the structure of expressions in $\mathsf{ML}({\cal L}^{\#})$, that the following properties hold;

1. (a)

if $e(A_{G})$ returns an $n\times n$-matrix, then for any pair $i,j=1,\ldots,\ell$ there exists a scalars $a_{ij},b_{ij}\in\mathbb{C}$ such that

[TABLE] 2. (b)

if $e(A_{G})$ returns an $n\times 1$-vector, then for any $i=1,\ldots,\ell$, there exists a scalar $a_{i}\in C$ such that

[TABLE]

Clearly, if (b) holds for every $i=1,\ldots,\ell$, then, $e(A_{G})=\sum_{i=1}^{\ell}a_{i}\times\mathbb{1}_{V_{i}}$ because $I=\sum_{i=1}^{\ell}\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})$. We remark these properties can be seen as generalization of the known fact that the vector space spanned by indicator vectors of an equitable partition of $G$ is invariant under multiplication by $A_{G}$ (See e.g., Lemma 5.2 in [16]). That is, for any linear combination $v=\sum_{i=1}^{\ell}a_{i}\times\mathbb{1}_{V_{i}}$ we have that $A\mskip 2.0mu{\cdot}\mskip 2.0muv=\sum_{i=1}^{\ell}b_{i}\times\mathbb{1}_{V_{i}}$. In our setting, (a) and (b) imply that $e(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0muv$ is again a linear combination of indicator vectors, when $e(A_{G})$ returns an $n\times n$-matrix. We next verify properties (a) and (b). We often use that $I=\sum_{i=1}^{\ell}\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})$ and $\mathbb{1}=\sum_{i=1}^{\ell}\mathbb{1}_{V_{i}}$.

(base case) Let $e(X):=X$. The required property is simply a restatement of the being equitable. That is,

[TABLE]

for an arbitrary vertex $v\in V_{i}$. So, we can take $a_{ij}=\mathsf{deg}(v,V_{j})$. Similarly, because we $A_{G}$ is a symmetric matrix,

[TABLE]

for an arbitrary vertex $v\in V_{i}$. So, we can take $a_{ij}=\mathsf{deg}(v,V_{j})$.

Below, for condition (a) we only verify that $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{V_{j}}=a_{ij}\times\mathbb{1}_{V_{i}}$ holds. The verification of $\mathbb{1}_{V_{j}}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0mu\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})=b_{ij}\times\mathbb{1}_{V_{i}}^{\mathsf{t}}$ is entirely similar.

(multiplication) Let $e(X):=e_{1}(X)\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(X)$. We distinguish between a number of cases, depending on the dimensions of $e_{1}(A_{G})$ and $e_{2}(A_{G})$. We first check the cases when $e(A_{G})$ returns an $n\times n$-matrix and need to show that property (a) holds.

• •

($\mathbf{n\times n,n\times n}$): $e_{1}(A_{G})$ and $e_{2}(A_{G})$ are of dimension $n\times n$. By induction, $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{V_{j}}=a_{ij}\times\mathbb{1}_{V_{j}}$ and

$\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{V_{j}}=b_{ij}\times\mathbb{1}_{V_{i}}$. Then, $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{V_{j}}$ is equal to

[TABLE]

as desired.

• •

($\mathbf{n\times 1,1\times n}$): $e_{1}(A_{G})$ is of dimension $n\times 1$ and $e_{2}(A_{G})$ is of dimension $1\times n$. By induction we have that $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{G})=a_{i}\times\mathbb{1}_{V_{i}}$ and $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mu(e_{2}(A_{G}))^{\mathsf{t}}=b_{i}\times\mathbb{1}_{V_{i}}$. Hence, $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{V_{j}}$ is equal to

[TABLE]

as desired. Here we used that $\mathbb{1}_{V_{k}}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{V_{j}}$ is either [math], in case that $k\neq j$, or $|V_{j}|$ in case that $j=k$.

We next check that condition (b) holds when $e(A_{G})$ returns an $n\times 1$-vector.

• •

($\mathbf{n\times n,n\times 1}$): $e_{1}(A_{G})$ is of dimension $n\times n$ and $e_{2}(A_{G})$ is of dimension $n\times 1$. By induction, we have that $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{V_{j}}=a_{ij}\times\mathbb{1}_{V_{i}}$ and $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{G})=b_{i}\times\mathbb{1}_{V_{i}}$. Hence, $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue(A_{G})$ is equal to

[TABLE]

as desired.

• •

($\mathbf{n\times 1,1\times 1}$): $e_{1}(A_{G})$ is of dimension $n\times 1$ and $e_{2}(A_{G})$ is of dimension $1\times 1$. By induction we have that $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{G})=a_{i}\times\mathbb{1}_{V_{i}}$ and $e_{2}(A_{G})=b\in\mathbb{C}$. Hence,

[TABLE]

as desired.

(ones vector) $e(X):=\mathbb{1}(e_{1}(X))$. We only need to consider the case when $e_{1}(A_{G})$ is an $n\times n$-matrix or $n\times 1$-vector. In both cases, it suffices to observe that $\mathbb{1}=\sum_{i=1}^{\ell}\mathbb{1}_{V_{i}i}$. Indeed,

[TABLE]

(conjugate transpose) $e(X):=(e_{1}(X))^{*}$. If $e_{1}(A_{G})$ returns a $1\times n$-vector, then $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mu(e_{1}(A_{G}))^{\mathsf{t}}=a_{i}\times\mathbb{1}_{V_{i}}$. Hence, $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{G})=a_{i}^{*}\times\mathbb{1}_{V_{i}}$. If $e_{1}(A_{G})$ returns an $n\times n$-matrix, then by induction, $\mathbb{1}_{V_{j}}^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{G})\mskip 2.0mu{\cdot}\mskip 2.0mu\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})=b_{ij}\times\mathbb{1}_{V_{i}}^{\mathsf{t}}$. Hence,

[TABLE]

as desired.

(diag operation) $e(X):=\operatorname{\mathsf{diag}}(e_{1}(X))$ where $e_{1}(A_{G})$ is an $n\times 1$-vector. By induction, $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{G})=a_{i}\times\mathbb{1}_{V_{i}}$. Hence, in view of the linearity of the diagonal operation,

[TABLE]

since $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{k}})\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{V_{j}}$ is $\mathbb{1}_{V_{j}}$ when $k=j$ and the zero vector otherwise.

(addition) $e(X):=e_{1}(X)+e_{2}(X)$. Clearly, when condition (a) or (b) hold for $e_{1}(A_{G})$ and $e_{2}(A_{G})$, they remain to hold for $e(A_{G})$.

(scalar multiplication) $e(X):=a\times e_{1}(X)$. Clearly, when condition (a) or (b) hold for $e_{1}(A_{G})$, they remain to hold for $e(A_{G})$.

(trace) $e(X):=\mathsf{tr}(e_{1}(X))$. Such sub-expressions do not return matrices or vectors.

(pointwise function applications) $e(X):=\operatorname{\mathsf{apply}}_{\mathsf{s}}[f](e_{1}(X),\dots,e_{p}(X))$ where each $e_{i}(X)$ is a sentence. Again, such sub-expressions do not return matrices or vectors. ∎

Proof of Proposition 8.2

Proposition 8.2**.**

$\mathsf{ML}(\mskip 2.0mu{\cdot}\mskip 2.0mu,{}^{*},\mathsf{tr},\mathbb{1},\odot_{v},\operatorname{\mathsf{diag}},+,\times,\operatorname{\mathsf{apply}}_{\mathsf{s}}[f],f\in\Omega)$-vectors are constant on equitable partitions

Proof.

Given that we verified this property of all operations except for $\odot_{v}$ in the proof of Proposition 7.4, we only need to verify that $\odot_{v}$ can be added to the list of supported operations. We use the same induction hypotheses as in the proof of Proposition 7.4 and verify that these hypotheses remain to hold for $\odot_{v}$:

(pointwise vector multiplication) $e(X):=e_{1}(X)\odot_{v}e_{2}(X)$ where $e_{1}(X)$ and $e_{2}(X)$ return vectors. By induction we have that $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue_{1}(A_{G})=a_{i}\times\mathbb{1}_{V_{i}}$ and $\operatorname{\mathsf{diag}}(\mathbb{1}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mue_{2}(A_{G})=b_{i}\times\mathbb{1}_{V_{i}}$. As a consequence,

[TABLE]

because $\mathbb{1}_{V_{i}}\odot_{v}\mathbb{1}_{V_{j}})$ is either $\mathbb{1}_{V_{i}}$ when $i=j$, or the zero vector when $i\neq j$. ∎

Continuation of the proof of Theorem 8.3

In the proof in the main body of the paper we left open the verification that $(\mathbb{1}_{V_{i}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}^{\mathsf{t}}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0mu(\mathbb{1}_{W_{i}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}^{\mathsf{t}}_{W_{i}})$, for $i=1,\ldots,\ell$, implies that $O$ preserves the coarsest equitable partitions of $G$ and $H$. In particular, we need to verify that $\mathbb{1}_{V_{i}}=O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$, for $i=1,\ldots,\ell$. This can be easily shown, just as in the proof of Theorem 7.4 (based on Lemma 4 in Thüne [68]), in which we verified that $J\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muJ$ implies that $\mathbb{1}=O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}$.

First, we observe that $(\mathbb{1}_{V_{i}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}^{\mathsf{t}}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0muO\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}=\mathbb{1}_{V_{i}}\mskip 2.0mu{\cdot}\mskip 2.0mu(\mathbb{1}^{\mathsf{t}}_{V_{i}}\mskip 2.0mu{\cdot}\mskip 2.0muO\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}})=\alpha_{i}\times\mathbb{1}_{V_{i}}$ with $\alpha_{i}=\mathbb{1}^{\mathsf{t}}_{V_{i}}\mskip 2.0mu{\cdot}\mskip 2.0muO\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$ and $(\mathbb{1}_{V_{i}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}^{\mathsf{t}}_{V_{i}})\mskip 2.0mu{\cdot}\mskip 2.0muO\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}=O\mskip 2.0mu{\cdot}\mskip 2.0mu(\mathbb{1}_{W_{i}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}^{\mathsf{t}}_{W_{i}})\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}=(\mathbb{1}^{\mathsf{t}}_{W_{i}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}})\times O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$. In other words, $O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}=\frac{\alpha_{i}}{n_{i}}\times\mathbb{1}_{V_{i}}$ where $\mathbb{1}^{\mathsf{t}}_{W_{i}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}=|W_{i}|=n_{i}$. Furthermore, because $\mathbb{1}^{\mathsf{t}}_{V_{i}}\mskip 2.0mu{\cdot}\mskip 2.0muO^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$ is a scalar, $\mathbb{1}^{\mathsf{t}}_{W_{i}}\mskip 2.0mu{\cdot}\mskip 2.0muO^{\mathsf{t}}\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{V_{i}}=(\mathbb{1}^{\mathsf{t}}_{V_{i}}\mskip 2.0mu{\cdot}\mskip 2.0muO\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}})^{\mathsf{t}}=\mathbb{1}^{\mathsf{t}}_{V_{i}}\mskip 2.0mu{\cdot}\mskip 2.0muO\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}=\alpha_{i}$. We next show that $\alpha=\pm n_{i}$. Indeed, since $O$ is an orthogonal matrix

[TABLE]

and thus $\alpha_{i}^{2}=n_{i}^{2}$ or $\alpha_{i}=\pm n_{i}$. Hence, $O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}=\pm\mathbb{1}_{V_{i}}$. We note that $\mathbb{1}=\sum_{i=1}^{\ell}\mathbb{1}_{V_{i}}=\sum_{i=1}^{\ell}\mathbb{1}_{W_{i}}$. We now argue that either $\mathbb{1}_{V_{i}}=O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$ for all $i=1,\ldots,\ell$, or $-\mathbb{1}_{V_{i}}=O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$ for all $i=1,\ldots,\ell$. Indeed, suppose that we have $\mathbb{1}_{V_{i}}=O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$ for $i\in K\subset\{1,\ldots,\ell\}$ and $-\mathbb{1}_{V_{i}}=O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$ for $i\in\bar{K}=\{1,\ldots,\ell\}\setminus K$, for some non-empty subset $K$ of $\{1,\ldots,\ell\}$. Then \sum_{i\in K}\mathbb{1}_{V_{i}}=O\mskip 2.0mu{\cdot}\mskip 2.0mu\bigl{(}\sum_{i\in K}\mathbb{1}_{W_{i}}\bigr{)} and hence since $\sum_{i\in\bar{K}}\mathbb{1}_{V_{i}}=\mathbb{1}-\sum_{i\in K}\mathbb{1}_{V_{i}}$ and $\sum_{i\in\bar{K}}\mathbb{1}_{W_{i}}=\mathbb{1}-\sum_{i\in K}\mathbb{1}_{W_{i}}$,

[TABLE]

This contradicts that -\sum_{i\in\bar{K}}\mathbb{1}_{V_{i}}=O\mskip 2.0mu{\cdot}\mskip 2.0mu\bigl{(}\sum_{i\in\bar{K}}\mathbb{1}_{W_{i}}\bigr{)}. Hence, when $\mathbb{1}_{V_{i}}=O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$ for all $i=1,\ldots,\ell$, $O$ satisfies the desired property already. Otherwise, when $-\mathbb{1}_{V_{i}}=O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}$ for all $i=1,\ldots,\ell$, we simply replace $O$ by $(-1)\times O$ to obtain that $O\mskip 2.0mu{\cdot}\mskip 2.0mu\mathbb{1}_{W_{i}}=\mathbb{1}_{V_{i}}$. This rescaling does not impact that $A_{G}\mskip 2.0mu{\cdot}\mskip 2.0muO=O\mskip 2.0mu{\cdot}\mskip 2.0muA_{H}$ and we can thus indeed conclude that $O$ preserves the coarsest equitable partitions of $G$ and $H$. ∎