Incidence Networks for Geometric Deep Learning

TL;DR

This paper formalizes incidence tensors in geometric deep learning, analyzing their structure and introducing equivariant networks that operate on them, enabling efficient processing of complex structured data.

Contribution

It introduces a formal framework for incidence tensors, analyzes their structure, and develops equivariant networks with efficient pooling and broadcasting mechanisms.

Findings

Incidence tensors decompose into invariant subsets.

Decomposition leads to efficient equivariant linear maps.

Proposed networks handle complex structured data effectively.

Abstract

Sparse incidence tensors can represent a variety of structured data. For example, we may represent attributed graphs using their node-node, node-edge, or edge-edge incidence matrices. In higher dimensions, incidence tensors can represent simplicial complexes and polytopes. In this paper, we formalize incidence tensors, analyze their structure, and present the family of equivariant networks that operate on them. We show that any incidence tensor decomposes into invariant subsets. This decomposition, in turn, leads to a decomposition of the corresponding equivariant linear maps, for which we prove an efficient pooling-and-broadcasting implementation.

Tables

Table 1. Table 1 : The nine operators of the 𝒮 N subscript 𝒮 𝑁 \mathcal{S}_{N} -equivariant map 𝖶 X subscript 𝖶 𝑋 \mathsf{W}_{X} for an undirected graph. An operator 𝖶 i m → m ′ subscript superscript 𝖶 → 𝑚 superscript 𝑚 ′ 𝑖 \mathsf{W}^{m\to m^{\prime}}_{i} maps faces of size m 𝑚 m to faces of size m ′ superscript 𝑚 ′ m^{\prime} (with m , m ′ = 1 𝑚 superscript 𝑚 ′ 1 m,m^{\prime}=1 for nodes and m , m ′ = 2 𝑚 superscript 𝑚 ′ 2 m,m^{\prime}=2 for edges), and with i 𝑖 i representing the size of partially pooled faces. Rows represent the pooled features, and columns targets each row is being broadcasted to. Partially-Pooled Edges corresponds to pooling the 𝐗 𝐗 {\mathbf{X}} (ignoring its diagonal) either across rows or columns. Pooled Nodes and Pooled Edges corresponds to pooling over all node and edge features, respectively.

Pooled Features / Broadcast to	Nodes	Edges
Edges	-	${𝖶}_2^{2\to 2}$
Nodes	${𝖶}_1^{1\to 1}$	${𝖶}_1^{1\to 2}$
Partially-Pooled Edges	${𝖶}_1^{2\to 2}$	${𝖶}_1^{2\to 2}$
Pooled Nodes	${𝖶}_0^{1\to 1}$	${𝖶}_0^{1\to 2}$
Pooled Edges	${𝖶}_0^{2\to 1}$	${𝖶}_0^{2\to 2}$

Equations

$$\varphi (\pi \cdot x)=\pi \cdot \varphi (x),\forall x\in \mathbb{X},\forall \pi \in \mathcal{G}.$$

$$\mathsf{W}(\pi \mathbf{X}{\pi }^{\top })=\pi \mathsf{W}(\mathbf{X}){\pi }^{\top },\forall \pi \in {\mathcal{S}}_N,\forall \mathbf{X}.$$

$$\mathbf{X}\cong {\mathbf{X}}_{\{{\delta }_1\}}\text{cupdot}{\mathbf{X}}_{\{{\delta }_1,{\delta }_2\}},$$

$$\mathsf{W}(\mathbf{X})={\text{bigcupdot}}_{m^{\prime }=1}^2\left({\mathsf{W}}^{1\to m^{\prime }}({\mathbf{X}}_{\{{\delta }_1\}})+{\mathsf{W}}^{2\to m^{\prime }}({\mathbf{X}}_{\{{\delta }_1,{\delta }_2\}})\right),$$

$${\mathsf{W}}_{\text{sp}}:\mathbf{X}\mapsto \mathsf{W}(\mathbf{X})\circ s(\mathbf{X}),$$

$$({\mathbf{X}}_{\pi \cdot {\mathbf{\delta }}_1,\dots ,\pi \cdot {\mathbf{\delta }}_D}\ne 0\iff {\mathbf{X}}_{{\mathbf{\delta }}_1,\dots ,{\mathbf{\delta }}_D}\ne 0)\iff \pi \in \mathcal{A}ut(\mathbf{X})$$

$$\pi \cdot ({\delta }_1,\dots ,{\delta }_M)=(\pi \cdot {\delta }_1,\dots ,\pi \cdot {\delta }_M).$$

$$({\mathbf{X}}_{{\mathbf{\delta }}_1,\dots ,{\mathbf{\delta }}_D},\Sigma )\cong {\text{bigcupdot}}_m{\kappa }_m{\mathbf{X}}_{{\mathbf{\delta }}^{(m)}},$$

$${\mathsf{W}}^{M\to M^{\prime }}:{\mathbb{R}}^{\stackrel{M}{\overbracket{N\times N\times \cdots \times N}}}$$

$${\mathbf{X}}_{({\delta }_1,\dots ,{\delta }_M)}$$

$${\mathsf{W}}_{\pi \cdot {\delta }_1^{\prime }\dots \pi \cdot {\delta }_{M^{\prime }}^{\prime }}^{\pi \cdot {\delta }_1,\dots ,\pi \cdot {\delta }_M}={\mathsf{W}}_{{\delta }_1^{\prime }\dots {\delta }_{M^{\prime }}^{\prime }}^{{\delta }_1\dots {\delta }_M}\forall \pi \in {\mathcal{S}}_N,$$

$${\mathrm{Pool}}_{\mathbb{P}}({\mathbf{X}}_{({\delta }_1\dots {\delta }_M)})=\sum _{{\delta }_{p_1}\in [N]}\dots \sum _{{\delta }_{p_L}\in [N]}{\mathbf{X}}_{({\delta }_1,\dots ,{\delta }_M)},$$

$$\displaystyle\big{(}\operatorname{Bcast}_{\mathbb{B},M^{\prime}}({\mathbf{X}})\big{)}_{(\delta_{1},\ldots,\delta_{M^{\prime}})}={\mathbf{X}}_{(\delta_{b_{1}},\ldots,\delta_{b_{M}})}.$$

$$\mathsf{W}^{M\to M^{\prime}}({\mathbf{X}})=\sum_{\begin{subarray}{c}\mathbb{P}\subseteq[M]\\ \mathbb{B}\subseteq\langle{1,\ldots,M^{\prime}}\rangle\\ |\mathbb{B}|=M-|\mathbb{P}|\end{subarray}}{\mathsf{w}}_{\mathbb{B},\mathbb{P}}\operatorname{Bcast}_{\mathbb{B},M^{\prime}}\big{(}\operatorname{Pool}_{\mathbb{P}}({\mathbf{X}})\big{)}.$$

$${\tau }^{M\to M^{\prime }}=\sum _{m=0}^{\min (M,M^{\prime })}\left(\genfrac{}{}{0ex}{}{M}{m}\right)\left(\genfrac{}{}{0ex}{}{M^{\prime }}{m}\right)m!.$$

$$\mathsf{W}({\mathbf{X}}_{{\mathbf{\delta }}_1,\dots ,{\mathbf{\delta }}_D},\Sigma )\cong {\text{bigcupdot}}_{m^{\prime }}{\text{bigcupdot}}_{k^{\prime }=1}^{{\kappa }_{m^{\prime }}^{\prime }}\sum _m\sum _{k=1}^{{\kappa }_m}{\mathsf{W}}^{k,m\to k^{\prime },m^{\prime }}({\mathbf{X}}_{{\mathbf{\delta }}^{(m)}}),$$

$$\tau =\sum _{m,m^{\prime }}{\kappa }_{m^{\prime }}^{\prime }{\kappa }_m{\tau }^{m\to m^{\prime }},$$

$${\mathbf{X}}_{{\mathbf{\delta }}_1,\dots ,{\mathbf{\delta }}_D}\cong {\text{bigcupdot}}_m\left\{\genfrac{}{}{0ex}{}{D}{m}\right\}{\mathbf{X}}_{{\mathbf{\delta }}^{(m)}},$$

$$\tau$$

$$\displaystyle=\sum_{l=0}^{D}\sum_{m=l}^{D}\sum_{m^{\prime}=l}^{D}\bigg{[}\binom{m}{l}\genfrac{\{}{\}}{0.0pt}{}{D}{m}\bigg{]}\bigg{[}\binom{m^{\prime}}{l}\genfrac{\{}{\}}{0.0pt}{}{D}{m^{\prime}}\bigg{]}l!=\mathsf{Bell}(2D).$$

$${\overline{\mathsf{W}}}_Y(\mathbf{Y})=\sum _{\mathbb{P}\in 2^{\{0,1\}}}{\lambda }_{\mathbb{P}}{\mathrm{Bcast}}_{\{0,1\}-\mathbb{P}}\left({\mathrm{Pool}}_{\mathbb{P}}\left(\mathbf{Y}\right)\right),$$

$$\begin{split}&{\mathbf{Y}}=\,\bordermatrix{&\scriptstyle{\{12\}}&\scriptstyle{\{13\}}&\scriptstyle{\{23\}}\cr\scriptstyle{1}&(\leavevmode\hbox to13.48pt{\vbox to13.48pt{\pgfpicture\makeatletter\hbox{\hskip 6.7393pt\lower-5.48875pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{{}\pgfsys@moveto{6.5393pt}{1.25055pt}\pgfsys@curveto{6.5393pt}{4.86215pt}{3.6116pt}{7.78986pt}{0.0pt}{7.78986pt}\pgfsys@curveto{-3.6116pt}{7.78986pt}{-6.5393pt}{4.86215pt}{-6.5393pt}{1.25055pt}\pgfsys@curveto{-6.5393pt}{-2.36105pt}{-3.6116pt}{-5.28876pt}{0.0pt}{-5.28876pt}\pgfsys@curveto{3.6116pt}{-5.28876pt}{6.5393pt}{-2.36105pt}{6.5393pt}{1.25055pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{1.25055pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.40117pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$n_{1}$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}},\boxed{e_{12}})&(n_{1},\boxed{e_{13}})&-\\[8.5359pt] \cr\scriptstyle{2}&(\leavevmode\hbox to13.48pt{\vbox to13.48pt{\pgfpicture\makeatletter\hbox{\hskip 6.7393pt\lower-5.48875pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{{}\pgfsys@moveto{6.5393pt}{1.25055pt}\pgfsys@curveto{6.5393pt}{4.86215pt}{3.6116pt}{7.78986pt}{0.0pt}{7.78986pt}\pgfsys@curveto{-3.6116pt}{7.78986pt}{-6.5393pt}{4.86215pt}{-6.5393pt}{1.25055pt}\pgfsys@curveto{-6.5393pt}{-2.36105pt}{-3.6116pt}{-5.28876pt}{0.0pt}{-5.28876pt}\pgfsys@curveto{3.6116pt}{-5.28876pt}{6.5393pt}{-2.36105pt}{6.5393pt}{1.25055pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{1.25055pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.40117pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$n_{2}$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}},e_{12})&-&(n_{2},\boxed{e_{23}})\\[8.5359pt] \cr\scriptstyle{3}&-&(\leavevmode\hbox to13.48pt{\vbox to13.48pt{\pgfpicture\makeatletter\hbox{\hskip 6.7393pt\lower-5.48875pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{{}\pgfsys@moveto{6.5393pt}{1.25055pt}\pgfsys@curveto{6.5393pt}{4.86215pt}{3.6116pt}{7.78986pt}{0.0pt}{7.78986pt}\pgfsys@curveto{-3.6116pt}{7.78986pt}{-6.5393pt}{4.86215pt}{-6.5393pt}{1.25055pt}\pgfsys@curveto{-6.5393pt}{-2.36105pt}{-3.6116pt}{-5.28876pt}{0.0pt}{-5.28876pt}\pgfsys@curveto{3.6116pt}{-5.28876pt}{6.5393pt}{-2.36105pt}{6.5393pt}{1.25055pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{1.25055pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.40117pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$n_{3}$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}},e_{13})&(n_{3},e_{23})}\\[5.69046pt] &\parbox{100.00015pt}{$\boxed{\phantom{e}}$ = edge features\\ $\leavevmode\hbox to10.19pt{\vbox to10.19pt{\pgfpicture\makeatletter\hbox{\hskip 5.09477pt\lower-2.942pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{{}}}{{}}{}{}{}{}{}{}{}{}{}{{}\pgfsys@moveto{4.89478pt}{2.15277pt}\pgfsys@curveto{4.89478pt}{4.85611pt}{2.70334pt}{7.04755pt}{0.0pt}{7.04755pt}\pgfsys@curveto{-2.70334pt}{7.04755pt}{-4.89478pt}{4.85611pt}{-4.89478pt}{2.15277pt}\pgfsys@curveto{-4.89478pt}{-0.55057pt}{-2.70334pt}{-2.742pt}{0.0pt}{-2.742pt}\pgfsys@curveto{2.70334pt}{-2.742pt}{4.89478pt}{-0.55057pt}{4.89478pt}{2.15277pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{2.15277pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.00117pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$\phantom{n}$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}$ = node features,}\end{split}$$

$$\bordermatrix{&\scriptstyle{\{12\}}&\scriptstyle{\{13\}}&\scriptstyle{\{23\}}\cr\scriptstyle{1}&(2n_{1},\leavevmode\hbox to43.55pt{\vbox to12.4pt{\pgfpicture\makeatletter\hbox{\hskip 21.77562pt\lower-4.18556pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{}}{} {}{}{}{}{}{}{}{}{}{{}\pgfsys@moveto{21.57562pt}{2.01443pt}\pgfsys@curveto{21.57562pt}{5.32819pt}{11.91603pt}{8.01443pt}{0.0pt}{8.01443pt}\pgfsys@curveto{-11.91603pt}{8.01443pt}{-21.57562pt}{5.32819pt}{-21.57562pt}{2.01443pt}\pgfsys@curveto{-21.57562pt}{-1.29932pt}{-11.91603pt}{-3.98557pt}{0.0pt}{-3.98557pt}\pgfsys@curveto{11.91603pt}{-3.98557pt}{21.57562pt}{-1.29932pt}{21.57562pt}{2.01443pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{2.01443pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-15.25626pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$e_{12}+e_{13}$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}})&(2n_{1},e_{12}+e_{13})&-\\[8.5359pt] \cr\scriptstyle{2}&(2n_{2},\leavevmode\hbox to43.55pt{\vbox to12.4pt{\pgfpicture\makeatletter\hbox{\hskip 21.77562pt\lower-4.18556pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{}}{} {}{}{}{}{}{}{}{}{}{{}\pgfsys@moveto{21.57562pt}{2.01443pt}\pgfsys@curveto{21.57562pt}{5.32819pt}{11.91603pt}{8.01443pt}{0.0pt}{8.01443pt}\pgfsys@curveto{-11.91603pt}{8.01443pt}{-21.57562pt}{5.32819pt}{-21.57562pt}{2.01443pt}\pgfsys@curveto{-21.57562pt}{-1.29932pt}{-11.91603pt}{-3.98557pt}{0.0pt}{-3.98557pt}\pgfsys@curveto{11.91603pt}{-3.98557pt}{21.57562pt}{-1.29932pt}{21.57562pt}{2.01443pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{2.01443pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-15.25626pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$e_{12}+e_{23}$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}})&-&(2n_{2},e_{12}+e_{23})\\[8.5359pt] \cr\scriptstyle{3}&-&(2n_{3},\leavevmode\hbox to43.55pt{\vbox to12.4pt{\pgfpicture\makeatletter\hbox{\hskip 21.77562pt\lower-4.18556pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{}}{} {}{}{}{}{}{}{}{}{}{{}\pgfsys@moveto{21.57562pt}{2.01443pt}\pgfsys@curveto{21.57562pt}{5.32819pt}{11.91603pt}{8.01443pt}{0.0pt}{8.01443pt}\pgfsys@curveto{-11.91603pt}{8.01443pt}{-21.57562pt}{5.32819pt}{-21.57562pt}{2.01443pt}\pgfsys@curveto{-21.57562pt}{-1.29932pt}{-11.91603pt}{-3.98557pt}{0.0pt}{-3.98557pt}\pgfsys@curveto{11.91603pt}{-3.98557pt}{21.57562pt}{-1.29932pt}{21.57562pt}{2.01443pt}\pgfsys@closepath\pgfsys@moveto{0.0pt}{2.01443pt}\pgfsys@stroke\pgfsys@invoke{ } }{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-15.25626pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{$e_{13}+e_{23}$}} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} } \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}})&(2n_{3},e_{13}+e_{23})}.$$

$$\{12\}$$

$$\mathsf{W}:{\mathbb{R}}^Q\mapsto {\mathbb{R}}^Q,$$

$$\phi=\big{(}\underbrace{\vphantom{n_{N_{2}}}n_{1},\ldots,n_{N}}_{\text{node features}},\underbrace{e_{1},\ldots,e_{N_{2}}}_{\text{edge features}}\big{)}^{\mathsf{T}}\equiv\big{(}\bm{n},\bm{e}\big{)}^{\mathsf{T}}.$$

$$\begin{array}{rl}\varphi \mapsto \mathsf{W}\varphi & =\left(\begin{array}{cc}{\mathsf{W}}^{1\to 1}& {\mathsf{W}}^{2\to 1}\\ {\mathsf{W}}^{1\to 2}& {\mathsf{W}}^{2\to 2}\end{array}\right)\left(\begin{array}{c}\mathbf{n}\\ \mathbf{e}\end{array}\right),\end{array}$$

$${\mathsf{W}}_X\simeq \left(\begin{array}{cc}{\mathsf{W}}_0^{1\to 1}+{\mathsf{W}}_1^{1\to 1}& {\mathsf{W}}_0^{2\to 1}+{\mathsf{W}}_1^{2\to 1}\\ {\mathsf{W}}_0^{1\to 2}+{\mathsf{W}}_1^{1\to 2}& {\mathsf{W}}_0^{2\to 2}+{\mathsf{W}}_1^{2\to 2}+{\mathsf{W}}_2^{2\to 2}\end{array}\right),$$

$${\mathbb{V}}_{{\mathsf{W}}_X}=\text{span}(\left\{\left(\begin{array}{cc}{\mathsf{W}}_0^{1\to 1}& 0\\ 0& 0\end{array}\right),\dots ,\left(\begin{array}{cc}0& 0\\ 0& {\mathsf{W}}_2^{2\to 2}\end{array}\right)\right\}).$$

$$\begin{array}{rl}{\mathsf{W}}_B& \simeq \underset{\text{identity}}{\underbrace{\left(\begin{array}{cc}{\mathsf{W}}_1^{1\to 1}& 0\\ 0& {\mathsf{W}}_2^{2\to 2}\end{array}\right)}}+\underset{\text{pool/broadcast edges}}{\underbrace{\left(\begin{array}{cc}{\mathsf{W}}_1^{1\to 1}& {\mathsf{W}}_1^{2\to 1}\\ 0& 0\end{array}\right)}}\\ & +\underset{\text{pool/broadcast nodes}}{\underbrace{\left(\begin{array}{cc}0& 0\\ {\mathsf{W}}_1^{1\to 2}& {\mathsf{W}}_2^{2\to 2}\end{array}\right)}}+\underset{\text{pool/broadcast all}}{\underbrace{\left(\begin{array}{cc}{\mathsf{W}}_0^{1\to 1}& {\mathsf{W}}_0^{2\to 1}\\ {\mathsf{W}}_0^{1\to 2}& {\mathsf{W}}_0^{2\to 2}\end{array}\right)}}.\end{array}$$

$${\mathbb{V}}_{{\mathsf{W}}_Y}={\mathbb{V}}_{{\mathsf{W}}_X}/\text{span}(\left\{\left(\begin{array}{cc}0& 0\\ 0& {\mathsf{W}}_1^{2\to 2}\end{array}\right)\right\})\subset {\mathbb{V}}_{{\mathsf{W}}_X}.$$

Full text

Machine Learning, ICML

1 Introduction

Many interesting data structures can be represented with sparse incidence tensors. For example, we can represent graphs using both node-node and node-edge sparse incidence matrices. We can extend this incidence representation to data defined on simplicial complexes and polytopes of arbitrary dimension, such as mesh, polygons, and polyhedra. The goal of this paper is to design deep models for these structures.

We represent an attributed geometric structure using its incidence tensor, which models the incidence pattern of its faces. For example, rows and columns in a node-edge incidence matrix are indexed by faces of size one (nodes) and two (edges). Moreover each edge (column) is incident to exactly two nodes (rows). The sparsity pattern of the incidence tensor has important information about the geometric structure. This is because sparsity preserving permutation of nodes often match the automorphism group of the geometric object; see Fig. 1(a,b).

We are interested in designing models that are informed by the symmetry of the underlying structure. We do so by making the model equivariant to symmetry transformations. When using the incidence tensor representation, a natural choice of symmetry transformations is the automorphism group of the geometric object. However, when working with a dataset comprising of different instances (e.g., different graphs or polyhedra), using individual automorphism groups is not practical. This is because each symmetry group dictates a different equivariant model, and we cannot train a single model on the whole dataset. A solution is to use the symmetric group (the group of all permutations of nodes) for all instances, which implicitly assumes a dense structure where all faces are present, e.g., all graphs are fully connected; see Fig. 1(c,d).

We show that under the action of the symmetric group, any incidence tensor decomposes into invariant subsets, or orbits, where each orbit corresponds to faces of particular size. For example, a node-node incidence matrix decomposes into: 1) diagonals, that can encode node attributes, and; 2) off-diagonals, corresponding to edge attributes. This is because any permutation of nodes, (i.e., simultaneous permutation of rows and columns) moves an (off-) diagonal entry to another (off-) diagonal entry in the node-node incidence. We can vectorize the diagonal and off-diagonal entries to get a node vector and an edge vector. These are examples of face-vectors in the general setting, and this example shows how and incidence tensor decomposes into face-vectors.

This decomposition into face-vectors also breaks up the design of equivariant linear maps for arbitrary incidence tensors into design of such maps between face-vectors of different size. We show that any such linear map can be written as a linear combination of efficient pooling-and-broadcasting operations. These equivariant linear maps replace the linear layer in a feedforward neural network to create an incidence network.

2 Related Works

Deep learning with structured data is a very active area of research. Here, we briefly review some of the closely related works in graph learning and equivariant deep learning.

Graph Learning.

The idea of graph neural networks goes back to the work of (Scarselli et al., 2009). More recently, Gilmer et al. (2017) introduced the message passing neural networks and showed that they subsume several other graph neural network architectures (Li et al., 2015; Duvenaud et al., 2015; Kearnes et al., 2016; Schütt et al., 2017), including the spectral methods that follows. Another body of work in graph deep learning extends convolution to graphs using the spectrum of the graph Laplacian (Bronstein et al., 2017; Bruna et al., 2014). While principled, in its complete form, the Fourier bases extracted from the Laplacian are instance-dependent and the lack of any parameter or function sharing across the graphs limits their generalization. Following (Henaff et al., 2015; Defferrard et al., 2016), Kipf & Welling (2016) propose a single-parameter simplification of spectral method that addresses this limitation and it is widely used in practice. Some notable extensions and related ideas include (Veličković et al., 2017; Hamilton et al., 2017; Xu et al., 2018; Zhang et al., 2018; Ying et al., 2018; Morris et al., 2018; Maron et al., 2019a).

Equivariant Deep Learning.

Equivariance constrains the predictions of a model $\phi:\mathbb{X}\mapsto\mathbb{Y}$ under a group $\mathcal{G}$ of transformations of the input, such that

[TABLE]

Here $\pi\cdot x$ is a consistently defined transformation of $x$ parameterized by $\pi\in\mathcal{G}$, while $\pi\cdot\phi(x)$ denotes the corresponding transformation of the output. For example, in a convolution layer (LeCun et al., 1998), $\mathcal{G}$ is the group of discrete translations, and Eq. 1 means that any translation of the input leads to the same translation of the output. When $\phi:x\mapsto\sigma(\mathsf{W}x)$ is a standard feed-forward layer with parameter matrix $\mathsf{W}$, the equivariance property Eq. 1 enforces parameter-sharing in $\mathsf{W}$ (Shawe-Taylor, 1993; Ravanbakhsh et al., 2017).

Most relevant to our work, are equivariant models proposed for geometric deep learning that we review next. Covariant compositional network (Kondor et al., 2018) extends the message passing framework by considering basic tensor operations that preserve equivariance. While the resulting architecture can be quite general, it comes at the cost of efficiency. Hartford et al. (2018) propose a linear map equivariant to independent permutation of different dimensions of a tensor. Equivariant graph networks of (Maron et al., 2018) model the interactions within a set of nodes. We will further discuss this model as a special type of incidence network. These equivariant layers for interactions between and within sets are further generalized to multiple types of interactions in (Graham & Ravanbakhsh, 2019). Several recent works investigate the universality of such equivariant networks (Maron et al., 2019b; Keriven & Peyré, 2019; Chen et al., 2019). A flexible approach to equivariant and geometric deep learning where a global symmetry is lacking is proposed in (Cohen et al., 2019).

3 Graphs

In this section we discuss graphs and later generalize the arguments to a broader set of geometric objects in Section 4. Without loss of generality, we assume a fully-connected graph $G=([N],\mathbb{E}\subseteq[N]\times[N])$, where $[N]=\{1,\ldots,N\}$ denotes a set of nodes, and $\mathbb{E}$ a set of edges.

3.1 Linear Layers

There are two common ways of representing $G$. The first approach is to use a node-node incidence matrix ${\mathbf{X}}_{\delta_{1},\delta_{2}}$ indexed by $\delta_{1},\delta_{2}\in[N]$. In this representation, node and edge features are encoded as diagonal and off-diagonal entries of ${\mathbf{X}}$, respectively. Here, we assume single input and output channel (i.e., scalar node and edge attributes) for simplicity; results trivially generalize to multiple channels.

Consider the group of all $N!$ permutations of nodes $\mathcal{S}_{N}$ and its action on ${\mathbf{X}}$, which simultaneously permutes the rows and columns of ${\mathbf{X}}$. Let $\mathsf{W}:\mathbb{R}^{N\times N}\to\mathbb{R}^{N\times N}$ be a linear map equivariant to this action of $\mathcal{S}_{N}$,

[TABLE]

The map $\mathsf{W}$ is constrained so that permuting the rows and columns of the input will have the same effect on the output. As shown in (Maron et al., 2018) this condition constrains the number of independent parameters in $\mathsf{W}$ to fifteen, regardless of the size of the graph.

Alternatively, one can represent $G$ with a node-edge incidence matrix ${\mathbf{Y}}_{\delta_{1},\{\delta_{2},\delta_{3}\}}$ where $\delta_{1}\in[N]$ labels nodes and the unordered pair $\{\delta_{2},\delta_{3}\}$ with $\delta_{2}\neq\delta_{3}\in[N]$ labels edges. ${\mathbf{Y}}$ has a special sparsity pattern: ${\mathbf{Y}}_{\delta_{1},\{\delta_{2},\delta_{3}\}}\neq 0$ iff node $\delta_{1}$ is incident to the edge $\{\delta_{2},\delta_{3}\}$. We identify this sparsity pattern implicitly by writing the matrix as ${\mathbf{Y}}_{\delta_{1},\{\delta_{1},\delta_{2}\}}$, so that we only index non-zero entries (note the repeated $\delta_{1}$). ${\mathbf{Y}}$ naturally encodes edge features at ${\mathbf{Y}}_{\delta_{1},\{\delta_{1},\delta_{2}\}}$ and ${\mathbf{Y}}_{\delta_{2},\{\delta_{1},\delta_{2}\}}$ for two different edge-directions of the edge $\{\delta_{1},\delta_{2}\}$.

The action of $\mathcal{S}_{N}$ on ${\mathbf{Y}}$ is also a simultaneous permutation of rows and columns, where the permutation of columns is defined by the action on the node pair that identifies each edge. $\pi\cdot\{\delta_{1},\delta_{2}\}=\{\pi\cdot\delta_{1},\pi\cdot\delta_{2}\}$. This action preserves the sparsity pattern of ${\mathbf{Y}}$ defined above (note that even a fully connected graph has a sparse node-edge incidence matrix.) The maximal equivariant linear map acting on ${\mathbf{Y}}$ is constrained to have seven independent parameters (assuming a single input and output channel). More alternatives beside node-node and node-edge representation for graph exist – e.g., one may use an edge-edge incidence matrix.

Since both ${\mathbf{X}}$ and ${\mathbf{Y}}$ represent the same graph $G$, and the corresponding linear maps are equivariant to the same group $\mathcal{S}_{N}$, one expects a relationship between the two representations and maps. This relationship is due to decomposition of ${\mathbf{X}}$ into orbits under the action of $\mathcal{S}_{N}$. In particular, ${\mathbf{X}}$ decomposes into two orbits: diagonal elements (${\mathbf{X}}_{\{\delta_{1}\}}$) and off-diagonal elements (${\mathbf{X}}_{\{\delta_{1},\delta_{2}\}}$), where each subset is invariant under the action of $\mathcal{S}_{N}$ – that is simultaneous permutations of rows and columns do not move a diagonal element to off-diagonal or vice-versa. We write this decomposition as

[TABLE]

where the diagonal orbit is isomorphic to the vector of nodes ${\mathbf{X}}_{\{\delta_{1}\}}$ and the off-diagonal orbit is isomorphic to the vector of edges ${\mathbf{X}}_{\{\delta_{1},\delta_{2}\}}$ with $\delta_{1}\neq\delta_{2}$. Consider the map $\mathsf{W}$ defined above, in which both input and target decompose in this way. It follows that the map itself also decomposes into four maps

[TABLE]

where $\mathsf{W}^{m\to m^{\prime}}$ maps a face-vector of faces of size $m$ to a face-vector of faces of size $m^{\prime}$. Equivariance to $\mathcal{S}_{N}$ for each of these maps constrains the number of independent parameters: $\mathsf{W}^{1\to 1}$ is the equivariant layer used in DeepSets (Zaheer et al., 2017), and has two parameters. $\mathsf{W}^{1\to 2}$ and $\mathsf{W}^{2\to 1}$ each have three parameters, and $\mathsf{W}^{2\to 2}$ has seven unique parameter, and maps input edge features into target edge features. One key point is that the edge-vector ${\mathbf{X}}_{\{\delta_{1},\delta_{2}\}}$ is isomorphic to the node-edge incidence matrix ${\mathbf{Y}}_{\delta_{1},\{\delta_{1},\delta_{2}\}}$ and thus the seven-parameters equivariant map for ${\mathbf{Y}}$ is exactly $\mathsf{W}^{2\to 2}({\mathbf{X}}_{\{\delta_{1},\delta_{2}\}})$ of Eq. 4. The parameter-sharing in two linear maps for the node-node and node-edge incidence matrices are visualized in Fig. 2 (left, middle). 111As a side note, one can also encode node features in a node-edge incidence matrix by doubling the number of channels and broadcasting node-features across all edges incident to a node. In this case all fifteen operations are retrieved, and the two layers for ${\mathbf{X}}$ and ${\mathbf{Y}}$ are equivalent.

Rather than using explicit parameter-sharing in $\mathsf{W}$, we show that $\mathsf{W}$ is a linear combination of pooling-and-broadcasting operations. In particular, the combination includes pooling the node indices of the input face-vector in “all possible ways”, and broadcasting the resulting collection of pooled vectors to the target face-vector, again in all ways possible. Each operation is associated with a learnable parameter; see Fig. 3 for the graph example.

3.2 Sparse Tensors and Non-Linear Layers

So far we have discussed equivariant linear layers for a fully connected graph. This means dense input/output node-node incidence ${\mathbf{X}}$, or equivalently a node-edge incidence ${\mathbf{Y}}$ with the sparsity pattern described in the previous section (which is maintained by the action of $\mathcal{S}_{N}$). To avoid the cost of a dense representation, one may apply a sparsity mask after the linear map, while preserving equivariance:

[TABLE]

where $\mathsf{W}$ is the equivariant linear map of Section 3.1, $s({\mathbf{X}})$ is the sparsity mask, and $\circ$ is the Hadamard product. For example, assuming the layer output has the same shape as the input, we can choose to preserve the sparsity of the input. In this case, $s({\mathbf{X}})$ will have zero entries where the input ${\mathbf{X}}$ has zero entries, and ones otherwise. However, the setting of Eq. 5 is more general as input and output may have different shapes. Since the sparsity mask $s({\mathbf{X}})$ depends on the input, the map of Eq. 5 is now non-linear. In practice, rather than calculating the dense output and applying the sparsity mask, we directly produce and store only the non-zero values.

3.3 Further Relaxation of the Symmetry Group

The neural layers discussed so far are equivariant to the group $\mathcal{G}=\mathcal{S}_{N}$, where $N$ is the number of nodes. A simplifying alternative is to assume independent permutations of rows and columns of the node-node or node-edge matrix. This is particularly relevant for the node-edge matrix, where one can consider node and edges as two interacting sets of distinct objects. The corresponding $(\mathcal{S}_{N}\times\mathcal{S}_{N_{2}})$-equivariant layer, where $N_{2}$ is the number of edges, was introduced in (Hartford et al., 2018), has 4 unique parameters, and it is substantially easier to implement compared to the layers introduced so far. In Appendix A we show how to construct a sparsity-preserving (and therefore non-linear) layer for this case. Even though a single layer is over-constrained by these symmetry assumptions, we prove that two such layers generate exactly the same node and edge features as a single $\mathcal{S}_{N}$-equivariant linear layer for a node-node incidence.

4 Higher Order Geometric Structures

In this section we generalize the results of Section 3 to geometric structures beyond graphs. In Section 4.1 we provide a definition of incidence tensors, which generalize node-node and node-edge matrices of graphs. We discuss several examples, showing how they can represent geometric structures such as polytopes and simplicial complexes. In Section 4.2 we generalize the orbit decomposition of Section 3.1. Finally, in Section 4.3, we show how to build equivariant layers using a linear combination of simple pooling-and-broadcasting operations for arbitrary incidence tensors.

4.1 Incidence Tensors

Recall that $[N]=\{1,\ldots,N\}$ denotes a set of nodes. A directed face of size $M$ is an ordered tuple of $M$ distinct nodes $\bm{\delta}\in[N]^{M}\;|\;\delta_{i}\neq\delta_{j}\forall i\neq j$. Following a similar logic, an undirected face $\bm{\delta}\subseteq[N]$, is a subset of $[N]$ of size $M$. We use $\bm{\delta}^{(M)}$ when identifying the size of the face – i.e., $|\bm{\delta}^{(M)}|=M$. For example, $\bm{\delta}^{(2)}$ identifies an edge in a graph, or a mesh, while $\bm{\delta}^{(3)}$ is a triangle in a triangulated mesh.

An incidence tensor ${\mathbf{X}}_{\bm{\delta}_{1},\ldots,\bm{\delta}_{D}}$ is a tensor of order $D$, where each dimension is indexed by all faces of size $M_{d}=|\bm{\delta}_{d}|$. For example, if $\bm{\delta}_{1}=\{\delta_{1,1}\}$ indexes nodes, and $\bm{\delta}_{2}=\{\delta_{2,1},\delta_{2,2}\}$ identifies an edge, ${\mathbf{X}}_{\bm{\delta}_{1},\bm{\delta}_{2}}$ becomes a node-edge incidence matrix. An incidence tensor has a sparsity structure, identified by a set of constraints $\Sigma=\{\bm{\sigma}_{1},\ldots,\bm{\sigma}_{C}\}$, where all the indices $\sigma_{m_{1}}\ldots,\sigma_{m_{c}}\in\bm{\sigma}_{c},\;\forall\bm{\sigma}_{c}\in\Sigma$ are equal for any non-zero entry of ${\mathbf{X}}$. For example, we have ${\mathbf{X}}_{\{\delta_{1,1}\},\{\delta_{2,1},\delta_{2,2}\}}\neq 0$, only if $\delta_{1,1}=\delta_{2,1}$. Therefore $\Sigma=\{\bm{\sigma}_{1}=\{\delta_{1,1},\delta_{2,1}\}\}$.

While in general the pair $({\mathbf{X}}_{\bm{\delta}_{1},\ldots,\bm{\delta}_{D}},\Sigma)$ defines the incidence tensor, whenever it is clear from context, we will only use ${\mathbf{X}}_{\bm{\delta}_{1},\ldots,\bm{\delta}_{D}}$ to denote it. This formalism can represent a variety of different geometric structure as demonstrated in the following sections.

4.1.1 Simplicial Complexes

Before discussing general simplicial complexes let us review graphs as an example of incidence tensors.

The node-node incidence matrix is an incidence tensor ${\mathbf{X}}_{\{\delta_{1}\},\{\delta_{2}\}}$ indexed by a pair of nodes, with no sparsity constraints. We denoted it with ${\mathbf{X}}_{\delta_{1},\delta_{2}}$ in Section 3.1 for simplicity. The node-edge incidence matrix is denoted by the pair $({\mathbf{X}}_{\{\delta_{1}\},\{\delta_{2},\delta_{3}\}},\{\delta_{1},\delta_{2}\})$. It is indexed by nodes $\{\delta_{1}\}$ and edges $\{\delta_{2},\delta_{3}\}$. The entries can be non-zero only when $\delta_{1}=\delta_{2}$, meaning that the edge $\{\delta_{1},\delta_{3}\}$ is adjacent to the node $\{\delta_{1}\}$. An alternative notation is ${\mathbf{X}}_{\{\delta_{1}\},\{\delta_{1},\delta_{2}\}}$. Again, we denoted it simply with ${\mathbf{X}}_{\delta_{1},\{\delta_{1},\delta_{2}\}}$ in Section 3.1. We have also denoted node and edge vectors with ${\mathbf{X}}_{\{\delta_{1}\}}$ and ${\mathbf{X}}_{\{\delta_{1},\delta_{2}\}}$, respectively. As a final example, ${\mathbf{X}}_{\{\delta_{1},\delta_{2}\},\{\delta_{1},\delta_{3}\}}$ would denote an edge-edge incidence matrix whose entries are non-zero wherever two edges are incident.

Let us now move to the definition of a general (undirected) simplicial complex. An abstract simplicial complex $\Delta\subseteq 2^{[N]}$ is a collection of faces, closed under the operation of taking subsets – that is $(\bm{\delta}_{1}\in\Delta\;\text{and}\;\bm{\delta}_{2}\subset\bm{\delta}_{1})\Rightarrow\bm{\delta}_{2}\in\Delta$. Dimension of a face $\bm{\delta}$ is its size minus one. Maximal faces are called facets and the dimension of $\Delta$ is the dimension of its largest facet. For example, an undirected graph is a one-dimensional simplicial complex. Each dimension of an incidence tensor ${\mathbf{X}}_{\bm{\delta}_{1},\ldots,\bm{\delta}_{D}}$ may be indexed by faces of specific dimension. Two undirected faces of different dimension $\bm{\delta},\bm{\delta}^{\prime}\in\Delta$ are incident if one is a subset of the other. This type of relationship as well as alternative definitions of incidence between faces of the same dimension can be easily accommodated in the form of equality constraints in $\Sigma$.

Although not widely used, a directed simplicial complex can be defined similarly. The main difference is that faces are sequences of the nodes, and $\Delta$ is closed under the operation of taking a subsequence. As one might expect, the incidence tensor for directed simplicial complexes can be built using directed faces in our notation.

Example 1**.**

A zero-dimensional simplicial complex is a set of points that we may represent using an incidence vector. At dimension one, we get undirected graphs, where faces of dimension one are the edges. Triangulated mesh is an example of two-dimensional simplicial complex; see figure below.

The triangular bi-pyramid of Fig. 1 is an example of 3 dimensional simplicial complex with 5 nodes, 9 edges, 7 faces of size 3, and two faces of size 4. The node-face incidence matrix in Fig. 1(a) is expressed by ${\mathbf{X}}_{\{\delta_{1}\},\{\delta_{1},\delta_{2},\delta_{3}\}}$ in our formalism.

4.1.2 Polygons, Polyhedra, and Polytopes

Another family of geometric objects with incidence structure is polytopes. A formal definition of abstract polytope and its representation using incidence tensors is given in Appendix D. A polytope is a generalization of polygone and polyhedron to higher dimensions. The structure of an (abstract) polytope is encoded using a partially ordered set (poset) that is graded, i.e., each element of the poset has a rank. For example, Fig. 4, shows the poset for a cube, where each level is a different rank, and subsets in each level identify faces of different size (nodes, edges, and squares). The idea of using incidence tensor representation for a polytope, is similar to its use for simplicial complexes. Each dimension of ${\mathbf{X}}_{\bm{\delta}_{1},\ldots,\bm{\delta}_{D}}$ indexes faces of different rank. Two faces of the same dimension may be considered incident if they have a face of specific lower rank in common. We may also define two faces of different dimension incident if one face is a subset of the other – i.e., $\bm{\delta}^{(m)}<\bm{\delta}^{(m^{\prime})}$ in the partial order.

4.2 Symmetry & Decomposition

The automorphism group $\mathcal{Aut}({\mathbf{X}})\leq\mathcal{S}_{N}$ associated with an incidence tensor is the set of all permutations of nodes that maps every face to another face, and therefore preserve the sparsity

[TABLE]

where the action of $\mathcal{Aut}({\mathbf{X}})$ on the faces is naturally defined as

[TABLE]

See Fig. 1(a,b) for an example. We may then construct $\mathcal{Aut}({\mathbf{X}})$-equivariant linear layers through parameter-sharing. However, the constraints on this linear operator varies if our dataset has incidence tensors with different sparsity patterns. For example, a directed graph dataset may contain a fully connect graph with automorphism group $\mathcal{S}_{N}$ and a cyclic graph with automorphism group $\mathcal{C}_{N}$. For these two graphs, node-node and node-edge incidence matrices are invariant to the corresponding automorphism groups, necessitating different constraints on their linear layer. To remedy the problem with model-sharing across instances, we densify all incidence tensors so that all directed or undirected faces of a given dimension are present. Now, one may use the same automorphism group $\mathcal{S}_{N}$ across all instances; see Fig. 1(c,d). Next, we consider the incidence tensor as a $\mathcal{G}$-set, and identify the orbits of $\mathcal{S}_{N}$ action.

Theorem 4.1**.**

The action of $\mathcal{S}_{N}$ on any incidence tensor ${\mathbf{X}}$ decomposes into orbits that are each isomorphic to a face-vector:

$\displaystyle({\mathbf{X}}_{\bm{\delta}_{1},\ldots,\bm{\delta}_{D}},\Sigma)\;\cong\;\bigcupdot_{m}\;\kappa_{m}\,{\mathbf{X}}_{\bm{\delta}^{(m)}},$

(7)

where $\kappa_{m}$ is the multiplicity of faces of size $m$. The value of $\kappa_{m}$ is equal to the number of partitioning of the set of all indices $\{\delta_{1,1},\ldots,\delta_{1,M_{1}},\ldots,\delta_{D,{M_{D}}}\}$ into $m$ non-empty partitions, such that $\delta_{d,m}\,\forall m\in[M_{d}]$ belong to different partitions, and members of $\bm{\sigma}\in\Sigma$ belong to the same partition.

The proof appears in Appendix B.

Example 2** (Node-adjacency tensors).**

Consider an order $D$ node-node-…-node incidence tensor ${\mathbf{X}}_{\{\delta_{1}\},\ldots\{\delta_{D}\}}$ with no sparsity constraints. In this case, the multiplicity $\kappa_{m}$ of Eq. 7 corresponds to the number of ways of partitioning a set of $D$ elements into $m$ non-empty subsets and it is also known as Stirling number of the second kind (written as $\genfrac{\{}{\}}{0.0pt}{}{D}{m}$). Each partition of size $m$ identifies a face-vector ${\mathbf{X}}_{\bm{\delta}^{(m)}}$ for a face of size $m$. These faces can be identified as hyper-diagonals of order $m$ in the original adjacency tensor ${\mathbf{X}}$. For example, as shown in the figure below, ${\mathbf{X}}_{\{\delta_{1}\},\{\delta_{2}\},\{\delta_{3}\}}$ decomposes into a node-vector (the main diagonal of the adjacency cube), three edge-vectors (isomorphic to the three diagonal planes of the cube adjacency, with the main diagonal removed), and one hyper-edge-vector (isomorphic to the adjacency cube, where the main diagonal and diagonal planes have been removed). Here, $\kappa_{1}=\genfrac{\{}{\}}{0.0pt}{}{3}{1}=1$, $\kappa_{2}=\genfrac{\{}{\}}{0.0pt}{}{3}{2}=3$, and $\kappa_{3}=\genfrac{\{}{\}}{0.0pt}{}{3}{3}=1$.

4.3 Equivariant Maps for Incidence Tensors

As shown in the previous section, any incidence tensor can be decomposed into disjoint union of face-vectors, that are invariant sets under the action of the symmetric group. An implication is that any equivariant map from an incidence tensor to another also decomposes into equivariant maps between face-vectors.

Let $\mathsf{W}^{M\to M^{\prime}}$ be a linear function (here represented as a tensor) that maps a vector of faces of size $M$ to a vector of faces of size $M^{\prime}$,

[TABLE]

where $\delta_{1},\ldots,\delta_{M}$ identifies faces of size $M$, and (using Einstein notation) repeated indices on are summed over. Equivariance to $\mathcal{S}_{N}$ is realized through a symmetry constraint on $\mathsf{W}$,

[TABLE]

which ties the elements within each orbit of the so called diagonal $\mathcal{S}_{N}$-action on $\mathsf{W}$; see Fig. 2 (left, middle) for a graph example.

4.3.1 Pool & Broadcast Interpretation

Each unique parameter in the constrained $\mathsf{W}$ corresponds to a linear operation that has a pool and broadcast interpretation – that is any linear equivariant map between two incidence tensors can be written as a linear combination of pooling-and-broadcasting operations. Moreover, this interpretation allows for a linear-time implementation of the equivariant layers, as we avoid the explicit construction of $\mathsf{W}$.

Definition 1** (Pooling).**

Given a face vector ${\mathbf{X}}_{(\delta_{1}\ldots\delta_{M})}$, for $\mathbb{P}=\{{p_{1}},\ldots,{p_{L}}\}\subseteq[M]$, the pooling operation sums over the indices in $\mathbb{P}$:

[TABLE]

In practice, the summation in the definition may be replaced with any permutation-invariant aggregation function.

Definition 2** (Broadcasting).**

$\operatorname{Bcast}_{\mathbb{B},M^{\prime}}({\mathbf{X}})$* broadcasts ${\mathbf{X}}$, a faces vector of size $M$, over a target vector of faces of size $M^{\prime}\geq M$. We identify ${\mathbf{X}}$ with a sequence of node indices of the target face-vector, $\mathbb{B}=(b_{1},\ldots,b_{M})$ with $b_{m}\in[M^{\prime}]$, and we broadcast across the remaining $M^{\prime}-M$ node indices – that is*

[TABLE]

For example, given an edge-vector ${\mathbf{X}}={\mathbf{X}}_{\{\delta_{1},\delta_{2}\}}$, $\operatorname{Bcast}_{\langle{0,1}\rangle,3}({\mathbf{X}})$ broadcasts ${\mathbf{X}}$ to a triangle-vector (i.e., vector of faces of size $3$), where ${\mathbf{X}}$ is mapped to the first two node indices and broadcasted along the third. Note that operations defined through pooling-and-broadcasting are equivariant to permutation of nodes. In fact, it turns out that an equivariant $\mathsf{W}$ can only linearly combine pooling and broadcasting of input incidence tensor into an output tensor.

Theorem 4.2**.**

Any equivariant linear map $\mathsf{W}^{M\to M^{\prime}}$ between face-vectors of size $M$ and $M^{\prime}$, defined in Eq. 8, can be written as

\mathsf{W}^{M\to M^{\prime}}({\mathbf{X}})=\sum_{\begin{subarray}{c}\mathbb{P}\subseteq[M]\\ \mathbb{B}\subseteq\langle{1,\ldots,M^{\prime}}\rangle\\ |\mathbb{B}|=M-|\mathbb{P}|\end{subarray}}{\mathsf{w}}_{\mathbb{B},\mathbb{P}}\operatorname{Bcast}_{\mathbb{B},M^{\prime}}\big{(}\operatorname{Pool}_{\mathbb{P}}({\mathbf{X}})\big{)}.

(10)

The proof appears in Appendix B. The sum of the pooling-and-broadcasting operations in Eq. 10 includes pooling the node indices of the input face-vector in all possible ways, and broadcasting the resulting collection of face-vectors to the target face-vector, again in all ways possible; ${\mathsf{w}}_{\mathbb{B},\mathbb{P}}\in\mathbb{R}$ is the parameter associated with each unique pooling-and-broadcasting combination. More details are discussed in Appendix C.

The number of operations in Eq. 10, is given by

[TABLE]

This counts the number of possible choices of $m$ indices out of $M$ input indices in Eq. 8 and $m$ indices out of $M^{\prime}$ output indices to for pool and broadcast. Once this set is fixed there are $m!$ different ways to match input indices to output indices.

4.3.2 Decomposition of Equivariant Maps

Let $\mathsf{W}:(\bigcupdot_{m}\kappa_{m}{\mathbf{X}}_{\bm{\delta}^{(m)}})\mapsto(\bigcupdot_{m^{\prime}}\kappa_{m^{\prime}}^{\prime}{\mathbf{X}}_{\bm{\delta}^{(m^{\prime})}})$ be an equivariant map between arbitrary incidence tensors, where both input and output decompose according to Eq. 7. Using the equivariant maps $\mathsf{W}^{m\to m^{\prime}}$ of Eq. 10, we get a decomposition of $\mathsf{W}$ into all possible combination of input-output face vectors

[TABLE]

where for each copy (out of $\kappa_{m^{\prime}}^{\prime}$ copies) of the output face of size $m^{\prime}$, we are summing over all the maps produced by different input faces having different multiplicities. Use of $k$ and $k^{\prime}$ in the map $\mathsf{W}^{k,m\to k^{\prime},m^{\prime}}$ is to indicate that for each input-output copy, the map $\mathsf{W}^{m\to m^{\prime}}$ uses a different set of parameters. The upshot is that input and output multiplicities $\kappa,\kappa^{\prime}$ play a role similar to input and output channels.

The total number of independent parameters in a layer is

[TABLE]

where $\tau^{m\to m^{\prime}}$ is given by Eq. 11.

Example 3** (Node-adjacency tensors).**

This example, is concerned with the incidence representation used in equivariant graph networks of (Maron et al., 2018) and derives their model as a special case, using our pooling-and-broadcasting layer and face-vectors decomposition. For an equivariant layer that maps a node-node-…-node incidence tensor ${\mathbf{X}}$ of order $D$ to the same structure, the decomposition in terms of face-vectors reads

[TABLE]

where $\genfrac{\{}{\}}{0.0pt}{}{D}{m}$ is the Stirling number of the second kind; see Example 2. The total number of operations according to Eq. 13 is then given by

[TABLE]

In the last line, $\mathsf{Bell}(2D)$ is the Bell number and counts the number of unique partitions of a set of size $2D$. To see the logic in the final equality: first divide $[2D]$ in half. Next, partition each half into subsets of different sizes ($0\leq m,m^{\prime}\leq D$) and choose $l$ of these partitions from each half and merge them in pairs. The first two terms count the number of ways we can partition each half into $m$ (or $m^{\prime}$) partitions and select a subset of size $l$ among them. The $l!$ term accounts for different ways in which $l$ partitions can be aligned. This result agrees with the result of (Maron et al., 2018). Therefore one may implement the hyper-graph networks using efficient pooling-and-broadcasting operations outlined in Eq. 10.

Recall that when discussing equivariant layers for graphs, we also considered independent permutations of rows and columns in a node-edge incidence matrix, and claimed that despite having only 4 parameters, stacking two such layers (with additional channels) is equivalent to the 15 parameter model. In Appendix E a similar result is given for higher dimensions, showing that one may use $\mathcal{S}_{N_{|\bm{\delta}_{1}|}}\times\ldots\times\mathcal{S}_{N_{|\bm{\delta}_{D}|}}$ as the symmetry group of an incidence tensor, where the equivariant model has $2^{D}$ parameters.

5 Conclusion and Future Work

This paper introduces a general approach to learning equivariant models for a large family of structured data through their incidence tensor representation. In particular, we showed various incidence tensor representations for graphs, simplicial complexes, and abstract polytopes.

The proposed family of incidence networks are 1) modular: they decompose to simple building blocks; 2) efficient: they all have linear-time pooling-and-broadcasting implementation.

In our systematic study of this family, we discussed implications of 1) added symmetry due to undirected faces; 2) sparsity preserving equivariant maps, and; 3) the successive relaxation of the symmetry group $\mathcal{Aut}({\mathbf{X}})\leq\mathcal{S}_{N}\leq\mathcal{S}_{N_{|\bm{\delta}_{1}|}}\times\ldots\times\mathcal{S}_{N_{|\bm{\delta}_{D}|}}$, from the automorphism group, to a direct product of symmetric groups that independently permutes each dimension of the incidence tensor. Here, moving to a larger group simplifies the neural layer by reducing the number of unique parameters (and linear operations), while increasing its bias.

Application of incidence networks to different domains, such as learning on triangulated mesh, is a natural extension of this work. Note that for a comprehensive study with nodes, edges, and triangles, as they appear in triangulated mesh, one could investigate multiple order three representations (node-node-node, node-edge-face, etc), as well as some order two representations (node-face, face-face, etc.), or simply work with face-vectors. For each of these, one potentially needs a different incidence network using a different set of sparse pool-broadcast operations. We hope to explore this direction in a follow-up work.

Acknowledgements

We thank the reviewers for helpful comments on the manuscript. The computational resources for this work were mainly provided by Compute Canada. This research was in part supported by Canada CIFAR AI Chair Program and NSERC Discovery Grant.

Appendix A Additional Results for Equivariant Layers Using Relaxed Symmetry Group

Consider an undirected graph with $N$ nodes and its node-node ${\mathbf{X}}_{\delta_{1},\delta_{2}}$ and node-edge ${\mathbf{Y}}_{\delta_{1},\{\delta_{1},\delta_{2}\}}$ incidence representations. We discussed the equivalence of their $\mathcal{S}_{N}$-equivariant linear layers in Section 3.1. Here, we study node-edge layers equivariant to $\mathcal{S}_{N}\times\mathcal{S}_{N_{2}}$, where $N_{2}=N(N-1)/2$ is the number of edges, and compare their expressive power with their $\mathcal{S}_{N}$-equivariant linear counterparts.

Node-Node Incidence.

Let ${\mathbf{X}}\in\mathbb{R}^{N\times N\times K}$ be a node-node incidence matrix with $K$ channels. Consider a linear layer $\mathsf{W}_{X}:{\mathbf{X}}\mapsto{\mathbf{X}}^{\prime}\in\mathbb{R}^{N\times N\times K^{\prime}}$, where $\mathsf{W}_{X}$ is defined as in Eq. 12, and $K^{\prime}$ are output channels. Diagonal elements of ${\mathbf{X}}$ encode input node features and off-diagonal elements input edge features. Since the graph is undirected, ${\mathbf{X}}$ is symmetric, and the corresponding number of independent pooling and broadcasting operations (see, e.g., equation Eq. 31) is $\tau_{X}=9$, for a total of $9KK^{\prime}$ parameters.

Node-Edge Incidence.

Alternatively, one could represent the graph with a node-edge incidence matrix ${\mathbf{Y}}\in\mathbb{R}^{N\times N_{2}\times 2K}$. Node features are mapped on the first $K$ channels along the node dimension and broadcasted across the edge dimension. Similarly, edge features are mapped on the last $K$ channels along the edge dimension and broadcasted across the node dimension. Consider a layer equivariant to $\mathcal{S}_{N}\times\mathcal{S}_{N_{2}}$ as described in Section 3.3, that also preserves the sparsity through the non-linear implementation of Section 3.2, $\mathsf{W}_{Y}:{\mathbf{Y}}\mapsto{\mathbf{Y}}^{\prime}\in\mathbb{R}^{N\times N_{2}\times 2K^{\prime}}$, where ${\mathbf{Y}}^{\prime}=\overline{\mathsf{W}}_{Y}({\mathbf{Y}})\circ s({\mathbf{Y}})$, and $\overline{\mathsf{W}}$ is an equivariant map defined as in (Hartford et al., 2018). We can write this linear map in our notation as

[TABLE]

where $2^{\{0,1\}}$ is the set of all subsets of $\{0,1\}$, and we have dropped the output dimension in the $\operatorname{Bcast}$ operator, as all features are broadcasted back to ${\mathbf{Y}}^{\prime}$. $\overline{\mathsf{W}}_{Y}({\mathbf{Y}})$ corresponds to pooling-and-broadcasting each of the two dimensions of ${\mathbf{Y}}$ independently, thus the summation has four terms, for pooling/broadcasting over rows, columns, both rows and columns and no pooling at all. The number of independent operations is $\tau_{Y}=4$ for a total of $16KK^{\prime}$ independent parameters.

Theorem A.1**.**

Let $\mathsf{W}_{X}$ and $\mathsf{W}_{Y}$ be the linear $\mathcal{S}_{N}$-equivariant and the non-linear ($\mathcal{S}_{N}\times\mathcal{S}_{N_{2}}$)-equivariant layers operating on node-node and node-edge incidence, respectively. The following statements hold:

(a)

a single $\mathsf{W}_{Y}$ layer spans a subspace of features spanned by $\mathsf{W}_{X}$,

(b)

two $\mathsf{W}_{Y}$ layers span the same feature space spanned by $\mathsf{W}_{X}$ (maximal feature space).

Proof.

First, we discuss how to interpret output features. Additionally, for the rest of the proof we will assume $K=K^{\prime}=1$ for simplicity, noting that the proof generalizes to the multi-channel case.

Output Features.

For a node-node incidence layer, it is natural to interpret diagonal and off-diagonal elements of ${\mathbf{X}}^{\prime}$ as output node and edge features, respectively.

For the node-edge incidence case, all four operations of the $\mathsf{W}_{Y}$ map return linear combinations of features that vary at most across one dimension, and are repeated across the remaining dimensions. This is the same pattern of input node and edge features, and we will use the same scheme to interpret them. In particular,

• •

an output feature that varies across the node dimension (but it is repeated across the edge dimension) is a node feature,

• •

an output feature that varies across the edge dimension (but it is repeated across the node dimension) is an edge feature,

• •

and finally a feature that is repeated across both node and edge dimension is either a node or edge feature.

For example, consider a complete graph with three nodes. Its incidence matrix with node and edge features $n_{\delta_{1}}$ and $e_{\delta_{1}\delta_{2}}$ repeat across rows and columns as follows

[TABLE]

Consider pooling and broadcasting across the edge dimension. Node features were broadcasted across it, and since every node is incident with two edges, we get back multiples of the original features. The edge channel will instead return new features that combine the edges incident on each node. These new node features vary across the node dimension and are broadcasted across the edge dimension,

[TABLE]

On the other hand, pooling and broadcasting across the node dimension returns multiples of input edge features, and generates new edge features from the two nodes incident on each edge,

[TABLE]

Proof of statement (a).

Given the feature encoding described above, $\mathsf{W}_{X}$ and $\mathsf{W}_{Y}$ layers can be represented as a function acting on the space of node and edge features,

[TABLE]

where $Q=N(N+1)/2$ is the number of graph elements, i.e., the sum of nodes and edges. Let us fix a basis in $\mathbb{R}^{Q}$ such that the first $N$ components of a vector $\phi\in\mathbb{R}^{Q}$ represent node features and the remaining $N_{2}=N(N-1)/2$ represent edge features

[TABLE]

Then a layer has a matrix representation $\mathsf{W}\in\mathbb{R}^{Q\times Q}$,

[TABLE]

where we have split the matrix into four sub-blocks acting on the sub-vectors of node and edge features. Here $\mathsf{W}^{m\to m^{\prime}}$ labels operators that maps faces of size $m$ into faces of size $m^{\prime}$, with $1\leq m,m^{\prime}\leq 2$. Nodes are faces of size 1 and edges faces of size 2; see also Section 4 for a general definition of faces. In particular, $\mathsf{W}^{1\to 1}\in\mathbb{R}^{N\times N}$ maps input node features to output node features, $\mathsf{W}^{2\to 1}\in\mathbb{R}^{N\times N_{2}}$ maps input edge features to output node features, $\mathsf{W}^{1\to 2}\in\mathbb{R}^{N_{2}\times N}$ maps input node features to output edge features, and $\mathsf{W}^{2\to 2}\in\mathbb{R}^{N_{2}\times N_{2}}$ maps input edge features to output edge features.

The nine operations of the $\mathsf{W}_{X}$ map can be written as

[TABLE]

and are summarized in Table 1. We have split the operations according to the sub-blocks defined above, where $\mathsf{W}^{m\to m^{\prime}}_{i}$ represents operators mapping faces of size $m$ to faces of size $m^{\prime}$, and with $0\leq i\leq m,m^{\prime}$ representing the size of the faces after pooling. In terms of pooling-and-broadcasting of Definition 1 and Definition 2,

[TABLE]

where ${\mathbf{X}}_{\bm{\delta}^{(m)}}={\mathbf{X}}_{(\delta_{1},\ldots,\delta_{m})}$ is a face-vector representing either node or edge features. For example, $\mathsf{W}^{1\to 1}_{0}$ is the operator that pools all node features (i.e., it pools the node-vector to dimension zero) and broadcasts the pooled scalar over nodes. The symbol $\simeq$ in Eq. 21 indicates that each operator is defined up to a multiplicative constant (i.e., the corresponding learnable parameter ${\mathsf{w}}_{\mathbb{B},\mathbb{P}}$ in Eq. 10). The action of $\mathsf{W}_{X}$ on the space of node and edge features can be uniquely identified by the nine-dimensional subspace

[TABLE]

On the other hand, the four operations on the two channels of the $\mathsf{W}_{Y}$ map can be written as

[TABLE]

In particular,

• •

identity: if no pooling is applied, we simply map input node and edge features into output node and edge features, respectively. Note that, since we map two input channels into two output channels, we have four independent parameters associated with this operation, but two of them are redundant.

• •

pool/broadcast edges: when pooling and broadcasting over the edge dimension we are mapping input edge features into output node features, like the example in Eq. 16. Input node features are also mapped into (multiples of) themselves. Similarly to the previous case, two of the four parameters are redundant.

• •

pool/broadcast nodes: when pooling and broadcasting over the node dimension we are mapping input node features into output edge features, like the example in Eq. 17. Input edge features are also mapped into (multiples of) themselves. Similarly to the previous case, two of the four parameters are redundant.

• •

pool/broadcast all: when pooling and broadcasting over both dimensions we get the pooled node and pooled edge features across the entire matrix, which we can interpret as either node or edge features as described in the previous paragraph. We can use the four independent parameters associated with this operation to identify it with the operators that map pooled nodes to node and edges, and pooled edges to node and edges.

From Eq. 24, a single node-edge $\mathsf{W}_{Y}$ layer does not generate $\mathsf{W}^{2\to 2}_{1}$, which corresponds to pooling one dimension of the edge-tensor and rebroadcasting the result to the full tensor (in order to preserve symmetry, the pooled one-dimensional tensor has to be rebroadcasted both across rows and columns). Thus, the subspace $\mathbb{V}_{\mathsf{W}_{B}}\subseteq\mathbb{R}^{Q\times Q}$ of node and edge features spanned by a single $\mathsf{W}_{Y}$ map is a subspace of $\mathbb{V}_{\mathsf{W}_{X}}$,

[TABLE]

Proof of statement (b).

Consider the definition of $\mathsf{W}^{m\to m^{\prime}}_{i}$ operators in Eq. 22. The following multiplication rule holds:

[TABLE]

In order to show this, consider the following

[TABLE]

where the first two lines are simply substituting the definition of the operators. The important step is going from second to third line: this is considering all the pooled tensors that can be created and broadcasting them to the target $m^{\prime\prime}$-dimensional tensor. Note that only the dimension of these pooled tensors is important as the constants are irrelevant and absorbed in the $\simeq$ symbol. In particular, the lowest dimension ${\mathbf{X}}_{\bm{\delta}^{(m)}}$ is pooled to, is $\max(0,m-(m-i)-(m^{\prime}-j))=\max(0,i+j-m^{\prime})$. On the other hand, the maximum dimension ${\mathbf{X}}_{\bm{\delta}^{(m)}}$ is pooled to is $\min(i,j)$. Also, note that Eq. 26 is consistent with the special case $m^{\prime}=m^{\prime\prime}=j$. In this case $\mathsf{W}^{m^{\prime}\to m^{\prime}}_{m^{\prime}}\simeq\bm{1}$ is proportional to the identity operator. Consistently, we get $\mathsf{W}^{m^{\prime}\to m^{\prime}}_{m^{\prime}}\mathsf{W}^{m\to m^{\prime}}_{i}\simeq\sum_{k=\max(0,i)}^{\min(m^{\prime},i)}\mathsf{W}^{m\to m^{\prime}}_{k}=\mathsf{W}^{m\to m^{\prime}}_{i}$. The same relation holds for the special case $m=m^{\prime}=i$.

Using Eq. 26 we get

[TABLE]

Two stacked $\mathsf{W}_{Y}$ maps span the same subspace of operators and thus output features as a single $\mathsf{W}_{X}$ map. Note that $\mathsf{W}^{2\to 2}_{1}$ missing from a single $\mathsf{W}_{Y}$ is generated by composing $\mathsf{W}^{2\to 2}_{1}\simeq\mathsf{W}^{1\to 2}_{1}\mathsf{W}^{2\to 1}_{1}$, that is edge features are pooled to dimension one and broadcasted to nodes in the first layer through $\mathsf{W}^{2\to 1}_{1}$, and then re-broadcasted across rows and columns, like all the other node features, in the second layer through $\mathsf{W}^{1\to 2}_{1}$. Furthermore, using the same multiplication rules, we find that

[TABLE]

thus by taking a single $\mathsf{W}_{X}$ map or by stacking two $\mathsf{W}_{Y}$ maps we span the maximal node and edge feature space. ∎

Appendix B Proofs

B.1 Proof of Theorem 4.1

Proof.

Consider an incidence tensor $({\mathbf{X}}_{\bm{\delta}_{1},\ldots,\bm{\delta}_{D}},\Sigma)$. Assume there exist two node indices $\delta_{d,m}$, $\delta_{d^{\prime},m^{\prime}}$ within two face indices $\bm{\delta}_{d}$ and $\bm{\delta}_{d^{\prime}},d^{\prime}\neq d$ that are not constrained to be equal by $\Sigma$; therefore they can be either equal or different. The action of $\mathcal{S}_{N}$ (and any of its subgroups) maintains this (lack of) equality, that is

[TABLE]

This means that the set of indices constrained by $\Sigma$ $\{\bm{\delta}_{1},\ldots,\bm{\delta}_{D}\mid\Sigma\}$, can be partitioned into two disjoint G-sets

[TABLE]

with $\delta_{d,m}=\delta_{d^{\prime},m^{\prime}}$ in one G-set and $\delta_{d,m}\neq\delta_{d^{\prime},m^{\prime}}$ in the other. We may repeat this partitioning recursively for each of the resulting index-sets. This process terminates with homogeneous G-sets where any two indices $\delta_{d,m}$ and $\delta_{d^{\prime},m^{\prime}}$ are either constrained to be equal or different. It follows that if we aggregate all equal indices, we are left with a set of indices that are constrained to be different, and therefore define a face $\bm{\delta}^{(m)}$.

The number of ways in which we are left with $m$ different indices $\kappa_{m}$ is given by the number of partitions of $\{\delta_{1,1},\ldots,\delta_{1,M_{1}},\ldots,\delta_{D,M_{D}}\}$ into $m$ non-empty sets, where two elements in the same partition are constrained to be equal. This partitioning is further constrained by the fact that elements within the same face $\bm{\delta}_{d}$, are constrained to be different and therefore should belong to different partitions. Moreover, $\Sigma$ adds its equality constraints. ∎

B.2 Proof of Theorem 4.2

Proof.

Consider the linear map $\mathsf{W}^{M\to M^{\prime}}$ of Eq. 8. Each unique parameter corresponds to an orbit under the action of $\mathcal{S}_{N}$. Orbits are partitions of the set $\{\delta_{1},\ldots,\delta_{M},\delta^{\prime}_{1},\ldots,\delta^{\prime}_{M^{\prime}}\}$ of input and output node indices, where any subset of the partition contains at most one input and one output index. That is, subsets have either size one, with a single input or output index, or size two, with a pair of input and output indices. Given such a partition, all and only $\mathsf{W}^{M\to M^{\prime}}$ elements such that indices in the same subset are equal and indices in separate subsets are different, identify an orbit. That is, given any two elements of $\mathsf{W}^{M\to M^{\prime}}$, there exists an element of $\mathcal{S}_{N}$ that maps one onto the other iff they satisfy the equality pattern described above. Each partition can be mapped to a pooling-and-broadcasting operation as detailed in Appendix C.

This is similar to the construction of (Maron et al., 2018). Here, we generalize their partitions approach and the pooling-and-broadcasting implementation to arbitrary incidence tensors (through the decomposition of Theorem 4.1). ∎

Appendix C Pooling-and-broadcasting from Partitions

Orbits of an equivariant linear map $\mathsf{W}^{M\to M^{\prime}}:{\mathbf{X}}_{(\delta_{1},\ldots,\delta_{M})}\mapsto{\mathbf{X}}_{(\delta^{\prime}_{1},\ldots,\delta^{\prime}_{M^{\prime}})}$ are partitions of the set $\{\delta_{1},\ldots,\delta_{M},\delta^{\prime}_{1},\ldots,\delta^{\prime}_{M^{\prime}}\}$ of input and output node indices, where any subset of the partition contains at most one input and one output index, as described in Appendix B. Note that for the partitions of interest, subsets have either size one, with a single input or output index, or size two, with a pair of input and output indices. We map each partition to the corresponding pooling-and-broadcasting operation:

• (a)

input indices in subsets of size one are pooled over,

• (b)

input and output indices in the same subsets are mapped onto each other,

• (c)

output indices in subsets of size one are broadcasted over.

Example 4**.**

Consider the function $\mathsf{W}^{2\to 1}$ that maps faces of size two (edges) to faces of size one (nodes). We introduced this map, e.g., in Section 3.1. From Eq. 11 $\mathsf{W}^{2\to 1}$ has three orbits. Let $\delta_{1},\delta_{2}$ be input node indices and $\delta^{\prime}_{1}$ the output node index. Following the rules outlined above we get: $\{\{\delta_{1}\},\{\delta_{2}\},\{\delta^{\prime}_{1}\}\}$ corresponds to pooling both edge dimensions and broadcasting the result to the output nodes, $\{\{\delta_{1},\delta^{\prime}_{1}\},\{\delta_{2}\}\}$ corresponds to pooling the edge second dimension and mapping the resulting one-dimensional vector to output nodes, finally $\{\{\delta_{2},\delta^{\prime}_{1}\},\{\delta_{1}\}\}$ corresponds to pooling the edge first dimension and mapping the resulting one-dimensional vector to output nodes.

Appendix D More on Abstract Polytopes

The structure of an abstract polytope is encoded using a graded partially ordered set (poset). A poset is a set equipped with a partial order that enables transitive comparison of certain members. A poset $\Pi$ is graded if there exists a rank function $\operatorname{rank}:\Pi\mapsto\mathbb{N}$ satisfying the following constraints:

[TABLE]

An abstract polytope is a set $\Pi$ of partially ordered faces of different dimension. In a geometric realisation, the partial order is naturally defined by the inclusion of a lower-dimensional face in a higher dimensional one (e.g., an edge that appears in a face of a cube). Fig. 4 shows the partial order for a cube, where we continue to use a set of nodes to identify a face.

We can define the incidence structure similar to simplicial complexes. For example, we may assume that two faces $\bm{\delta},\bm{\delta}^{\prime}\in\Pi$ of different dimension (rank) are incident iff $\bm{\delta}<\bm{\delta}^{\prime}$, or $\bm{\delta}>\bm{\delta}^{\prime}$. Similarly, we may assume that two faces $\bm{\delta},\bm{\delta}^{\prime}$ of the same dimension $d$ are incident iff there is a face $\bm{\delta}^{\prime\prime}$ of dimension $d-1$ incident to both of them $\bm{\delta}^{\prime\prime}<\bm{\delta},\bm{\delta}^{\prime}$. Note that if the polytope is irregular, faces of similar rank may have different sizes – e.g., consider the soccer ball where pentagons and hexagons have the same rank.

Appendix E Additional Results for Higher Order Geometric Structures

E.1 Additional Symmetry of Undirected Faces

When counting the number of unique pooling and broadcasting operations, we assumed that ${\mathbf{X}}_{\bm{\delta}^{(M)}}={\mathbf{X}}_{(\delta_{1},\ldots,\delta_{M})}\neq{\mathbf{X}}_{(\pi\cdot\delta_{1},\ldots,\pi\cdot\delta_{M})}$. However, if $\bm{\delta}^{(M)}$ is an undirected face, the face-vector is invariant to permutation of its nodes. This symmetry reduces the number of independent parameters in Eq. 10. Furthermore, if we enforce the same symmetry on the output face-vector, some of the weights need to be tied together. In particular, there is only one vector of faces of size $m$ that can be extracted through pooling for each value of $0\leq m\leq\min(M,M^{\prime})$. Similarly, all possible ways of broadcasting the pooled face-vector to the target face-vector have to carry the same parameter in order to restore symmetry of the output. Thus, in comparison to Eq. 11, for the symmetric input and symmetric output case, the degrees of freedom of the equivariant map are significantly reduced:

[TABLE]

Example 5** (Symmetric node-adjacency tensors).**

Consider a similar setup, but for the case of undirected faces. In this case we map a symmetric input into a symmetric output. From Eq. 13, where we use now Eq. 30 for the symmetric case, we get

[TABLE]

Note that we have omitted the multiplicity coefficients $\kappa_{m,m^{\prime}}$, assuming that all faces of a given dimension are equal.

E.2 Representation and Expressiveness

Consider a simplicial complex or polytope where faces of particular dimension have associated attributes. This information may be directly represented using face-vectors $\bigcupdot_{m=1}^{M}{\mathbf{X}}_{\bm{\delta}^{(m)}}$. Alternatively, we may only use the largest face $\bm{\delta}^{(M)}$, and broadcast all lower dimensional data to the maximal face-vectors. This gives an equivalent representation

[TABLE]

that resembles having multiple channels, indexed by $k$.

Yet another alternative is to use an incidence tensor ${\mathbf{X}}_{\bm{\delta}_{1},\ldots,\bm{\delta}_{M}\mid\Sigma}\cong\bigcupdot_{m=1}^{M}\kappa_{m}{\mathbf{X}}_{\bm{\delta}^{(m)}}$ that decomposes into face vectors according to Theorem 4.1. We have similarly diverse alternatives for the “output” of an equivariant map. Observe that the corresponding equivariant maps have the same expressiveness up to change in the number of channels. This is because we could produce the same pool-broadcast features of Eq. 10 across different representations.

An example is the node-edge incidence matrix discussed in Section 3.1, where node features are broadcasted across all edges incident to each node and whose equivariant layer has the same expressiveness as the layer for the corresponding node-node matrix.

E.3 Non-Linear Layers and Relaxation of the Symmetry Group

Non-linear layers, where the sparsity of the output is controlled by an input-dependent mask as in Eq. 5, are easily generalized to arbitrary incidence tensors.

Similarly, as discussed in Section 3.3 for a node-edge incidence matrix, one can consider independent permutation for each dimension of the incidence tensor. Let $N_{m}=|\{\bm{\delta}^{(m)}\}|$ denote the number of faces of size $m$, so that $N_{1}=N$. Consider the action of $\bm{\pi}=(\pi^{1}\ldots,\pi^{D})\in\mathcal{S}_{N_{|\bm{\delta}_{1}|}}\times\ldots\times\mathcal{S}_{N_{|\bm{\delta}_{D}|}}$ on ${\mathbf{X}}_{\bm{\delta}_{1},\ldots,\bm{\delta}_{D}}$:

[TABLE]

where $\pi^{d}\cdot\bm{\delta}_{d}$ is one of $N_{|\bm{\delta}_{d}|}!$ permutations of these faces. The corresponding equivariant layer introduced in (Hartford et al., 2018) has $\tau=2^{D}$ unique parameters.