Current Symmetry Group Equivariant Convolution Frameworks for Representation Learning

TL;DR

This paper reviews symmetry group equivariant convolution frameworks in geometric deep learning, emphasizing their mathematical foundations, types, and applications to improve robustness in non-Euclidean data representations.

Contribution

It categorizes and analyzes various symmetry-equivariant convolution methods, linking group theory to practical deep learning models on complex geometric data.

Findings

Classification of convolution types: regular, steerable, PDE-based

Analysis of symmetries in input spaces and representations

Discussion of datasets, applications, and future research directions

Abstract

Euclidean deep learning is often inadequate for addressing real-world signals where the representation space is irregular and curved with complex topologies. Interpreting the geometric properties of such feature spaces has become paramount in obtaining robust and compact feature representations that remain unaffected by nontrivial geometric transformations, which vanilla CNNs cannot effectively handle. Recognizing rotation, translation, permutation, or scale symmetries can lead to equivariance properties in the learned representations. This has led to notable advancements in computer vision and machine learning tasks under the framework of geometric deep learning, as compared to their invariant counterparts. In this report, we emphasize the importance of symmetry group equivariant deep learning models and their realization of convolution-like operations on graphs, 3D shapes, and non-Euclidean spaces by leveraging group theory and symmetry. We categorize them as regular, steerable, and PDE-based convolutions and thoroughly examine the inherent symmetries of their input spaces and ensuing representations. We also outline the mathematical link between group convolutions or message aggregation operations and the concept of equivariance. The report also highlights various datasets, their application scopes, limitations, and insightful observations on future directions to serve as a valuable reference and stimulate further research in this emerging discipline.