Embed Me If You Can: A Geometric Perceptron

TL;DR

This paper introduces the multilayer geometric perceptron (MLGP), a model that leverages geometric embeddings and Clifford algebra to improve 3D shape classification, offering interpretability and robustness without complex activation functions.

Contribution

The paper extends the MLHP to MLGP, incorporating geometric neurons consistent with 3D geometry, enabling rotation and translation equivariance, and demonstrating superior performance on 3D shape classification tasks.

Findings

MLGP outperforms vanilla MLP on 3D shape classification.

MLGP requires no activation functions in hidden layers beyond embedding.

MLGP is more robust to noise in data.

Abstract

Solving geometric tasks involving point clouds by using machine learning is a challenging problem. Standard feed-forward neural networks combine linear or, if the bias parameter is included, affine layers and activation functions. Their geometric modeling is limited, which motivated the prior work introducing the multilayer hypersphere perceptron (MLHP). Its constituent part, i.e., the hypersphere neuron, is obtained by applying a conformal embedding of Euclidean space. By virtue of Clifford algebra, it can be implemented as the Cartesian dot product of inputs and weights. If the embedding is applied in a manner consistent with the dimensionality of the input space geometry, the decision surfaces of the model units become combinations of hyperspheres and make the decision-making process geometrically interpretable for humans. Our extension of the MLHP model, the multilayer geometric perceptron (MLGP), and its respective layer units, i.e., geometric neurons, are consistent with the 3D geometry and provide a geometric handle of the learned coefficients. In particular, the geometric neuron activations are isometric in 3D, which is necessary for rotation and translation equivariance. When classifying the 3D Tetris shapes, we quantitatively show that our model requires no activation function in the hidden layers other than the embedding to outperform the vanilla multilayer perceptron. In the presence of noise in the data, our model is also superior to the MLHP.