LieGG: Studying Learned Lie Group Generators

TL;DR

This paper introduces a method to extract and evaluate symmetries learned by neural networks, revealing how network architecture influences the ability to learn and represent symmetries without prior symmetry assumptions.

Contribution

We propose a novel technique to explicitly retrieve learned Lie group generators from neural networks, enabling analysis of symmetry learning without prior knowledge.

Findings

Networks can learn symmetries across various architectures

The quality of learned symmetries depends on depth and parameters

Symmetry learning generalizes across different network configurations

Abstract

Symmetries built into a neural network have appeared to be very beneficial for a wide range of tasks as it saves the data to learn them. We depart from the position that when symmetries are not built into a model a priori, it is advantageous for robust networks to learn symmetries directly from the data to fit a task function. In this paper, we present a method to extract symmetries learned by a neural network and to evaluate the degree to which a network is invariant to them. With our method, we are able to explicitly retrieve learned invariances in a form of the generators of corresponding Lie-groups without prior knowledge of symmetries in the data. We use the proposed method to study how symmetrical properties depend on a neural network's parameterization and configuration. We found that the ability of a network to learn symmetries generalizes over a range of architectures. However, the quality of learned symmetries depends on the depth and the number of parameters.