Multiset Transformer: Advancing Representation Learning in Persistence Diagrams

TL;DR

The paper introduces Multiset Transformer, a neural network leveraging attention mechanisms for multisets, with theoretical guarantees of permutation invariance, improved efficiency, and superior performance in persistence diagram representation learning.

Contribution

It presents the first neural network with attention designed for multisets, combining multiset-enhanced attentions with a pool-decomposition scheme for better efficiency and effectiveness.

Findings

Outperforms existing neural methods in persistence diagram tasks.

Offers theoretical guarantees of permutation invariance.

Reduces computational and spatial complexity compared to Set Transformer.

Abstract

To improve persistence diagram representation learning, we propose Multiset Transformer. This is the first neural network that utilizes attention mechanisms specifically designed for multisets as inputs and offers rigorous theoretical guarantees of permutation invariance. The architecture integrates multiset-enhanced attentions with a pool-decomposition scheme, allowing multiplicities to be preserved across equivariant layers. This capability enables full leverage of multiplicities while significantly reducing both computational and spatial complexity compared to the Set Transformer. Additionally, our method can greatly benefit from clustering as a preprocessing step to further minimize complexity, an advantage not possessed by the Set Transformer. Experimental results demonstrate that the Multiset Transformer outperforms existing neural network methods in the realm of persistence diagram representation learning.