Equivariant vs. Invariant Layers: A Comparison of Backbone and Pooling for Point Cloud Classification

TL;DR

This paper investigates how different pooling methods interact with permutation equivariant backbones in point cloud classification, revealing that pooling choices can significantly influence model performance, often more than backbone complexity.

Contribution

It provides a comprehensive analysis of the impact of various pooling layers on permutation equivariant backbones in point cloud classification, highlighting their importance and interaction.

Findings

Complex pooling improves simple backbones significantly.

Pooling benefits are more pronounced in low-data scenarios.

Pooling choice can outweigh backbone complexity in performance.

Abstract

Learning from set-structured data, such as point clouds, has gained significant attention from the machine learning community. Geometric deep learning provides a blueprint for designing effective set neural networks that preserve the permutation symmetry of set-structured data. Of our interest are permutation invariant networks, which are composed of a permutation equivariant backbone, permutation invariant global pooling, and regression/classification head. While existing literature has focused on improving equivariant backbones, the impact of the pooling layer is often overlooked. In this paper, we examine the interplay between permutation equivariant backbones and permutation invariant global pooling on three benchmark point cloud classification datasets. Our findings reveal that: 1) complex pooling methods, such as transport-based or attention-based poolings, can significantly boost the performance of simple backbones, but the benefits diminish for more complex backbones, 2) even complex backbones can benefit from pooling layers in low data scenarios, 3) surprisingly, the choice of pooling layers can have a more significant impact on the model's performance than adjusting the width and depth of the backbone, and 4) pairwise combination of pooling layers can significantly improve the performance of a fixed backbone. Our comprehensive study provides insights for practitioners to design better permutation invariant set neural networks. Our code is available at https://github.com/mint-vu/backbone_vs_pooling.