When and How to Canonize: A Generalization Perspective

TL;DR

This paper develops a theoretical framework to analyze the generalization error of canonization methods in symmetric data processing, comparing their bounds and demonstrating the advantages of Hilbert curve canonization in point clouds.

Contribution

It introduces a formal analysis of canonization strategies, establishing bounds and optimality conditions, and provides the first theoretical justification for Hilbert curve serialization in point cloud models.

Findings

Canonized models have error bounds at best equal to invariant models and at worst equal to non-invariant baselines.

Optimal canonizations can attain the best error bounds, while poor canonizations can match non-invariant bounds.

Hilbert curve canonization exhibits polynomial growth in covering numbers, unlike exponential growth for lexicographical sorting.

Abstract

While invariant architectures are standard for processing symmetric data, there is growing interest in achieving invariance by applying group averaging or canonization to non-invariant backbones. However, the theoretical generalization properties of these alternative strategies remain poorly understood. We introduce a theoretical framework to analyze the generalization error of these methods by bounding their covering numbers. We establish a rigorous generalization hierarchy: the error bounds of canonized models are at best equal to the error bounds of structurally invariant and group-averaged models, and at worst equal to the bounds of non-invariant baselines. Furthermore, we show that there exist optimal canonizations which attain the optimal error bounds, and poor canonizations which attain the non-invariant error bounds, and that this depends on the regularity of the canonization. Finally, applying this framework to permutation groups in point cloud processing, we rigorously prove that the covering number of lexicographical sorting grows exponentially with point cloud dimension, whereas Hilbert curve canonization guarantees polynomial growth. This provides the first formal theoretical justification for the empirical success of Hilbert curve serialization in state-of-the-art point cloud architectures. We conclude with experiments that support our theoretical claims. Code is available at https://github.com/yonatansverdlov/Canonization