Capsule Network Projectors are Equivariant and Invariant Learners

TL;DR

This paper introduces CapsIE, a novel self-supervised architecture using Capsule Networks to learn invariant and equivariant representations, achieving state-of-the-art results on 3D rotation tasks with improved efficiency.

Contribution

The work presents a new CapsNet-based architecture with a specialized entropy minimization objective for invariant-equivariant learning, outperforming prior methods on equivariant tasks.

Findings

CapsIE achieves state-of-the-art on 3DIEBench rotation tasks.

CapsNets effectively learn complex, generalised representations.

The approach is more efficient with fewer parameters.

Abstract

Learning invariant representations has been the long-standing approach to self-supervised learning. However, recently progress has been made in preserving equivariant properties in representations, yet do so with highly prescribed architectures. In this work, we propose an invariant-equivariant self-supervised architecture that employs Capsule Networks (CapsNets), which have been shown to capture equivariance with respect to novel viewpoints. We demonstrate that the use of CapsNets in equivariant self-supervised architectures achieves improved downstream performance on equivariant tasks with higher efficiency and fewer network parameters. To accommodate the architectural changes of CapsNets, we introduce a new objective function based on entropy minimisation. This approach, which we name CapsIE (Capsule Invariant Equivariant Network), achieves state-of-the-art performance on the equivariant rotation tasks on the 3DIEBench dataset compared to prior equivariant SSL methods, while performing competitively against supervised counterparts. Our results demonstrate the ability of CapsNets to learn complex and generalised representations for large-scale, multi-task datasets compared to previous CapsNet benchmarks. Code is available at https://github.com/AberdeenML/CapsIE.