Improving Set Function Approximation with Quasi-Arithmetic Neural Networks

TL;DR

This paper introduces Quasi-Arithmetic Neural Networks (QUANNs), a novel learnable aggregation method for set data that enhances expressivity and transferability, supported by theoretical universality and superior empirical performance.

Contribution

The paper proposes the Neuralized Kolmogorov Mean (NKM) and QUANNs, advancing set function approximation with a trainable, invertible aggregation framework that improves over fixed pooling methods.

Findings

QUANNs outperform state-of-the-art baselines on various benchmarks.

QUANNs learn transferable embeddings effective even outside set tasks.

Theoretical proof of universality for broad set-function classes.

Abstract

Sets represent a fundamental abstraction across many types of data. To handle the unordered nature of set-structured data, models such as DeepSets and PointNet rely on fixed, non-learnable pooling operations (e.g., sum or max) -- a design choice that can hinder the transferability of learned embeddings and limits model expressivity. More recently, learnable aggregation functions have been proposed as more expressive alternatives. In this work, we advance this line of research by introducing the Neuralized Kolmogorov Mean (NKM) -- a novel, trainable framework for learning a generalized measure of central tendency through an invertible neural function. We further propose quasi-arithmetic neural networks (QUANNs), which incorporate the NKM as a learnable aggregation function. We provide a theoretical analysis showing that, QUANNs are universal approximators for a broad class of common set-function decompositions and, thanks to their invertible neural components, learn more structured latent representations. Empirically, QUANNs outperform state-of-the-art baselines across diverse benchmarks, while learning embeddings that transfer effectively even to tasks that do not involve sets.