# On Deep Set Learning and the Choice of Aggregations

## TL;DR

This paper explores the impact of different aggregation functions in Deep Set learning, demonstrating that learnable aggregations improve performance, reduce hyper-parameter sensitivity, and enhance out-of-distribution generalization.

## Contribution

It introduces and empirically evaluates learnable aggregation functions as alternatives to standard ones in Deep Set architectures.

## Key findings

- Learnable aggregations outperform fixed ones in accuracy.
- Deep Set networks are highly sensitive to aggregation choice.
- Learnable aggregations improve out-of-distribution generalization.

## Abstract

Recently, it has been shown that many functions on sets can be represented by sum decompositions. These decompositons easily lend themselves to neural approximations, extending the applicability of neural nets to set-valued inputs---Deep Set learning. This work investigates a core component of Deep Set architecture: aggregation functions. We suggest and examine alternatives to commonly used aggregation functions, including learnable recurrent aggregation functions. Empirically, we show that the Deep Set networks are highly sensitive to the choice of aggregation functions: beyond improved performance, we find that learnable aggregations lower hyper-parameter sensitivity and generalize better to out-of-distribution input size.