# Set Cross Entropy: Likelihood-based Permutation Invariant Loss Function for Probability Distributions

## TL;DR

This paper introduces Set Cross Entropy, a permutation-invariant loss function for neural networks that reconstruct sets without relying on specific network architectures or sequential algorithms, with applications in object reconstruction and rule learning.

## Contribution

The paper presents a novel likelihood-based loss function for set reconstruction that is permutation-invariant and does not depend on network topology or sequential processing.

## Key findings

- Effective in object reconstruction tasks
- Applicable to rule learning tasks
- Has a natural information-theoretic interpretation

## Abstract

We propose a permutation-invariant loss function designed for the neural networks reconstructing a set of elements without considering the order within its vector representation. Unlike popular approaches for encoding and decoding a set, our work does not rely on a carefully engineered network topology nor by any additional sequential algorithm. The proposed method, Set Cross Entropy, has a natural information-theoretic interpretation and is related to the metrics defined for sets. We evaluate the proposed approach in two object reconstruction tasks and a rule learning task.