Max-Entropy Feed-Forward Clustering Neural Network

TL;DR

This paper introduces a novel clustering method using a max-entropy principle integrated into feed-forward neural networks, which models sample distributions for improved clustering performance.

Contribution

It proposes a new clustering approach that incorporates entropy-based principles into neural networks, enhancing clustering accuracy over existing methods.

Findings

Outperforms popular clustering baselines on six UCI datasets

Uses purity as a measurement to evaluate clustering quality

Demonstrates the effectiveness of entropy-based neural clustering

Abstract

The outputs of non-linear feed-forward neural network are positive, which could be treated as probability when they are normalized to one. If we take Entropy-Based Principle into consideration, the outputs for each sample could be represented as the distribution of this sample for different clusters. Entropy-Based Principle is the principle with which we could estimate the unknown distribution under some limited conditions. As this paper defines two processes in Feed-Forward Neural Network, our limited condition is the abstracted features of samples which are worked out in the abstraction process. And the final outputs are the probability distribution for different clusters in the clustering process. As Entropy-Based Principle is considered into the feed-forward neural network, a clustering method is born. We have conducted some experiments on six open UCI datasets, comparing with a few baselines and applied purity as the measurement . The results illustrate that our method outperforms all the other baselines that are most popular clustering methods.