SCAT: Second Chance Autoencoder for Textual Data

TL;DR

SCAT introduces a k-competitive autoencoder that enhances textual feature representation by focusing on top and bottom activations, leading to improved performance in classification, topic modeling, and visualization.

Contribution

The paper proposes the Second Chance Autoencoder (SCAT), a novel k-competitive learning method that emphasizes significant features for better textual data representation.

Findings

SCAT outperforms LDA, K-Sparse, NVCTM, and KATE in various tasks.

SCAT effectively captures well-representative features for topics.

The method improves classification accuracy and document visualization.

Abstract

We present a k-competitive learning approach for textual autoencoders named Second Chance Autoencoder (SCAT). SCAT selects the $k$ largest and smallest positive activations as the winner neurons, which gain the activation values of the loser neurons during the learning process, and thus focus on retrieving well-representative features for topics. Our experiments show that SCAT achieves outstanding performance in classification, topic modeling, and document visualization compared to LDA, K-Sparse, NVCTM, and KATE.