Automated Text Identification Using CNN and Training Dynamics

TL;DR

This paper analyzes training dynamics of CNNs on text data, revealing sample difficulty regions and showing that selective training on ambiguous samples enhances out-of-distribution generalization.

Contribution

It introduces a data mapping approach to characterize training samples and demonstrates the benefit of training on ambiguous examples for better generalization.

Findings

Identification of three sample difficulty regions: easy, ambiguous, hard

Training on ambiguous samples improves out-of-distribution performance

Insights into sample behavior during CNN training on text datasets

Abstract

We used Data Maps to model and characterize the AuTexTification dataset. This provides insights about the behaviour of individual samples during training across epochs (training dynamics). We characterized the samples across 3 dimensions: confidence, variability and correctness. This shows the presence of 3 regions: easy-to-learn, ambiguous and hard-to-learn examples. We used a classic CNN architecture and found out that training the model only on a subset of ambiguous examples improves the model's out-of-distribution generalization.