Iterative Augmentation with Summarization Refinement (IASR) Evaluation for Unstructured Survey data Modeling and Analysis

TL;DR

This paper proposes a new evaluation framework for LLM-based text augmentation, focusing on semantic consistency and drift, and demonstrates its effectiveness in improving topic modeling in NLP tasks.

Contribution

It introduces a scalable, iterative augmentation evaluation method with summarization refinement, enhancing semantic fidelity and diversity in NLP data augmentation.

Findings

GPT-3.5 Turbo achieved optimal balance of fidelity and efficiency

400% increase in topic granularity in real-world application

Complete elimination of topic overlaps in topic modeling

Abstract

Text data augmentation is a widely used strategy for mitigating data sparsity in natural language processing (NLP), particularly in low-resource settings where limited samples hinder effective semantic modeling. While augmentation can improve input diversity and downstream interpretability, existing techniques often lack mechanisms to ensure semantic preservation during large-scale or iterative generation, leading to redundancy and instability. This work introduces a principled evaluation framework for large language model (LLM) based text augmentation, comprising two components: (1) Scalability Analysis, which measures semantic consistency as augmentation volume increases, and (2) Iterative Augmentation with Summarization Refinement (IASR), which evaluates semantic drift across recursive paraphrasing cycles. Empirical evaluations across state-of-the-art LLMs show that GPT-3.5 Turbo achieved the best balance of semantic fidelity, diversity, and generation efficiency. Applied to a real-world topic modeling task using BERTopic with GPT-enhanced few-shot labeling, the proposed approach results in a 400% increase in topic granularity and complete elimination of topic overlaps. These findings validated the utility of the proposed frameworks for structured evaluation of LLM-based augmentation in practical NLP pipelines.