Data Curation Matters: Model Collapse and Spurious Shift Performance Prediction from Training on Uncurated Text Embeddings

TL;DR

Training models on uncurated text embeddings from raw data often causes model collapse, leading to poor predictions and spurious performance metrics, emphasizing the importance of proper data curation and evaluation.

Contribution

This paper identifies model collapse as a common failure mode in TE-based training and introduces metrics to assess TE quality and its impact on model performance.

Findings

Model collapse occurs consistently when training on TE-derived data.

TE quality significantly affects downstream learning outcomes.

Model collapse can artificially inflate performance metrics.

Abstract

Training models on uncurated Text Embeddings (TEs) derived from raw tabular data can lead to a severe failure mode known as model collapse, where predictions converge to a single class regardless of input. By comparing models trained with identical hyper-parameter configurations on both raw tabular data and their TE-derived counterparts, we find that collapse is a consistent failure mode in the latter setting. We introduce a set of metrics that capture the extent of model collapse, offering a new perspective on TE quality as a proxy for data curation. Our results reveal that TE alone does not effectively function as a curation layer - and that their quality significantly influences downstream learning. More insidiously, we observe that the presence of model collapse can yield artificially inflated and spurious Accuracy-on-the-Line correlation. These findings highlight the need for more nuanced curation and evaluation of embedding-based representations, particularly in out-of-distribution settings.