TAB-AUDIT: Detecting AI-Fabricated Scientific Tables via Multi-View Likelihood Mismatch

TL;DR

This paper introduces TAB-AUDIT, a novel method for detecting AI-generated fabricated scientific tables, using a new benchmark dataset and features that reveal systematic differences between real and fabricated tables, achieving high accuracy.

Contribution

The paper presents the first systematic study and detection framework for AI-generated fabricated scientific tables, along with a new benchmark dataset FabTab and discriminative features.

Findings

RandomForest classifier achieves 0.987 AUROC in-domain

Features based on table likelihood mismatch are highly effective

Systematic differences exist between real and fabricated tables

Abstract

AI-generated fabricated scientific manuscripts raise growing concerns with large-scale breaches of academic integrity. In this work, we present the first systematic study on detecting AI-generated fabricated scientific tables in empirical NLP papers, as information in tables serve as critical evidence for claims. We construct FabTab, the first benchmark dataset of fabricated manuscripts with tables, comprising 1,173 AI-generated papers and 1,215 human-authored ones in empirical NLP. Through a comprehensive analysis, we identify systematic differences between fabricated and real tables and operationalize them into a set of discriminative features within the TAB-AUDIT framework. The key feature, within-table mismatch, captures the perplexity gap between a table's skeleton and its numerical content. Experimental results show that RandomForest built on these features significantly outperform prior state-of-the-art methods, achieving 0.987 AUROC in-domain and 0.883 AUROC out-of-domain. Our findings highlight experimental tables as a critical forensic signal for detecting AI-generated scientific fraud and provide a new benchmark for future research.