Knowledge-Informed Automatic Feature Extraction via Collaborative Large Language Model Agents

TL;DR

Rogue One is a multi-agent LLM framework that enhances feature extraction for tabular data by integrating external knowledge, qualitative feedback, and iterative collaboration, leading to more meaningful and powerful features.

Contribution

It introduces a decentralized multi-agent system with qualitative feedback and knowledge retrieval to improve automatic feature extraction beyond existing monolithic LLM approaches.

Findings

Outperforms state-of-the-art methods on multiple datasets

Generates semantically meaningful and interpretable features

Identifies novel hypotheses like potential biomarkers

Abstract

The performance of machine learning models on tabular data is critically dependent on high-quality feature engineering. While Large Language Models (LLMs) have shown promise in automating feature extraction (AutoFE), existing methods are often limited by monolithic LLM architectures, simplistic quantitative feedback, and a failure to systematically integrate external domain knowledge. This paper introduces Rogue One, a novel, LLM-based multi-agent framework for knowledge-informed automatic feature extraction. Rogue One operationalizes a decentralized system of three specialized agents-Scientist, Extractor, and Tester-that collaborate iteratively to discover, generate, and validate predictive features. Crucially, the framework moves beyond primitive accuracy scores by introducing a rich, qualitative feedback mechanism and a "flooding-pruning" strategy, allowing it to dynamically balance feature exploration and exploitation. By actively incorporating external knowledge via an integrated retrieval-augmented (RAG) system, Rogue One generates features that are not only statistically powerful but also semantically meaningful and interpretable. We demonstrate that Rogue One significantly outperforms state-of-the-art methods on a comprehensive suite of 19 classification and 9 regression datasets. Furthermore, we show qualitatively that the system surfaces novel, testable hypotheses, such as identifying a new potential biomarker in the myocardial dataset, underscoring its utility as a tool for scientific discovery.