LLM-Driven Reasoning for Constraint-Aware Feature Selection in Industrial Systems

TL;DR

This paper introduces MoFA, a reasoning-based framework leveraging large language models for constraint-aware feature selection in industrial systems, improving model performance and efficiency with limited labeled data.

Contribution

The paper presents MoFA, a novel model-driven, reasoning-based feature selection method that incorporates semantic and quantitative feature information for industrial applications.

Findings

MoFA improves prediction accuracy in real-world industrial tasks.

MoFA reduces feature set complexity while maintaining performance.

MoFA discovers high-value feature interactions enhancing online engagement.

Abstract

Feature selection is a crucial step in large-scale industrial machine learning systems, directly affecting model accuracy, efficiency, and maintainability. Traditional feature selection methods rely on labeled data and statistical heuristics, making them difficult to apply in production environments where labeled data are limited and multiple operational constraints must be satisfied. To address this, we propose Model Feature Agent (MoFA), a model-driven framework that performs sequential, reasoning-based feature selection using both semantic and quantitative feature information. MoFA incorporates feature definitions, importance scores, correlations, and metadata (e.g., feature groups or types) into structured prompts and selects features through interpretable, constraint-aware reasoning. We evaluate MoFA in three real-world industrial applications: (1) True Interest and Time-Worthiness Prediction, where it improves accuracy while reducing feature group complexity, (2) Value Model Enhancement, where it discovers high-order interaction terms that yield substantial engagement gains in online experiments, and (3) Notification Behavior Prediction, where it selects compact, high-value feature subsets that improve both model accuracy and inference efficiency. Together, these results demonstrate the practicality and effectiveness of LLM-based reasoning for feature selection in real production systems.