"FRAME: Forward Recursive Adaptive Model Extraction-A Technique for Advance Feature Selection"

TL;DR

FRAME is a hybrid feature selection method combining Forward Selection and Recursive Feature Elimination, improving predictive accuracy and interpretability across diverse datasets in machine learning applications.

Contribution

This paper introduces FRAME, a novel hybrid feature selection technique that enhances model performance and interpretability by combining forward selection with recursive elimination.

Findings

FRAME outperforms traditional methods like SelectKBest and Lasso Regression.

It achieves superior predictive performance on high-dimensional, noisy datasets.

FRAME is effective for applications requiring interpretable and accurate models, such as biomedical diagnostics.

Abstract

The challenges in feature selection, particularly in balancing model accuracy, interpretability, and computational efficiency, remain a critical issue in advancing machine learning methodologies. To address these complexities, this study introduces a novel hybrid approach, the Forward Recursive Adaptive Model Extraction Technique (FRAME), which combines Forward Selection and Recursive Feature Elimination (RFE) to enhance feature selection across diverse datasets. By combining the exploratory capabilities of Forward Selection with the refinement strengths of RFE, FRAME systematically identifies optimal feature subsets, striking a harmonious trade-off between experimentation and precision. A comprehensive evaluation of FRAME is conducted against traditional methods such as SelectKBest and Lasso Regression, using high-dimensional, noisy, and heterogeneous datasets. The results demonstrate that FRAME consistently delivers superior predictive performance based on downstream machine learning evaluation metrics. It efficiently performs dimensionality reduction with strong model performance, thus being especially useful for applications that need interpretable and accurate predictions, e.g., biomedical diagnostics. This research emphasizes the need to evaluate feature selection techniques on diverse datasets to test their robustness and generalizability. The results indicate that FRAME has great potential for further development, especially by incorporating deep learning frameworks for adaptive and real-time feature selection in dynamic settings. By advancing feature selection methodologies, FRAME offers a practical and effective solution to improve machine learning applications across multiple domains.