Quantum Annealing based Feature Selection in Machine Learning

TL;DR

This paper explores using quantum annealing to perform feature selection in machine learning by maximizing mutual information, demonstrating potential improvements over classical methods especially for datasets with specific information characteristics.

Contribution

It introduces a MIQUBO formulation for quantum annealing-based feature selection, enabling efficient identification of optimal feature sets for maximizing mutual information.

Findings

Quantum annealing can effectively select features that maximize mutual information.

The approach shows significant improvements over classical methods for certain datasets.

Application to used excavator price forecasting demonstrates real-world utility.

Abstract

Feature selection is crucial for enhancing the accuracy and efficiency of machine learning (ML) models. This work investigates the utility of quantum annealing for the feature selection process in an ML-pipeline, used for maximizing the mutual information (MI) or conditional mutual information (CMI) of the underlying feature space. Calculating the optimal set of features that maximize the MI or CMI is computationally intractable for large datasets on classical computers, even with approximative methods. This study employs a Mutual Information Quadratic Unconstrained Binary Optimization (MIQUBO) formulation, enabling its solution on a quantum annealer. We demonstrate the capability of this approach to identify the best feature combinations that maximize the MI or CMI. To showcase its real-world applicability, we solve the MIQUBO problem to forecast the prices of used excavators. Our results demonstrate that for datasets with a small MI concentration the MIQUBO approach can provide a significant improvement over MI-only based approaches, dependent on the dimension of the selected feature space.