A branch-and-bound feature selection algorithm for U-shaped cost functions

TL;DR

This paper introduces a novel branch-and-bound feature selection algorithm that exploits lattice properties and U-shaped cost functions, demonstrating superior performance over heuristics like SFFS in pattern recognition tasks.

Contribution

The paper proposes a new branch-and-bound algorithm leveraging lattice structure and U-shaped curves, advancing feature selection methods with proven efficiency and effectiveness.

Findings

Outperforms SFFS in accuracy on public datasets.

Achieves similar or better results with less computational time.

Utilizes new lattice properties for efficient search exploration.

Abstract

This paper presents the formulation of a combinatorial optimization problem with the following characteristics: i.the search space is the power set of a finite set structured as a Boolean lattice; ii.the cost function forms a U-shaped curve when applied to any lattice chain. This formulation applies for feature selection in the context of pattern recognition. The known approaches for this problem are branch-and-bound algorithms and heuristics, that explore partially the search space. Branch-and-bound algorithms are equivalent to the full search, while heuristics are not. This paper presents a branch-and-bound algorithm that differs from the others known by exploring the lattice structure and the U-shaped chain curves of the search space. The main contribution of this paper is the architecture of this algorithm that is based on the representation and exploration of the search space by new lattice properties proven here. Several experiments, with well known public data, indicate the superiority of the proposed method to SFFS, which is a popular heuristic that gives good results in very short computational time. In all experiments, the proposed method got better or equal results in similar or even smaller computational time.