Dilemma First Search for Effortless Optimization of NP-Hard Problems

TL;DR

This paper introduces Dilemma First Search (DFS), a simple yet effective method for optimizing NP-hard problems by leveraging decision heuristics, demonstrated on knapsack and decision tree inference.

Contribution

The paper presents a new theoretical framework and a novel search method, DFS, that improves optimization of NP-hard problems without extensive problem-specific information.

Findings

DFS outperforms greedy solutions in fewer iterations

Demonstrated on knapsack and decision trees

Achieves better optimization with less computational effort

Abstract

To tackle the exponentiality associated with NP-hard problems, two paradigms have been proposed. First, Branch & Bound, like Dynamic Programming, achieve efficient exact inference but requires extensive information and analysis about the problem at hand. Second, meta-heuristics are easier to implement but comparatively inefficient. As a result, a number of problems have been left unoptimized and plain greedy solutions are used. We introduce a theoretical framework and propose a powerful yet simple search method called Dilemma First Search (DFS). DFS exploits the decision heuristic needed for the greedy solution for further optimization. DFS is useful when it is hard to design efficient exact inference. We evaluate DFS on two problems: First, the Knapsack problem, for which efficient algorithms exist, serves as a toy example. Second, Decision Tree inference, where state-of-the-art algorithms rely on the greedy or randomness-based solutions. We further show that decision trees benefit from optimizations that are performed in a fraction of the iterations required by a random-based search.