Understanding Algorithm Performance on an Oversubscribed Scheduling Application

TL;DR

This paper analyzes the performance of algorithms on the AFSCN scheduling problem, revealing that search space structure influences algorithm effectiveness and leading to a new, improved algorithm.

Contribution

It links search space characteristics to algorithm performance and introduces a novel algorithm that outperforms previous methods.

Findings

Plateaus dominate the search space, favoring larger solution changes.

Random exploration is critical due to lack of gradient information.

The new algorithm outperforms previous best algorithms.

Abstract

The best performing algorithms for a particular oversubscribed scheduling application, Air Force Satellite Control Network (AFSCN) scheduling, appear to have little in common. Yet, through careful experimentation and modeling of performance in real problem instances, we can relate characteristics of the best algorithms to characteristics of the application. In particular, we find that plateaus dominate the search spaces (thus favoring algorithms that make larger changes to solutions) and that some randomization in exploration is critical to good performance (due to the lack of gradient information on the plateaus). Based on our explanations of algorithm performance, we develop a new algorithm that combines characteristics of the best performers; the new algorithms performance is better than the previous best. We show how hypothesis driven experimentation and search modeling can both explain algorithm performance and motivate the design of a new algorithm.