Neural Networks for Predicting Algorithm Runtime Distributions

TL;DR

This paper introduces neural networks that jointly predict the parameters of runtime distributions for stochastic algorithms, significantly improving accuracy over previous methods, especially with limited training data.

Contribution

It demonstrates that neural networks can effectively learn RTD parameters by directly optimizing likelihood, advancing the state-of-the-art in RTD prediction.

Findings

Neural networks outperform previous RTD prediction methods.

Joint learning of RTD parameters improves prediction accuracy.

Effective even with few runtime observations per training instance.

Abstract

Many state-of-the-art algorithms for solving hard combinatorial problems in artificial intelligence (AI) include elements of stochasticity that lead to high variations in runtime, even for a fixed problem instance. Knowledge about the resulting runtime distributions (RTDs) of algorithms on given problem instances can be exploited in various meta-algorithmic procedures, such as algorithm selection, portfolios, and randomized restarts. Previous work has shown that machine learning can be used to individually predict mean, median and variance of RTDs. To establish a new state-of-the-art in predicting RTDs, we demonstrate that the parameters of an RTD should be learned jointly and that neural networks can do this well by directly optimizing the likelihood of an RTD given runtime observations. In an empirical study involving five algorithms for SAT solving and AI planning, we show that neural networks predict the true RTDs of unseen instances better than previous methods, and can even do so when only few runtime observations are available per training instance.