Symbolic Snapshot Ensembles

TL;DR

This paper introduces a novel ensemble method for inductive logic programming that combines multiple hypotheses from a single training run using a description length weighting scheme, improving accuracy efficiently.

Contribution

It proposes a new ensemble approach for ILP that avoids multiple training runs by leveraging intermediate hypotheses and a description length-based weighting scheme.

Findings

Improves predictive accuracy by 4% on benchmarks.

Achieves this with less than 1% additional computational cost.

Effective on game playing and visual reasoning tasks.

Abstract

Inductive logic programming (ILP) is a form of logical machine learning. Most ILP algorithms learn a single hypothesis from a single training run. Ensemble methods train an ILP algorithm multiple times to learn multiple hypotheses. In this paper, we train an ILP algorithm only once and save intermediate hypotheses. We then combine the hypotheses using a minimum description length weighting scheme. Our experiments on multiple benchmarks, including game playing and visual reasoning, show that our approach improves predictive accuracy by 4% with less than 1% computational overhead.