Tuning structure learning algorithms with out-of-sample and resampling strategies

TL;DR

This paper introduces OTSL, a new hyperparameter tuning method for structure learning algorithms that uses out-of-sample and resampling strategies to improve accuracy on synthetic and real datasets.

Contribution

The paper presents OTSL, a novel hyperparameter tuning approach specifically designed for structure learning algorithms, enhancing their performance through out-of-sample and resampling techniques.

Findings

OTSL improves graphical accuracy over state-of-the-art methods.

OTSL is effective on both synthetic and real datasets.

The approach adapts to different data characteristics and algorithms.

Abstract

One of the challenges practitioners face when applying structure learning algorithms to their data involves determining a set of hyperparameters; otherwise, a set of hyperparameter defaults is assumed. The optimal hyperparameter configuration often depends on multiple factors, including the size and density of the usually unknown underlying true graph, the sample size of the input data, and the structure learning algorithm. We propose a novel hyperparameter tuning method, called the Out-of-sample Tuning for Structure Learning (OTSL), that employs out-of-sample and resampling strategies to estimate the optimal hyperparameter configuration for structure learning, given the input data set and structure learning algorithm. Synthetic experiments show that employing OTSL as a means to tune the hyperparameters of hybrid and score-based structure learning algorithms leads to improvements in graphical accuracy compared to the state-of-the-art. We also illustrate the applicability of this approach to real datasets from different disciplines.