Surrogate Model Based Hyperparameter Tuning for Deep Learning with SPOT

TL;DR

This paper introduces a hyperparameter tuning method for deep learning models using surrogate models, implemented in R with accessible tools, demonstrated on MNIST with Keras/TensorFlow.

Contribution

It presents a practical, accessible approach to hyperparameter tuning in deep learning using surrogate models within the R environment.

Findings

Effective hyperparameter optimization demonstrated on MNIST dataset

Implementation is simple with few lines of R code

Accessible for practitioners using Keras/TensorFlow in R

Abstract

A surrogate model based hyperparameter tuning approach for deep learning is presented. This article demonstrates how the architecture-level parameters (hyperparameters) of deep learning models that were implemented in Keras/tensorflow can be optimized. The implementation of the tuning procedure is 100% accessible from R, the software environment for statistical computing. With a few lines of code, existing R packages (tfruns and SPOT) can be combined to perform hyperparameter tuning. An elementary hyperparameter tuning task (neural network and the MNIST data) is used to exemplify this approach