The observer-assisted method for adjusting hyper-parameters in deep learning algorithms

TL;DR

This paper introduces an observer-assisted approach for tuning hyper-parameters in deep learning by modeling algorithm responses through an external agent, enabling iterative optimization towards optimal performance.

Contribution

It proposes a novel external observer-based architecture that models and predicts deep learning performance to guide hyper-parameter tuning.

Findings

The method effectively predicts DL performance based on hyper-parameters.

It enables incremental optimization with controlled steps.

The approach improves hyper-parameter tuning efficiency.

Abstract

This paper presents a concept of a novel method for adjusting hyper-parameters in Deep Learning (DL) algorithms. An external agent-observer monitors a performance of a selected Deep Learning algorithm. The observer learns to model the DL algorithm using a series of random experiments. Consequently, it may be used for predicting a response of the DL algorithm in terms of a selected quality measurement to a set of hyper-parameters. This allows to construct an ensemble composed of a series of evaluators which constitute an observer-assisted architecture. The architecture may be used to gradually iterate towards to the best achievable quality score in tiny steps governed by a unit of progress. The algorithm is stopped when the maximum number of steps is reached or no further progress is made.