Data Classification with Dynamically Growing and Shrinking Neural Networks

TL;DR

This paper introduces a novel neural network architecture that dynamically grows and shrinks during training, using Monte Carlo tree search to optimize structure, significantly improving multivariate time series classification performance.

Contribution

The paper presents a new method for neural network architecture adaptation during training, combining dynamic structural modifications with Monte Carlo tree search for improved data-driven model construction.

Findings

Effective in multivariate time series classification

Demonstrates robustness and adaptability

Outperforms fixed-architecture models

Abstract

The issue of data-driven neural network model construction is one of the core problems in the domain of Artificial Intelligence. A standard approach assumes a fixed architecture with trainable weights. A conceptually more advanced assumption is that we not only train the weights, but also find out the optimal model architecture. We present a new method that realizes just that. This article is an extended version of our conference paper titled "Dynamic Growing and Shrinking of Neural Networks with Monte Carlo Tree Search [26]". In the paper, we show in detail how to create a neural network with a procedure that allows dynamic shrinking and growing of the model while it is being trained. The decision-making mechanism for the architectural design is governed by a Monte Carlo tree search procedure which simulates network behavior and allows to compare several candidate architecture changes to choose the best one. The proposed method was validated using both visual and time series datasets, demonstrating its particular effectiveness in multivariate time series classification. This is attributed to the architecture's ability to adapt dynamically, allowing independent modifications for each time series. The approach is supplemented by Python source code for reproducibility. Experimental evaluations in visual pattern and multivariate time series classification tasks revealed highly promising performance, underscoring the method's robustness and adaptability.