Otimizacao de pesos e funcoes de ativacao de redes neurais aplicadas na previsao de series temporais

TL;DR

This paper introduces a method to optimize both weights and activation functions in neural networks for time series prediction, enhancing accuracy by using a family of parameterized asymmetric activation functions and advanced optimization techniques.

Contribution

It proposes a novel approach to simultaneously optimize activation functions and weights in neural networks, improving time series forecasting performance.

Findings

Optimized activation functions improve prediction accuracy.

Combined optimization methods outperform traditional training.

Method satisfies universal approximation theorem requirements.

Abstract

Neural Networks have been applied for time series prediction with good experimental results that indicate the high capacity to approximate functions with good precision. Most neural models used in these applications use activation functions with fixed parameters. However, it is known that the choice of activation function strongly influences the complexity and performance of the neural network and that a limited number of activation functions have been used. In this work, we propose the use of a family of free parameter asymmetric activation functions for neural networks and show that this family of defined activation functions satisfies the requirements of the universal approximation theorem. A methodology for the global optimization of this family of activation functions with free parameter and the weights of the connections between the processing units of the neural network is used. The central idea of the proposed methodology is to simultaneously optimize the weights and the activation function used in a multilayer perceptron network (MLP), through an approach that combines the advantages of simulated annealing, tabu search and a local learning algorithm, with the purpose of improving performance in the adjustment and forecasting of time series. We chose two learning algorithms: backpropagation with the term momentum (BPM) and LevenbergMarquardt (LM).