Autonomously and Simultaneously Refining Deep Neural Network Parameters by Generative Adversarial Networks

TL;DR

This paper introduces a GAN-based method that autonomously refines multiple parameters of deep neural networks, leading to improved accuracy across various architectures and datasets.

Contribution

It presents a novel systematic approach using GANs to simultaneously optimize neural network parameters, reducing reliance on trial-and-error.

Findings

Successfully optimized parameters for three different neural network architectures

Achieved increased accuracy across three diverse datasets and applications

Demonstrated the method's effectiveness in different scenarios

Abstract

The choice of parameters, and the design of the network architecture are important factors affecting the performance of deep neural networks. However, there has not been much work on developing an established and systematic way of building the structure and choosing the parameters of a neural network, and this task heavily depends on trial and error and empirical results. Considering that there are many design and parameter choices, such as the number of neurons in each layer, the type of activation function, the choice of using drop out or not, it is very hard to cover every configuration, and find the optimal structure. In this paper, we propose a novel and systematic method that autonomously and simultaneously optimizes multiple parameters of any given deep neural network by using a generative adversarial network (GAN). In our proposed approach, two different models compete and improve each other progressively with a GAN-based strategy. Our proposed approach can be used to autonomously refine the parameters, and improve the accuracy of different deep neural network architectures. Without loss of generality, the proposed method has been tested with three different neural network architectures, and three very different datasets and applications. The results show that the presented approach can simultaneously and successfully optimize multiple neural network parameters, and achieve increased accuracy in all three scenarios.