DSReLU: A Novel Dynamic Slope Function for Superior Model Training

TL;DR

This paper proposes DSReLU, a dynamic slope activation function that adapts during training to improve deep neural network performance in image recognition tasks, addressing limitations of traditional functions like ReLU.

Contribution

Introduction of DSReLU, a novel activation function with a dynamic slope that enhances adaptability and performance in deep learning models.

Findings

Improved classification accuracy on Mini-ImageNet, CIFAR-100, and MIT-BIH datasets.

Enhanced generalization capabilities of neural networks using DSReLU.

Demonstrated superiority over traditional activation functions in various image recognition benchmarks.

Abstract

This study introduces a novel activation function, characterized by a dynamic slope that adjusts throughout the training process, aimed at enhancing adaptability and performance in deep neural networks for computer vision tasks. The rationale behind this approach is to overcome limitations associated with traditional activation functions, such as ReLU, by providing a more flexible mechanism that can adapt to different stages of the learning process. Evaluated on the Mini-ImageNet, CIFAR-100, and MIT-BIH datasets, our method demonstrated improvements in classification metrics and generalization capabilities. These results suggest that our dynamic slope activation function could offer a new tool for improving the performance of deep learning models in various image recognition tasks.