An Effective Weight Initialization Method for Deep Learning: Application to Satellite Image Classification

TL;DR

This paper introduces a novel weight initialization method for CNNs tailored to satellite image classification, demonstrating improved accuracy over existing techniques through extensive experiments on real-world datasets.

Contribution

The paper proposes a new weight initialization technique specifically designed for satellite image classification CNNs, with detailed mathematical formulation and empirical validation.

Findings

Outperforms existing weight initialization methods in accuracy

Validated on six real-world satellite datasets

Available code implementation for reproducibility

Abstract

The growing interest in satellite imagery has triggered the need for efficient mechanisms to extract valuable information from these vast data sources, providing deeper insights. Even though deep learning has shown significant progress in satellite image classification. Nevertheless, in the literature, only a few results can be found on weight initialization techniques. These techniques traditionally involve initializing the networks' weights before training on extensive datasets, distinct from fine-tuning the weights of pre-trained networks. In this study, a novel weight initialization method is proposed in the context of satellite image classification. The proposed weight initialization method is mathematically detailed during the forward and backward passes of the convolutional neural network (CNN) model. Extensive experiments are carried out using six real-world datasets. Comparative analyses with existing weight initialization techniques made on various well-known CNN models reveal that the proposed weight initialization technique outperforms the previous competitive techniques in classification accuracy. The complete code of the proposed technique, along with the obtained results, is available at https://github.com/WadiiBoulila/Weight-Initialization