Adaptive Growth: Real-time CNN Layer Expansion

TL;DR

This paper introduces an unsupervised, real-time CNN layer expansion algorithm that dynamically adapts network capacity based on input data, improving flexibility and performance across various datasets and transfer learning scenarios.

Contribution

It presents a novel unsupervised method for real-time CNN layer growth, enabling adaptive and scalable deep neural networks in dynamic environments.

Findings

Outperforms supervised methods on multiple datasets

Enhances transfer learning adaptability

Demonstrates effective real-time layer expansion

Abstract

Deep Neural Networks (DNNs) have shown unparalleled achievements in numerous applications, reflecting their proficiency in managing vast data sets. Yet, their static structure limits their adaptability in ever-changing environments. This research presents a new algorithm that allows the convolutional layer of a Convolutional Neural Network (CNN) to dynamically evolve based on data input, while still being seamlessly integrated into existing DNNs. Instead of a rigid architecture, our approach iteratively introduces kernels to the convolutional layer, gauging its real-time response to varying data. This process is refined by evaluating the layer's capacity to discern image features, guiding its growth. Remarkably, our unsupervised method has outstripped its supervised counterparts across diverse datasets like MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100. It also showcases enhanced adaptability in transfer learning scenarios. By introducing a data-driven model scalability strategy, we are filling a void in deep learning, leading to more flexible and efficient DNNs suited for dynamic settings. Code:(https://github.com/YunjieZhu/Extensible-Convolutional-Layer-git-version).