A layer-stress learning framework universally augments deep neural network tasks

TL;DR

This paper introduces a layer-stress learning framework (x-NN) that adaptively selects and combines features from multiple network layers using attention, improving performance in neural network tasks like Alzheimer's disease classification.

Contribution

The novel x-NN framework automatically determines optimal network depth and leverages multi-layer features via attention, enhancing deep learning model flexibility and accuracy.

Findings

Won top prize in Alzheimer's Disease Classification Challenge 2021

Outperformed all competing AI models in the challenge

Validated effectiveness on multiple datasets and tasks

Abstract

Deep neural networks (DNN) such as Multi-Layer Perception (MLP) and Convolutional Neural Networks (CNN) represent one of the most established deep learning algorithms. Given the tremendous effects of the number of hidden layers on network architecture and performance, it is very important to choose the number of hidden layers but still a serious challenge. More importantly, the current network architectures can only process the information from the last layer of the feature extractor, which greatly limited us to further improve its performance. Here we presented a layer-stress deep learning framework (x-NN) which implemented automatic and wise depth decision on shallow or deep feature map in a deep network through firstly designing enough number of layers and then trading off them by Multi-Head Attention Block. The x-NN can make use of features from various depth layers through attention allocation and then help to make final decision as well. As a result, x-NN showed outstanding prediction ability in the Alzheimer's Disease Classification Technique Challenge PRCV 2021, in which it won the top laurel and outperformed all other AI models. Moreover, the performance of x-NN was verified by one more AD neuroimaging dataset and other AI tasks.