Deep-and-Wide Learning: Enhancing Data-Driven Inference via Synergistic Learning of Inter- and Intra-Data Representations

TL;DR

This paper introduces deep-and-wide learning (DWL), a novel approach that captures both intra- and inter-data features to improve accuracy and efficiency in deep neural networks, especially with limited data.

Contribution

The paper proposes a new deep-and-wide learning scheme and a dual-interactive-channel network (D-Net) that synergistically combines low-dimensional inter-data features with high-dimensional intra-data features.

Findings

DWL surpasses state-of-the-art DNNs in accuracy with limited data

DWL improves computational efficiency by an order of magnitude

The approach enhances data-driven inference across various disciplines

Abstract

Advancements in deep learning are revolutionizing science and engineering. The immense success of deep learning is largely due to its ability to extract essential high-dimensional (HD) features from input data and make inference decisions based on this information. However, current deep neural network (DNN) models face several challenges, such as the requirements of extensive amounts of data and computational resources. Here, we introduce a new learning scheme, referred to as deep-and-wide learning (DWL), to systematically capture features not only within individual input data (intra-data features) but also across the data (inter-data features). Furthermore, we propose a dual-interactive-channel network (D-Net) to realize the DWL, which leverages our Bayesian formulation of low-dimensional (LD) inter-data feature extraction and its synergistic interaction with the conventional HD representation of the dataset, for substantially enhanced computational efficiency and inference. The proposed technique has been applied to data across various disciplines for both classification and regression tasks. Our results demonstrate that DWL surpasses state-of-the-art DNNs in accuracy by a substantial margin with limited training data and improves the computational efficiency by order(s) of magnitude. The proposed DWL strategy dramatically alters the data-driven learning techniques, including emerging large foundation models, and sheds significant insights into the evolving field of AI.