Selective Embedding for Deep Learning

TL;DR

This paper introduces selective embedding, a novel data loading strategy that alternates data segments from multiple sources within a single input channel, improving generalization and efficiency in deep learning models for time-domain data.

Contribution

The study proposes selective embedding, inspired by cognitive psychology, as a new method to enhance deep learning performance across multiple data sources and reduce computational costs.

Findings

Consistently high classification accuracy across six datasets.

Significant reduction in training times.

Effective for complex, multi-source systems.

Abstract

Deep learning has revolutionized many industries by enabling models to automatically learn complex patterns from raw data, reducing dependence on manual feature engineering. However, deep learning algorithms are sensitive to input data, and performance often deteriorates under nonstationary conditions and across dissimilar domains, especially when using time-domain data. Conventional single-channel or parallel multi-source data loading strategies either limit generalization or increase computational costs. This study introduces selective embedding, a novel data loading strategy, which alternates short segments of data from multiple sources within a single input channel. Drawing inspiration from cognitive psychology, selective embedding mimics human-like information processing to reduce model overfitting, enhance generalization, and improve computational efficiency. Validation is conducted using six time-domain datasets, demonstrating that the proposed method consistently achieves high classification accuracy across various deep learning architectures while significantly reducing training times. The approach proves particularly effective for complex systems with multiple data sources, offering a scalable and resource-efficient solution for real-world applications in healthcare, heavy machinery, marine, railway, and agriculture, where robustness and adaptability are critical.