TensAIR: Real-Time Training of Neural Networks from Data-streams

TL;DR

TensAIR is a novel online learning system that enables real-time training of neural networks from data streams with high scalability and performance, outperforming existing stream-processing solutions significantly.

Contribution

It introduces TensAIR, the first decentralized and asynchronous OL system for real-time ANN training, achieving near-linear scalability and high throughput in stream-processing environments.

Findings

TensAIR achieves nearly linear scale-out performance with more worker nodes.

It attains 6 to 116 times higher throughput than state-of-the-art systems.

Demonstrated versatility with both sparse and dense data use cases.

Abstract

Online learning (OL) from data streams is an emerging area of research that encompasses numerous challenges from stream processing, machine learning, and networking. Stream-processing platforms, such as Apache Kafka and Flink, have basic extensions for the training of Artificial Neural Networks (ANNs) in a stream-processing pipeline. However, these extensions were not designed to train ANNs in real-time, and they suffer from performance and scalability issues when doing so. This paper presents TensAIR, the first OL system for training ANNs in real time. TensAIR achieves remarkable performance and scalability by using a decentralized and asynchronous architecture to train ANN models (either freshly initialized or pre-trained) via DASGD (decentralized and asynchronous stochastic gradient descent). We empirically demonstrate that TensAIR achieves a nearly linear scale-out performance in terms of (1) the number of worker nodes deployed in the network, and (2) the throughput at which the data batches arrive at the dataflow operators. We depict the versatility of TensAIR by investigating both sparse (word embedding) and dense (image classification) use cases, for which TensAIR achieved from 6 to 116 times higher sustainable throughput rates than state-of-the-art systems for training ANN in a stream-processing pipeline.