TensorFlow: A system for large-scale machine learning

TL;DR

TensorFlow is a scalable, flexible machine learning system that uses dataflow graphs across heterogeneous hardware, enabling efficient training and inference for deep neural networks and supporting a wide range of applications.

Contribution

The paper introduces TensorFlow's dataflow graph architecture, highlighting its scalability, flexibility, and support for diverse hardware, which advances large-scale machine learning capabilities.

Findings

TensorFlow achieves high performance on real-world applications.

It supports heterogeneous hardware including TPUs, GPUs, and CPUs.

The system enables flexible experimentation with training algorithms.

Abstract

TensorFlow is a machine learning system that operates at large scale and in heterogeneous environments. TensorFlow uses dataflow graphs to represent computation, shared state, and the operations that mutate that state. It maps the nodes of a dataflow graph across many machines in a cluster, and within a machine across multiple computational devices, including multicore CPUs, general-purpose GPUs, and custom designed ASICs known as Tensor Processing Units (TPUs). This architecture gives flexibility to the application developer: whereas in previous "parameter server" designs the management of shared state is built into the system, TensorFlow enables developers to experiment with novel optimizations and training algorithms. TensorFlow supports a variety of applications, with particularly strong support for training and inference on deep neural networks. Several Google services use TensorFlow in production, we have released it as an open-source project, and it has become widely used for machine learning research. In this paper, we describe the TensorFlow dataflow model in contrast to existing systems, and demonstrate the compelling performance that TensorFlow achieves for several real-world applications.