Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications

TL;DR

This paper characterizes Facebook's deep learning models, discusses performance optimizations and hardware implications, and emphasizes the need for co-designed algorithms and hardware to improve inference in data centers.

Contribution

It provides detailed insights into deep learning model characteristics, optimization techniques, and hardware considerations specific to Facebook's data center environment.

Findings

Deep learning models have specific computational characteristics.

Optimizations improve inference performance but have limitations.

Recommendations for future hardware and algorithm co-design.

Abstract

The application of deep learning techniques resulted in remarkable improvement of machine learning models. In this paper provides detailed characterizations of deep learning models used in many Facebook social network services. We present computational characteristics of our models, describe high performance optimizations targeting existing systems, point out their limitations and make suggestions for the future general-purpose/accelerated inference hardware. Also, we highlight the need for better co-design of algorithms, numerics and computing platforms to address the challenges of workloads often run in data centers.