Lincoln AI Computing Survey (LAICS) and Trends

TL;DR

This paper updates the Lincoln AI Computing Survey to include recent generative AI accelerators, analyzing performance and power trends across market segments with new architecture categorizations.

Contribution

It provides the latest comprehensive overview of commercial AI accelerators, including performance, power metrics, and a new architecture classification scheme.

Findings

Performance and power trends analyzed across accelerators

Market segment distinctions highlighted in scatter plots

Introduction of a new architecture categorization

Abstract

In the past year, generative AI (GenAI) models have received a tremendous amount of attention, which in turn has increased attention to computing systems for training and inference for GenAI. Hence, an update to this survey is due. This paper is an update of the survey of AI accelerators and processors from past seven years, which is called the Lincoln AI Computing Survey -- LAICS (pronounced "lace"). This multi-year survey collects and summarizes the current commercial accelerators that have been publicly announced with peak performance and peak power consumption numbers. In the same tradition of past papers of this survey, the performance and power values are plotted on a scatter graph, and a number of dimensions and observations from the trends on this plot are again discussed and analyzed. Market segments are highlighted on the scatter plot, and zoomed plots of each segment are also included. A brief description of each of the new accelerators that have been added in the survey this year is included, and this update features a new categorization of computing architectures that implement each of the accelerators.