Jack Unit: An Area- and Energy-Efficient Multiply-Accumulate (MAC) Unit Supporting Diverse Data Formats

TL;DR

This paper presents the Jack unit, an energy- and area-efficient MAC supporting multiple data formats, improving hardware efficiency and energy consumption in AI accelerators.

Contribution

The Jack unit introduces a flexible, multi-format MAC design with novel bit-level adjustments and parallelism, outperforming baseline units in area, power, and energy efficiency.

Findings

Reduces area by up to 2.01x compared to baseline MACs.

Consumes up to 1.84x less power.

Improves energy efficiency by up to 5.41x on AI benchmarks.

Abstract

In this work, we introduce an area- and energy-efficient multiply-accumulate (MAC) unit, named Jack unit, that is a jack-of-all-trades, supporting various data formats such as integer (INT), floating point (FP), and microscaling data format (MX). It provides bit-level flexibility and enhances hardware efficiency by i) replacing the carry-save multiplier (CSM) in the FP multiplier with a precision-scalable CSM, ii) performing the adjustment of significands based on the exponent differences within the CSM, and iii) utilizing 2D sub-word parallelism. To assess effectiveness, we implemented the layout of the Jack unit and three baseline MAC units. Additionally, we designed an AI accelerator equipped with our Jack units to compare with a state-of-the-art AI accelerator supporting various data formats. The proposed MAC unit occupies 1.17~2.01x smaller area and consumes 1.05~1.84x lower power compared to the baseline MAC units. On five AI benchmarks, the accelerator designed with our Jack units improves energy efficiency by 1.32~5.41x over the baseline across various data formats.