Multi-Dimensional Vector ISA Extension for Mobile In-Cache Computing

TL;DR

This paper introduces a multi-dimensional vector ISA extension for mobile in-cache computing, significantly improving performance and energy efficiency by better utilizing SIMD resources across multiple dimensions.

Contribution

It proposes a novel multi-dimensional vector ISA extension (MVE) that enhances in-cache computing by supporting multi-dimensional memory access and masked execution, addressing limitations of existing one-dimensional ISAs.

Findings

Achieves 2.9x performance improvement on average

Reduces energy consumption by 8.8x on average

Implements with only 3.6% area overhead

Abstract

In-cache computing technology transforms existing caches into long-vector compute units and offers low-cost alternatives to building expensive vector engines for mobile CPUs. Unfortunately, existing long-vector Instruction Set Architecture (ISA) extensions, such as RISC-V Vector Extension (RVV) and Arm Scalable Vector Extension (SVE), provide only one-dimensional strided and random memory accesses. While this is sufficient for typical vector engines, it fails to effectively utilize the large Single Instruction, Multiple Data (SIMD) widths of in-cache vector engines. This is because mobile data-parallel kernels expose limited parallelism across a single dimension. Based on our analysis of mobile vector kernels, we introduce a long-vector Multi-dimensional Vector ISA Extension (MVE) for mobile in-cache computing. MVE achieves high SIMD resource utilization and enables flexible programming by abstracting cache geometry and data layout. The proposed ISA features multi-dimensional strided and random memory accesses and efficient dimension-level masked execution to encode parallelism across multiple dimensions. Using a wide range of data-parallel mobile workloads, we demonstrate that MVE offers significant performance and energy reduction benefits of 2.9x and 8.8x, on average, compared to the SIMD units of a commercial mobile processor, at an area overhead of 3.6%.