Bringing Data Transformations Near-Memory for Low-Latency Analytics in HTAP Environments

TL;DR

This paper introduces a near-memory data transformation approach for HTAP systems, reducing data movement and latency, and demonstrating improved performance and resource efficiency.

Contribution

It proposes executing data transformations near or in storage, addressing performance issues of traditional data movement-heavy methods.

Findings

Robust performance of foreground workloads

Lower resource contention

Potential for architectural reuse

Abstract

In this paper we propose an approach for executing data transformations near- or in-storage on intelligent storage systems. The currently prevailing approach of extracting the data and then transforming it to a target format suffers degraded performance during transformation and causes heavy data movement. Our results show robust performance of foreground workloads and lower resource contention. Our vision draws architectural opportunities in multi-engine and multi-system settings, as well as for reuse.