Premature Dimensional Collapse and Tensor-based Execution Paths for High-Dimensional Relational Operations in Cost-Based Database Systems

TL;DR

This paper introduces a tensor-based execution approach for high-dimensional relational operations in cost-based DBMSs, reducing latency spikes caused by premature linearization under memory pressure.

Contribution

It proposes delaying linearization and preserving higher-dimensional locality through structured intermediate layouts to improve execution stability.

Findings

Reduces P99 latency from multi-second to sub-second levels under memory constraints.

Maintains stable execution by avoiding memory-regime-induced phase transitions.

Demonstrates effectiveness using a modified PostgreSQL prototype and microbenchmarks.

Abstract

Modern cost-based DBMSs frequently exhibit execution instability and tail-latency amplification when high-dimensional relational operations trigger memory-regime transitions such as hash-table spilling and external materialization. We identify a structural failure mode in which intermediate representations are prematurely linearized under memory pressure, causing disproportionate I/O amplification and phase-transition-like latency behavior. To mitigate this, we propose a tensor-based execution path that delays premature linearization and preserves higher-dimensional locality through late materialization and structured intermediate layouts. Using a modified PostgreSQL-based prototype and controlled microbenchmarks, we show that under constrained memory settings (e.g., work_mem=1MB) conventional execution can spill hundreds of megabytes and exceed multi-second P99 latency, while the proposed path maintains stable execution and reduces P99 latency to sub-second levels. Our results suggest that representation timing is a first-class design variable for execution stability, complementing traditional optimization efforts focused on cardinality estimation and operator throughput.