Streaming enumeration on nested documents

TL;DR

This paper introduces a streaming algorithm for enumerating query results over nested documents like XML and JSON with constant delay, optimizing for efficiency and memory usage.

Contribution

It presents a novel single-pass, constant-delay enumeration algorithm for nested document queries modeled as Visibly Pushdown Transducers, proving its optimality.

Findings

Algorithm achieves constant-delay enumeration after one pass

The method is worst-case optimal in update-time and memory

Applicable to XML and JSON stream processing

Abstract

Some of the most relevant document schemas used online, such as XML and JSON, have a nested format. In the last decade, the task of extracting data from nested documents over streams has become especially relevant. We focus on the streaming evaluation of queries with outputs of varied sizes over nested documents. We model queries of this kind as Visibly Pushdown Transducers (VPT), a computational model that extends visibly pushdown automata with outputs and has the same expressive power as MSO over nested documents. Since processing a document through a VPT can generate a massive number of results, we are interested in reading the input in a streaming fashion and enumerating the outputs one after another as efficiently as possible, namely, with constant-delay. This paper presents an algorithm that enumerates these elements with constant-delay after processing the document stream in a single pass. Furthermore, we show that this algorithm is worst-case optimal in terms of update-time per symbol and memory usage.