The ViP2P Platform: XML Views in P2P

TL;DR

ViP2P is a distributed P2P platform for managing large XML data collections using DHT overlays, enabling efficient query processing through materialized views and supporting both subscription and ad-hoc query modes.

Contribution

The paper introduces ViP2P, a novel P2P system that uses expressive XML queries and materialized views for scalable, efficient XML data sharing and querying in large distributed networks.

Findings

Outperforms similar systems by orders of magnitude in data volume and throughput.

Supports flexible query modes including long-term subscriptions and immediate snapshot queries.

Demonstrates scalability and efficiency in real-world deployments with hundreds of computers.

Abstract

The growing volumes of XML data sources on the Web or produced by enterprises, organizations etc. raise many performance challenges for data management applications. In this work, we are concerned with the distributed, peer-to-peer management of large corpora of XML documents, based on distributed hash table (or DHT, in short) overlay networks. We present ViP2P (standing for Views in Peer-to-Peer), a distributed platform for sharing XML documents based on a structured P2P network infrastructure (DHT). At the core of ViP2P stand distributed materialized XML views, defined by arbitrary XML queries, filled in with data published anywhere in the network, and exploited to efficiently answer queries issued by any network peer. ViP2P allows user queries to be evaluated over XML documents published by peers in two modes. First, a long-running subscription mode, when a query can be registered in the system and receive answers incrementally when and if published data matches the query. Second, queries can also be asked in an ad-hoc, snapshot mode, where results are required immediately and must be computed based on the results of other long-running, subscription queries. ViP2P innovates over other similar DHT-based XML sharing platforms by using a very expressive structured XML query language. This expressivity leads to a very flexible distribution of XML content in the ViP2P network, and to efficient snapshot query execution. ViP2P has been tested in real deployments of hundreds of computers. We present the platform architecture, its internal algorithms, and demonstrate its efficiency and scalability through a set of experiments. Our experimental results outgrow by orders of magnitude similar competitor systems in terms of data volumes, network size and data dissemination throughput.