Towards More Data-Aware Application Integration (extended version)

TL;DR

This paper proposes a declarative, table-centric approach to application integration within standard systems, replacing traditional operators with in-memory table processing to improve data handling efficiency.

Contribution

It introduces a novel method for integrating table-centric semantics into existing systems by replacing operator implementations with in-memory table processing.

Findings

Table-centric processing shows promising performance in message routing.

Potential to surpass traditional methods in multi-record message throughput.

Practical realization demonstrated in a conventional integration system.

Abstract

Although most business application data is stored in relational databases, programming languages and wire formats in integration middleware systems are not table-centric. Due to costly format conversions, data-shipments and faster computation, the trend is to "push-down" the integration operations closer to the storage representation. We address the alternative case of defining declarative, table-centric integration semantics within standard integration systems. For that, we replace the current operator implementations for the well-known Enterprise Integration Patterns by equivalent "in-memory" table processing, and show a practical realization in a conventional integration system for a non-reliable, "data-intensive" messaging example. The results of the runtime analysis show that table-centric processing is promising already in standard, "single-record" message routing and transformations, and can potentially excel the message throughput for "multi-record" table messages.