A Grid Based Architecture for High-Performance NLP

TL;DR

This paper presents a flexible, component-based software architecture for NLP applications that integrates with distributed computing services, enabling scalable and adaptable natural language processing solutions.

Contribution

It introduces a novel grid-based architecture that combines linguistic resource management with high-performance distributed computing for NLP.

Findings

Design and implementation of an extensible architecture

Integration with distributed computing services

Enhanced flexibility in application composition

Abstract

We describe the design and early implementation of an extensible, component-based software architecture for natural language engineering applications which interfaces with high performance distributed computing services. The architecture leverages existing linguistic resource description and discovery mechanisms based on metadata descriptions, combining these in a compatible fashion with other software definition abstractions. Within this architecture, application design is highly flexible, allowing disparate components to be combined to suit the overall application functionality, and formally described independently of processing concerns. An application specification language provides abstraction from the programming environment and allows ease of interface with high performance computational grids via a broker.