Comparative Analysis of AI Agent Architectures for Entity Relationship Classification

TL;DR

This paper compares three AI agent architectures for entity relationship classification using large language models, highlighting the effectiveness of multi-agent coordination and dynamic example generation in improving relation extraction performance.

Contribution

It introduces a novel multi-agent dynamic example generation mechanism and provides a comprehensive comparison of different AI architectures for relation classification.

Findings

Multi-agent coordination outperforms standard few-shot prompting.

Dynamic example generation approaches the performance of fine-tuned models.

The study offers practical guidance for designing modular LLM-based relation extraction systems.

Abstract

Entity relationship classification remains a challenging task in information extraction, especially in scenarios with limited labeled data and complex relational structures. In this study, we conduct a comparative analysis of three distinct AI agent architectures designed to perform relation classification using large language models (LLMs). The agentic architectures explored include (1) reflective self-evaluation, (2) hierarchical task decomposition, and (3) a novel multi-agent dynamic example generation mechanism, each leveraging different modes of reasoning and prompt adaptation. In particular, our dynamic example generation approach introduces real-time cooperative and adversarial prompting. We systematically compare their performance across multiple domains and model backends. Our experiments demonstrate that multi-agent coordination consistently outperforms standard few-shot prompting and approaches the performance of fine-tuned models. These findings offer practical guidance for the design of modular, generalizable LLM-based systems for structured relation extraction. The source codes and dataset are available at https://github.com/maryambrj/ALIEN.git.