Structuring the Unstructured: A Systematic Review of Text-to-Structure Generation for Agentic AI with a Universal Evaluation Framework

TL;DR

This paper systematically reviews methods for converting unstructured text into structured formats, evaluates existing datasets and metrics, and proposes a universal evaluation framework to advance agentic AI capabilities.

Contribution

It provides a comprehensive synthesis of text-to-structure techniques, datasets, and evaluation metrics, and introduces a universal framework for assessing structured outputs in AI.

Findings

Current datasets lack diversity and standardization.

Evaluation metrics are inconsistent across studies.

The proposed framework enables consistent assessment of structured outputs.

Abstract

The evolution of AI systems toward agentic operation and context-aware retrieval necessitates transforming unstructured text into structured formats like tables, knowledge graphs, and charts. While such conversions enable critical applications from summarization to data mining, current research lacks a comprehensive synthesis of methodologies, datasets, and metrics. This systematic review examines text-to-structure techniques and the encountered challenges, evaluates current datasets and assessment criteria, and outlines potential directions for future research. We also introduce a universal evaluation framework for structured outputs, establishing text-to-structure as foundational infrastructure for next-generation AI systems.