An LLM Agent-Based Complex Semantic Table Annotation Approach

TL;DR

This paper introduces an LLM-based agent approach for semantic table annotation that dynamically adapts strategies to complex table challenges, significantly improving accuracy and efficiency in ontology mapping tasks.

Contribution

The paper presents a novel LLM agent framework with external tools and tailored prompts for improved semantic table annotation of complex tables.

Findings

Outperforms existing methods on Tough Tables and BiodivTab datasets.

Reduces annotation time costs by 70%.

Lowers LLM token usage by 60%.

Abstract

The Semantic Table Annotation (STA) task, which includes Column Type Annotation (CTA) and Cell Entity Annotation (CEA), maps table contents to ontology entities and plays important roles in various semantic applications. However, complex tables often pose challenges such as semantic loss of column names or cell values, strict ontological hierarchy requirements, homonyms, spelling errors, and abbreviations, which hinder annotation accuracy. To address these issues, this paper proposes an LLM-based agent approach for CTA and CEA. We design and implement five external tools with tailored prompts based on the ReAct framework, enabling the STA agent to dynamically select suitable annotation strategies depending on table characteristics. Experiments are conducted on the Tough Tables and BiodivTab datasets from the SemTab challenge, which contain the aforementioned challenges. Our method outperforms existing approaches across various metrics. Furthermore, by leveraging Levenshtein distance to reduce redundant annotations, we achieve a 70% reduction in time costs and a 60% reduction in LLM token usage, providing an efficient and cost-effective solution for STA.