LLM Based Multi-Agent Generation of Semi-structured Documents from Semantic Templates in the Public Administration Domain

TL;DR

This paper presents a novel method combining large language models, prompt engineering, and multi-agent systems to generate semi-structured public administration documents based on semantic retrieval, improving automation and accuracy.

Contribution

It introduces replacing manual prompts with semantic retrieval-generated task descriptions for structured document generation using LLMs in public administration.

Findings

Effective document generation demonstrated in real-world PA scenarios

Improved accuracy over manual prompting methods

Scalable approach for diverse semi-structured documents

Abstract

In the last years' digitalization process, the creation and management of documents in various domains, particularly in Public Administration (PA), have become increasingly complex and diverse. This complexity arises from the need to handle a wide range of document types, often characterized by semi-structured forms. Semi-structured documents present a fixed set of data without a fixed format. As a consequence, a template-based solution cannot be used, as understanding a document requires the extraction of the data structure. The recent introduction of Large Language Models (LLMs) has enabled the creation of customized text output satisfying user requests. In this work, we propose a novel approach that combines the LLMs with prompt engineering and multi-agent systems for generating new documents compliant with a desired structure. The main contribution of this work concerns replacing the commonly used manual prompting with a task description generated by semantic retrieval from an LLM. The potential of this approach is demonstrated through a series of experiments and case studies, showcasing its effectiveness in real-world PA scenarios.