Large Language Models are Pattern Matchers: Editing Semi-Structured and Structured Documents with ChatGPT

TL;DR

This paper explores how large language models like ChatGPT can be used to edit structured and semi-structured documents effectively through simple prompts, highlighting their pattern matching capabilities and implications for understanding hallucinations.

Contribution

It demonstrates that LLMs can successfully edit structured documents with minimal prompts and reveals their strong pattern matching skills, contributing to understanding their internal processes.

Findings

LLMs can effectively edit structured documents with basic prompts.

ChatGPT recognizes and processes document structure well.

Pattern matching skills in ChatGPT are impressive and warrant further study.

Abstract

Large Language Models (LLMs) offer numerous applications, the full extent of which is not yet understood. This paper investigates if LLMs can be applied for editing structured and semi-structured documents with minimal effort. Using a qualitative research approach, we conduct two case studies with ChatGPT and thoroughly analyze the results. Our experiments indicate that LLMs can effectively edit structured and semi-structured documents when provided with basic, straightforward prompts. ChatGPT demonstrates a strong ability to recognize and process the structure of annotated documents. This suggests that explicitly structuring tasks and data in prompts might enhance an LLM's ability to understand and solve tasks. Furthermore, the experiments also reveal impressive pattern matching skills in ChatGPT. This observation deserves further investigation, as it may contribute to understanding the processes leading to hallucinations in LLMs.