Classifying Proposals of Decentralized Autonomous Organizations Using Large Language Models

TL;DR

This paper explores using Large Language Models to automate the classification of DAO proposals, achieving high accuracy and reducing reliance on costly human expertise.

Contribution

It introduces an iterative prompting approach that refines classification categories, demonstrating effective automation of complex, context-dependent data labeling tasks.

Findings

Achieved 95% accuracy in classifying 100 DAO proposals

Demonstrated LLMs' potential to automate context-dependent data labeling

Validated iterative prompt refinement improves classification performance

Abstract

Our study demonstrates the effective use of Large Language Models (LLMs) for automating the classification of complex datasets. We specifically target proposals of Decentralized Autonomous Organizations (DAOs), as the clas-sification of this data requires the understanding of context and, therefore, depends on human expertise, leading to high costs associated with the task. The study applies an iterative approach to specify categories and further re-fine them and the prompt in each iteration, which led to an accuracy rate of 95% in classifying a set of 100 proposals. With this, we demonstrate the po-tential of LLMs to automate data labeling tasks that depend on textual con-text effectively.