Assessing Open-Source Large Language Models on Argumentation Mining Subtasks

TL;DR

This paper evaluates the argumentation mining capabilities of four open-source large language models across multiple datasets and subtasks, providing insights into their effectiveness in zero-shot and few-shot settings.

Contribution

It offers a comprehensive assessment of open-source LLMs in argumentation mining, highlighting their strengths and limitations in different scenarios.

Findings

Open-source LLMs show promising performance in argumentation tasks.

Performance varies across datasets and subtasks.

Zero-shot and few-shot capabilities are demonstrated.

Abstract

We explore the capability of four open-sourcelarge language models (LLMs) in argumentation mining (AM). We conduct experiments on three different corpora; persuasive essays(PE), argumentative microtexts (AMT) Part 1 and Part 2, based on two argumentation mining sub-tasks: (i) argumentative discourse units classifications (ADUC), and (ii) argumentative relation classification (ARC). This work aims to assess the argumentation capability of open-source LLMs, including Mistral 7B, Mixtral8x7B, LlamA2 7B and LlamA3 8B in both, zero-shot and few-shot scenarios. Our analysis contributes to further assessing computational argumentation with open-source LLMs in future research efforts.