Deep Active Learning for Data Mining from Conflict Text Corpora

TL;DR

This paper introduces an active learning approach to efficiently extract detailed conflict event information from textual data, significantly reducing human annotation effort while maintaining high accuracy.

Contribution

It presents a novel, cost-effective active learning method for fine-tuning language models to extract conflict dynamics from text, outperforming traditional annotation efforts.

Findings

Achieves performance comparable to human coding

Reduces human annotation effort by up to 99%

Effectively extracts detailed conflict event information

Abstract

High-resolution event data on armed conflict and related processes have revolutionized the study of political contention with datasets like UCDP GED, ACLED etc. However, most of these datasets limit themselves to collecting spatio-temporal (high-resolution) and intensity data. Information on dynamics, such as targets, tactics, purposes etc. are rarely collected owing to the extreme workload of collecting data. However, most datasets rely on a rich corpus of textual data allowing further mining of further information connected to each event. This paper proposes one such approach that is inexpensive and high performance, leveraging active learning - an iterative process of improving a machine learning model based on sequential (guided) human input. Active learning is employed to then step-wise train (fine-tuning) of a large, encoder-only language model adapted for extracting sub-classes of events relating to conflict dynamics. The approach shows performance similar to human (gold-standard) coding while reducing the amount of required human annotation by as much as 99%.