Misinformation with Legal Consequences (MisLC): A New Task Towards Harnessing Societal Harm of Misinformation

TL;DR

This paper introduces the MisLC task, which assesses misinformation based on legal consequences across various legal domains, aiming to better understand and mitigate societal harm caused by misinformation.

Contribution

It proposes a novel interdisciplinary task, combining legal analysis with misinformation detection, and develops a dataset using crowd-sourcing and expert evaluation methods.

Findings

Large language models perform well but still lag behind experts.

The dataset covers 4 legal topics and 11 legal issues.

Empirical evidence highlights challenges in automating legal consequence detection.

Abstract

Misinformation, defined as false or inaccurate information, can result in significant societal harm when it is spread with malicious or even innocuous intent. The rapid online information exchange necessitates advanced detection mechanisms to mitigate misinformation-induced harm. Existing research, however, has predominantly focused on assessing veracity, overlooking the legal implications and social consequences of misinformation. In this work, we take a novel angle to consolidate the definition of misinformation detection using legal issues as a measurement of societal ramifications, aiming to bring interdisciplinary efforts to tackle misinformation and its consequence. We introduce a new task: Misinformation with Legal Consequence (MisLC), which leverages definitions from a wide range of legal domains covering 4 broader legal topics and 11 fine-grained legal issues, including hate speech, election laws, and privacy regulations. For this task, we advocate a two-step dataset curation approach that utilizes crowd-sourced checkworthiness and expert evaluations of misinformation. We provide insights about the MisLC task through empirical evidence, from the problem definition to experiments and expert involvement. While the latest large language models and retrieval-augmented generation are effective baselines for the task, we find they are still far from replicating expert performance.