Modeling Users and Online Communities for Abuse Detection: A Position on Ethics and Explainability

TL;DR

This paper discusses how modeling users and online communities can improve abuse detection in NLP, emphasizing ethical considerations and the importance of explainability for responsible AI development.

Contribution

It reviews current methods leveraging user and community data, explores ethical challenges, and proposes properties for explainable abusive language detection models.

Findings

User and community modeling enhances abuse detection accuracy

Ethical challenges include privacy and bias considerations

Explainability properties can guide responsible model development

Abstract

Abuse on the Internet is an important societal problem of our time. Millions of Internet users face harassment, racism, personal attacks, and other types of abuse across various platforms. The psychological effects of abuse on individuals can be profound and lasting. Consequently, over the past few years, there has been a substantial research effort towards automated abusive language detection in the field of NLP. In this position paper, we discuss the role that modeling of users and online communities plays in abuse detection. Specifically, we review and analyze the state of the art methods that leverage user or community information to enhance the understanding and detection of abusive language. We then explore the ethical challenges of incorporating user and community information, laying out considerations to guide future research. Finally, we address the topic of explainability in abusive language detection, proposing properties that an explainable method should aim to exhibit. We describe how user and community information can facilitate the realization of these properties and discuss the effective operationalization of explainability in view of the properties.