ARTAI: An Evaluation Platform to Assess Societal Risk of Recommender Algorithms

TL;DR

ARTAI is an evaluation platform designed to assess societal risks of recommender algorithms, facilitating large-scale audits and supporting regulatory compliance to mitigate online content dissemination harms.

Contribution

The paper introduces ARTAI, a novel evaluation environment that enables comprehensive assessment of recommender algorithms for societal risks and transparency.

Findings

Supports large-scale societal risk assessments

Identifies harmful content distribution patterns

Facilitates regulatory compliance and transparency

Abstract

Societal risk emanating from how recommender algorithms disseminate content online is now well documented. Emergent regulation aims to mitigate this risk through ethical audits and enabling new research on the social impact of algorithms. However, there is currently a need for tools and methods that enable such evaluation. This paper presents ARTAI, an evaluation environment that enables large-scale assessments of recommender algorithms to identify harmful patterns in how content is distributed online and enables the implementation of new regulatory requirements for increased transparency in recommender systems.