Transparency in Maintenance of Recruitment Chatbots

TL;DR

This paper discusses the challenges of maintaining transparency in recruitment chatbots that use machine learning, highlighting issues with error understanding and the role of a key contact in organizational transparency.

Contribution

It introduces the concept of a key contact role to improve transparency in the maintenance of ML-based recruitment chatbots, analyzing its effectiveness and limitations.

Findings

Key contact role improves initial transparency

Centralization poses challenges for sustained transparency

Errors span UX, ML, and software domains

Abstract

We report on experiences with implementing conversational agents in the recruitment domain based on a machine learning (ML) system. Recruitment chatbots mediate communication between job-seekers and recruiters by exposing ML data to recruiter teams. Errors are difficult to understand, communicate, and resolve because they may span and combine UX, ML, and software issues. In an effort to improve organizational and technical transparency, we came to rely on a key contact role. Though effective for design and development, the centralization of this role poses challenges for transparency in sustained maintenance of this kind of ML-based mediating system.