Multi-Objective Personalization in Multi-Stakeholder Organizational Bulk E-mail: A Field Experiment

TL;DR

This study investigates how personalized bulk emails in organizations influence employee engagement and recognition, finding that strategic and tactical goals can be balanced through targeted personalization of message content and placement.

Contribution

The paper presents an experimental analysis of multi-objective personalization in organizational bulk emails, offering design insights for balancing organizational priorities and employee preferences.

Findings

Recognition of important messages improves when placed in subject lines or top news.

Mixing organizational and employee-preferred messages in top news does not enhance recognition of organizational messages.

Personalizing top news with employee-preferred messages increases overall newsletter interest.

Abstract

Bulk email is often used in organizations to communicate ``important-to-organization'' messages such as policy changes, organizational plans, and administrative updates. However, normal employees may prefer messages more relevant to their jobs or interests. Organizations face the challenge of balancing prioritizing the messages they prefer employees to know (tactical goals) while maintaining employees' positive experiences with these bulk emails, then they continue to read these emails in the future (strategic goals). Could personalization help organizations achieve these tactical and strategic goals? In an 8-week field experiment with a university newsletter, we implemented a 4x5x5 factorial design on personalizing subject lines, top news, and message order based on both the employees' and the organization's preferences. We measured these designs' influences on the open/interest/recognition/read-in-detail rate of the whole newsletter and the single messages within it. We found that ``important-to-organization'' messages only got higher recognition rates when being put on subject lines / top news (tactical goal). Mixing them with employee-preferred messages in top news did not bring further improvement to their own recognition rates but could improve the whole newsletter's recognition rate. Only when the top news solely contained the employee-preferred messages were the employees slightly more interested in the newsletter (strategic goal). We further analyze on which topics the employees and the organization's preferences conflicted. Finally, we discuss the design suggestions on personalization and recommendation for organizational bulk email.