Towards an Effective Organization-Wide Bulk Email System

TL;DR

This paper investigates how to improve organizational bulk email systems by understanding stakeholder needs, evaluating personalization effects, and designing tools to enhance message effectiveness and employee engagement.

Contribution

It introduces a multi-stakeholder personalization framework and a neural network-based message reading estimation tool, advancing effective communication in large organizations.

Findings

Current system retains only 22% of communicated information.

Personalization improved message recognition rate by 20%.

Neural network estimation achieved 73% accuracy.

Abstract

Bulk email is widely used in organizations to communicate messages to employees. It is an important tool in making employees aware of policies, events, leadership updates, etc. However, in large organizations, the problem of overwhelming communication is widespread. Ineffective organizational bulk emails waste employees' time and organizations' money, and cause a lack of awareness or compliance with organizations' missions and priorities. This thesis focuses on improving organizational bulk email systems by 1) conducting qualitative research to understand different stakeholders; 2) conducting field studies to evaluate personalization's effects on getting employees to read bulk messages; 3) designing tools to support communicators in evaluating bulk emails. We performed these studies at the University of Minnesota, interviewing 25 employees (both senders and recipients), and including 317 participants in total. We found that the university's current bulk email system is ineffective as only 22% of the information communicated was retained by employees. To encourage employees to read high-level information, we implemented a multi-stakeholder personalization framework that mixed important-to-organization messages with employee-preferred messages and improved the studied bulk email's recognition rate by 20%. On the sender side, we iteratively designed a prototype of a bulk email evaluation platform. In field evaluation, we found bulk emails' message-level performance helped communicators in designing bulk emails. We collected eye-tracking data and developed a neural network technique to estimate how much time each message is being read using recipients' interactions with browsers only, which improved the estimation accuracy to 73%. In summary, this work sheds light on how to design organizational bulk email systems that communicate effectively and respect different stakeholders' value.