Actionable Cybersecurity Notifications for Smart Homes: A User Study on the Role of Length and Complexity

TL;DR

This study investigates how the length and complexity of cybersecurity notifications generated by Large Language Models influence user understanding and response in smart home environments.

Contribution

It provides empirical evidence on optimal notification design, highlighting that intermediate complexity messages are most effective for diverse users.

Findings

Intermediate complexity notifications are most effective.

Longer beginner messages are rated more effective.

Shorter expert messages are rated more effective.

Abstract

The proliferation of smart home devices has increased convenience but also introduced cybersecurity risks for everyday users, as many devices lack robust security features. Intrusion Detection Systems are a prominent approach to detecting cybersecurity threats. However, their alerts often use technical terms and require users to interpret them correctly, which is challenging for a typical smart home user. Large Language Models can bridge this gap by translating IDS alerts into actionable security notifications. However, it has not yet been clear what an actionable cybersecurity notification should look like. In this paper, we conduct an experimental online user study with 130 participants to examine how the length and complexity of LLM-generated notifications affect user likability, understandability, and motivation to act. Our results show that intermediate-complexity notifications are the most effective across all user groups, regardless of their technological proficiency. Across the board, users rated beginner-level messages as more effective when they were longer, while expert-level messages were rated marginally more effective when they were shorter. These findings provide insights for designing security notifications that are both actionable and broadly accessible to smart home users.