Using Online Customer Reviews to Classify, Predict, and Learn about Domestic Robot Failures

TL;DR

This study classifies and analyzes 10,072 customer reviews of domestic robots to identify failure types, their impact on customer ratings, and develops an NLP model to predict failure-related content in reviews.

Contribution

Introduces an updated failure taxonomy for domestic robots, analyzes failure impact on customer ratings, and develops an NLP model to predict failure descriptions in reviews.

Findings

Technical failures significantly lower customer ratings.

Task and robustness failures are most impactful.

Usability issues are less detrimental to customer experience.

Abstract

There is a knowledge gap regarding which types of failures robots undergo in domestic settings and how these failures influence customer experience. We classified 10,072 customer reviews of small utilitarian domestic robots on Amazon by the robotic failures described in them, grouping failures into twelve types and three categories (Technical, Interaction, and Service). We identified sources and types of failures previously overlooked in the literature, combining them into an updated failure taxonomy. We analyzed their frequencies and relations to customer star ratings. Results indicate that for utilitarian domestic robots, Technical failures were more detrimental to customer experience than Interaction or Service failures. Issues with Task Completion and Robustness & Resilience were commonly reported and had the most significant negative impact. Future failure-prevention and response strategies should address the technical ability of the robot to meet functional goals, operate and maintain structural integrity over time. Usability and interaction design were less detrimental to customer experience, indicating that customers may be more forgiving of failures that impact these aspects for the robots and practical uses examined. Further, we developed a Natural Language Processing model capable of predicting whether a customer review contains content that describes a failure and the type of failure it describes. With this knowledge, designers and researchers of robotic systems can prioritize design and development efforts towards essential issues.