Incorporating Failure Knowledge into Design Decisions for IoT Systems: A Controlled Experiment on Novices

TL;DR

This study investigates how incorporating failure stories into IoT system design education influences novice engineers' decision-making, highlighting differences in their reasoning processes and suggesting avenues for failure-aware IoT engineering.

Contribution

It introduces a novel approach of using failure stories as a learning treatment to improve IoT design decisions among novices.

Findings

Subjects exposed to failure stories used safety as a rationale more often.

Design decisions were comparable across groups, but reasoning differed.

Failure stories influenced the emphasis on criticality and safety in design rationale.

Abstract

Internet of Things (IoT) systems allow software to directly interact with the physical world. Recent IoT failures can be attributed to recurring software design flaws, suggesting IoT software engineers may not be learning from past failures. We examine the use of failure stories to improve IoT system designs. We conducted an experiment to evaluate the influence of failure-related learning treatments on design decisions. Our experiment used a between-subjects comparison of novices (computer engineering students) completing a design questionnaire. There were three treatments: a control group (N=7); a group considering a set of design guidelines (N=8); and a group considering failure stories (proposed treatment, N=6). We measured their design decisions and their design rationales. All subjects made comparable decisions. Their rationales varied by treatment: subjects treated with guidelines and failure stories made greater use of criticality as a rationale, while subjects exposed to failure stories more frequently used safety as a rationale. Building on these findings, we suggest several research directions toward a failure-aware IoT engineering process.