Sixteen Years of Phishing User Studies: What Have We Learned?

TL;DR

This paper systematically reviews and meta-analyzes 16 years of user studies on phishing, revealing key factors like age, gender, and training that influence susceptibility, with training notably improving detection skills.

Contribution

It provides a comprehensive meta-analysis of phishing susceptibility studies, clarifying the effects of demographic factors and training on user vulnerability.

Findings

Older users show mixed susceptibility results

Females are more susceptible than males

Training significantly improves detection ability

Abstract

Several previous studies have investigated user susceptibility to phishing attacks. A thorough meta-analysis or systematic review is required to gain a better understanding of these findings and to assess the strength of evidence for phishing susceptibility of a subpopulation, e.g., older users. We aim to determine whether an effect exists; another aim is to determine whether the effect is positive or negative and to obtain a single summary estimate of the effect. OBJECTIVES: We systematically review the results of previous user studies on phishing susceptibility and conduct a meta-analysis. METHOD: We searched four online databases for English studies on phishing. We included all user studies in phishing detection and prevention, whether they proposed new training techniques or analyzed users' vulnerability. FINDINGS: A careful analysis reveals some discrepancies between the findings. More than half of the studies that analyzed the effect of age reported no statistically significant relationship between age and users' performance. Some studies reported older people performed better while some reported the opposite. A similar finding holds for the gender difference. The meta-analysis shows: 1) a significant relationship between participants' age and their susceptibility 2) females are more susceptible than males 3) users training significantly improves their detection ability