What's in Score for Website Users: A Data-driven Long-term Study on Risk-based Authentication Characteristics

TL;DR

This study provides an in-depth, data-driven analysis of risk-based authentication (RBA) in real-world online services, highlighting its characteristics, feature effectiveness, and factors influencing its security and usability over a long-term deployment.

Contribution

It offers the first comprehensive analysis of RBA characteristics in practical deployment, including feature benchmarking, a novel feature proposal, and insights on configuration impacts.

Findings

RBA performance varies significantly with configuration adjustments.

A new feature for RBA effectiveness is identified.

Behavioral analysis of RBA implementations in the wild.

Abstract

Risk-based authentication (RBA) aims to strengthen password-based authentication rather than replacing it. RBA does this by monitoring and recording additional features during the login process. If feature values at login time differ significantly from those observed before, RBA requests an additional proof of identification. Although RBA is recommended in the NIST digital identity guidelines, it has so far been used almost exclusively by major online services. This is partly due to a lack of open knowledge and implementations that would allow any service provider to roll out RBA protection to its users. To close this gap, we provide a first in-depth analysis of RBA characteristics in a practical deployment. We observed N=780 users with 247 unique features on a real-world online service for over 1.8 years. Based on our collected data set, we provide (i) a behavior analysis of two RBA implementations that were apparently used by major online services in the wild, (ii) a benchmark of the features to extract a subset that is most suitable for RBA use, (iii) a new feature that has not been used in RBA before, and (iv) factors which have a significant effect on RBA performance. Our results show that RBA needs to be carefully tailored to each online service, as even small configuration adjustments can greatly impact RBA's security and usability properties. We provide insights on the selection of features, their weightings, and the risk classification in order to benefit from RBA after a minimum number of login attempts.