Did You Remember to Test Your Tokens?

TL;DR

This paper develops a comprehensive, developer-friendly guide of 53 unit test cases for token authentication in Spring Security, based on analysis of real-world Java tests, to improve security testing practices.

Contribution

It introduces a novel, empirically-derived catalog of unit test cases for token authentication, filling a gap in security testing resources for developers.

Findings

Identified 53 common test cases from 481 JUnit tests across 53 projects.

Provided a structured test guide with scenarios, conditions, and expected outcomes.

Gathered developer feedback confirming the guide's usefulness.

Abstract

Authentication is a critical security feature for confirming the identity of a system's users, typically implemented with help from frameworks like Spring Security. It is a complex feature which should be robustly tested at all stages of development. Unit testing is an effective technique for fine-grained verification of feature behaviors that is not widely-used to test authentication. Part of the problem is that resources to help developers unit test security features are limited. Most security testing guides recommend test cases in a "black box" or penetration testing perspective. These resources are not easily applicable to developers writing new unit tests, or who want a security-focused perspective on coverage. In this paper, we address these issues by applying a grounded theory-based approach to identify common (unit) test cases for token authentication through analysis of 481 JUnit tests exercising Spring Security-based authentication implementations from 53 open source Java projects. The outcome of this study is a developer-friendly unit testing guide organized as a catalog of 53 test cases for token authentication, representing unique combinations of 17 scenarios, 40 conditions, and 30 expected outcomes learned from the data set in our analysis. We supplement the test guide with common test smells to avoid. To verify the accuracy and usefulness of our testing guide, we sought feedback from selected developers, some of whom authored unit tests in our dataset.