The Bootstrapped Robustness Assessment for Qualitative Comparative Analysis

TL;DR

This paper introduces baQCA, a bootstrap-based method to assess the robustness of QCA results, helping distinguish genuine patterns from random noise and improving the method's reliability.

Contribution

It adapts a hypothesis testing technique to QCA, providing a practical tool for evaluating the significance of configurations and guiding threshold choices.

Findings

QCA's tendency for spurious results decreases with appropriate thresholds

The method offers case-specific robustness assessment

Threshold recommendations vary with data structure

Abstract

Qualitative Comparative Analysis (QCA) has been increasingly used in recent years due to its purported construction of a middle path between case-oriented and variable-oriented methods. Despite its popularity, a key element of the method has been criticized for possibly not distinguishing random from real patterns in data, rendering its usefulness questionable. Critics of the method suggest a straightforward technique to test whether QCA will return a configuration when given random data. We adapt this technique to determine the probability that a given QCA application would return a random result. This assessment can be used as a hypothesis test for QCA, with an interpretation similar to a p-value. Using repeated applications of QCA to randomly-generated data, we first show that generally, the tendency for QCA to return spurious results is attenuated by using reasonable consistency score and configurational N thresholds; however, this varies considerably according to the basic structure of the data. Second, we suggest an application-specific assessment of QCA results, illustrated using the case of Tea Party rallies in Florida. This method, which we coin the Bootstrapped Robustness Assessment for QCA (baQCA), can provide researchers with recommendations for consistency score and configurational N thresholds.