Cluster Size Matters: A Comparative Study of Notip and pARI for Post Hoc Inference in fMRI

TL;DR

This study compares Notip and pARI, two permutation-based post hoc inference methods for fMRI data, revealing their complementary strengths in sensitivity and robustness depending on cluster size.

Contribution

It provides a systematic reanalysis showing that Notip and pARI perform differently across cluster sizes, challenging previous conclusions and guiding their appropriate use.

Findings

pARI has higher sensitivity for large clusters

Notip offers more robust results for small clusters

Notip supports detailed exploration of activation subregions

Abstract

All Resolutions Inference (ARI) is a post hoc inference method for functional Magnetic Resonance Imaging (fMRI) data analysis that provides valid lower bounds on the proportion of truly active voxels within any, possibly data-driven, cluster. As such, it addresses the paradox of spatial specificity encountered with more classical cluster-extent thresholding methods. It allows the cluster-forming threshold to be increased in order to locate the signal with greater spatial precision without overfitting, also known as the drill-down approach. Notip and pARI are two recent permutation-based extensions of ARI designed to increase statistical power by accounting for the strong dependence structure typical of fMRI data. A recent comparison between these papers based on large voxel clusters concluded that pARI outperforms Notip. We revisit this conclusion by conducting a systematic comparison of the two. Our reanalysis of the same fMRI data sets from the Neurovault database demonstrates the existence of complementary performance regimes: while pARI indeed achieves higher sensitivity for large clusters, Notip provides more informative and robust results for smaller clusters. In particular, while Notip supports informative ``drill-down'' exploration into subregions of activation, pARI often yields non-informative bounds in such cases, and can even underperform the baseline ARI method.