Adaptive Sample Size Simulations with R package adsasi

TL;DR

adsasi is an R package that simplifies sample size determination for complex experiments through simulation-based iterative methods, especially when closed-form solutions are unavailable.

Contribution

It introduces a novel simulation-first approach with iterative algorithms and probit regression to efficiently estimate sample sizes for complex experimental designs.

Findings

Results match closed-form solutions within Monte Carlo variance.

Successfully applied to intractable designs including bootstrap from medical data.

Provides standard errors using a custom analytical Hessian matrix.

Abstract

Planning empirical experiments such as clinical trials or A/B tests requires sample size determination, which in many interesting cases has no closed-form solution (e.g. factorial or adaptive designs). adsasi is a new R package that enables simulations-first sample size calculations for any trial that can be simulated in short compute time. First, the user specifies as a function that takes a sample size as argument, simulates the experiment, and returns a boolean for success/failure. Then, adsasi functions adsasi_0d and adsasi_1d iteratively call it on different sample sizes and progressively home in on the one with nominal success rate (power), assuming that increasing sample size increases power. adsasi_1d can also draw, purely empirically, the relationship between a design parameter and sample size. The implementation uses a modified probit regression (with success/failure as the dependent variable), informed by simulations conducted around the target size, and provides standard errors at each stage using the Cram\'er-Rao bound derived from a custom analytical Hessian matrix. Simple examples are first presented, yielding results within Monte Carlo variance of their closed-form expressions, then intractable ones (including bootstrapping from an existing medical cohort). adsasi will hopefully facilitate the funding and conduct of interesting, highly complex experimental designs by making their sizing straightforward.