Universal Inference Meets Random Projections: A Scalable Test for Log-concavity

TL;DR

This paper introduces a scalable, valid finite-sample test for log-concavity of distributions using universal inference and random projections, applicable in multiple dimensions with high power and efficiency.

Contribution

It develops the first finite-sample valid test for log-concavity in any dimension, combining universal inference with random projections for efficiency.

Findings

The test is valid in finite samples across dimensions.

Random projections improve test power and computational efficiency.

The method demonstrates strong empirical performance.

Abstract

Shape constraints yield flexible middle grounds between fully nonparametric and fully parametric approaches to modeling distributions of data. The specific assumption of log-concavity is motivated by applications across economics, survival modeling, and reliability theory. However, there do not currently exist valid tests for whether the underlying density of given data is log-concave. The recent universal inference methodology provides a valid test. The universal test relies on maximum likelihood estimation (MLE), and efficient methods already exist for finding the log-concave MLE. This yields the first test of log-concavity that is provably valid in finite samples in any dimension, for which we also establish asymptotic consistency results. Empirically, we find that a random projections approach that converts the d-dimensional testing problem into many one-dimensional problems can yield high power, leading to a simple procedure that is statistically and computationally efficient.