Large-Scale Mode Identification and Data-Driven Sciences

TL;DR

This paper introduces LPMode, a scalable, nonparametric algorithm for automatic mode detection in large datasets, applicable across diverse scientific fields for data-driven discovery.

Contribution

The paper presents LPMode, a novel theory-based algorithm for objective, nonparametric mode identification that scales to large data sets, filling a gap in existing tools.

Findings

LPMode effectively detects multimodality in large datasets

Applied successfully across multiple scientific disciplines

Provides an automatic alternative to manual mode detection

Abstract

Bump-hunting or mode identification is a fundamental problem that arises in almost every scientific field of data-driven discovery. Surprisingly, very few data modeling tools are available for automatic (not requiring manual case-by-base investigation), objective (not subjective), and nonparametric (not based on restrictive parametric model assumptions) mode discovery, which can scale to large data sets. This article introduces LPMode--an algorithm based on a new theory for detecting multimodality of a probability density. We apply LPMode to answer important research questions arising in various fields from environmental science, ecology, econometrics, analytical chemistry to astronomy and cancer genomics.