Nonparametric estimation of genewise variance for microarray data

TL;DR

This paper introduces novel nonparametric estimators for genewise variance in microarray data, addressing high-dimensional challenges and improving differential gene expression detection.

Contribution

It proposes two new nonparametric estimators for genewise variance and correlation, extending the Neyman--Scott model beyond microarray data.

Findings

Estimators are asymptotically normal.

Simulation studies demonstrate finite sample effectiveness.

Method improves power of differential expression tests.

Abstract

Estimation of genewise variance arises from two important applications in microarray data analysis: selecting significantly differentially expressed genes and validation tests for normalization of microarray data. We approach the problem by introducing a two-way nonparametric model, which is an extension of the famous Neyman--Scott model and is applicable beyond microarray data. The problem itself poses interesting challenges because the number of nuisance parameters is proportional to the sample size and it is not obvious how the variance function can be estimated when measurements are correlated. In such a high-dimensional nonparametric problem, we proposed two novel nonparametric estimators for genewise variance function and semiparametric estimators for measurement correlation, via solving a system of nonlinear equations. Their asymptotic normality is established. The finite sample property is demonstrated by simulation studies. The estimators also improve the power of the tests for detecting statistically differentially expressed genes. The methodology is illustrated by the data from microarray quality control (MAQC) project.