Individualized Group Learning

TL;DR

This paper introduces the individualized group learning (iGroup) method, which enhances personalized inference by pooling similar individuals' data, effectively balancing individual heterogeneity and population information.

Contribution

The paper develops the iGroup approach using local nonparametric techniques to improve individual inference in heterogeneous populations, supported by theoretical analysis and empirical validation.

Findings

iGroup improves individual inference accuracy

Theoretical asymptotic performance is established

Empirical simulations demonstrate effectiveness

Abstract

Many massive data are assembled through collections of information of a large number of individuals in a population. The analysis of such data, especially in the aspect of individualized inferences and solutions, has the potential to create significant value for practical applications. Traditionally, inference for an individual in the data set is either solely relying on the information of the individual or from summarizing the information about the whole population. However, with the availability of big data, we have the opportunity, as well as a unique challenge, to make a more effective individualized inference that takes into consideration of both the population information and the individual discrepancy. To deal with the possible heterogeneity within the population while providing effective and credible inferences for individuals in a data set, this article develops a new approach called the individualized group learning (iGroup). The iGroup approach uses local nonparametric techniques to generate an individualized group by pooling other entities in the population which share similar characteristics with the target individual, even when individual estimates are biased due to limited number of observations. Three general cases of iGroup are discussed, and their asymptotic performances are investigated. Both theoretical results and empirical simulations reveal that, by applying iGroup, the performance of statistical inference on the individual level are ensured and can be substantially improved from inference based on either solely individual information or entire population information. The method has a broad range of applications. Two examples in financial statistics and maritime anomaly detection are presented.