Statistical Inference, Learning and Models in Big Data

TL;DR

This paper reviews the challenges and strategies of statistical inference, learning, and modeling in big data across various scientific fields, highlighting common issues and applications discussed during a 2015 Canadian conference.

Contribution

It provides an overview of key topics, challenges, and strategies in statistical methods for big data, with examples from diverse applications.

Findings

Identification of common challenges in big data statistical analysis

Strategies for scalable inference and learning in large datasets

Application examples illustrating practical approaches

Abstract

The need for new methods to deal with big data is a common theme in most scientific fields, although its definition tends to vary with the context. Statistical ideas are an essential part of this, and as a partial response, a thematic program on statistical inference, learning, and models in big data was held in 2015 in Canada, under the general direction of the Canadian Statistical Sciences Institute, with major funding from, and most activities located at, the Fields Institute for Research in Mathematical Sciences. This paper gives an overview of the topics covered, describing challenges and strategies that seem common to many different areas of application, and including some examples of applications to make these challenges and strategies more concrete.