# Outliers (typically) cannot cause type I errors in one-sample/paired t-tests

## TL;DR

Outliers rarely cause false positives in t-tests unless specific conditions are met, such as small effect sizes and large sample sizes.

## Contribution

The study derives mathematical bounds and empirical evidence showing when outliers can increase t-statistics.

## Key findings

- Outliers can increase t-statistics only under narrow conditions like concordant direction and small effect sizes.
- Monte-Carlo simulations and data set surveys support the derived mathematical bounds.
- The risk of type I errors from outliers is low in small sample sizes.

## Abstract

The presence of outlying data points can have a significant impact on statistical modeling and significance testing. In the specific context of one-sample t-tests, prior studies have shown (primarily through simulations) that outliers make it more likely for t-tests to fail to reject the null hypothesis. In this study, we investigate the opposite scenario: when an outlier can cause the rejection of the null hypothesis. While it may seem intuitive that outliers aligned with the direction of an effect strengthen that effect, prior studies have shown that this is not always the case. Towards this end, we introduce mathematical bounds on how large outliers can be while still increasing the t-statistic in a given sample. These bounds are validated and supported using Monte-Carlo simulations and a survey of available data sets. From these results, we find that although it is not impossible for outliers to cause significant results in paired or one-sample t-tests, it can only occur under rather narrow circumstances. Specifically, it requires a concordant outlier, a minimal sample size of (n≥15), and a sufficiently small effect size (μ^/σ^≤1/2). Based on these findings, we argue that the risk of isolated outliers causing type I errors is low in many practical situations, especially when sample sizes are small.