OmVitalis Academy ▶  Watch Episode 7All guidesCourseEnroll free
Outlier Tests Reference Guide
Omics Stats Decoded · Episode 7 · OmVitalis Systems

Outlier Tests Reference Guide

Two common tests for omics tables: the Grubbs test for single values and the Mahalanobis distance for whole samples. Formulas, cutoff tables, the quick rules, what to do after a flag, and a checklist that remembers what you ticked.

Two levels, two tests

A test cannot tell a technical fault from a real biological difference. It tells you where to look.

Level 1

A single value

One measurement in one feature. Use the Grubbs test on roughly normal data, so log transform first.

Level 2

A whole sample

One sample across all features. Use the Mahalanobis distance on the first principal components.

Technical

Fix or remove, and say so

Pipetting slip, failed injection, swapped tube.

Biological

Keep and investigate

A patient who truly responds differently may be your discovery.

The Grubbs test

Tests whether the single most extreme value is an outlier.

Statistic

G = max |x - mean| / SD

How many standard deviations the farthest point sits from the mean.

Critical value

(n-1)/sqrt(n) × sqrt(t² / (n-2+t²))

t is the t quantile with n minus 2 degrees of freedom at probability alpha / (2n). Flag the value if G exceeds it.

Example from the video

G = 2.75 vs 2.41

12 samples, mean 20.1, SD 0.86, alpha 0.05. Flagged.

Masking

Two outliers hide each other

Add a second extreme value and G drops to 2.09, below 2.41. Nothing is flagged.

Conditions: roughly normal data, one outlier at a time (remove, then repeat on what is left), at least 7 samples.

Critical values of G, two-sided

n samplesalpha 0.05alpha 0.01
72.0202.139
82.1272.274
92.2152.387
102.2902.482
122.4122.636
152.5482.806
202.7083.001
252.8223.135
302.9083.236
403.0363.381
503.1283.482

Running Grubbs on every feature multiplies the chance of a false flag. In our clean simulation of 600 features, 31 were flagged at alpha 0.05, about 5 percent. At alpha 0.05 / 600, none were. Use value tests with care and treat sample level tests as the main tool.

The Mahalanobis distance

Distance from the centre in units that respect the shape and correlation of the cloud.

Distance

D² = (x - mean)ᵀ S⁻¹ (x - mean)

S is the covariance matrix of the features. A point across the trend gets a large D, a point along it a small one.

Cutoff

Chi-squared with p degrees

Flag a sample when D² exceeds the chi-squared quantile, for example 97.5 percent. The table below gives D itself.

Omics tables

Use the first PCs

With more features than samples the covariance cannot be estimated. Run it on the first few principal component scores, as in Episode 6.

Robust version

Minimum covariance determinant

Fits the covariance to the most central part of the samples, so outliers cannot inflate it. Use it when more than one outlier is possible.

Cutoff for D, by number of components

p97.5 percent99.9 percent
22.723.72
33.064.03
43.344.30
53.584.53
63.804.74
84.195.11
104.535.44

These are approximations that use the chi-squared distribution with the estimated mean and covariance. Treat a flag as a reason to look, not a verdict.

Two quick rules

Not formal tests, but robust and free of normality assumptions.

RuleFlags a value whenWatch for
Box plotIt lies below Q1 minus 1.5 × IQR or above Q3 plus 1.5 × IQROver-flags skewed data. An adjusted box plot exists for skewed distributions.
MAD|x - median| / (1.4826 × MAD) is greater than 3The cutoff is a convention. Some authors use 2.5.

What our simulation showed

30 samples, 600 features, two groups, 3 samples with a shared technical fault, first 4 principal components, cutoff D = 3.34.

MethodResult
Classical Mahalanobis0 of 3 faulty samples flagged. The outliers inflated the covariance.
Robust (MCD)3 of 3 flagged, all above 30. One extra clean sample at 4.5 was also flagged: check it, do not remove it automatically.
Grubbs on every feature, clean data31 of 600 flagged at alpha 0.05. None at alpha 0.05 / 600.

One simulated dataset illustrates the ideas. It is not a universal rate.

After the flag

1

Look

Sample notes, run order, QC and the raw data.

2

Decide

Technical or biological. Technical can go, and you say so. Biological is information.

3

Compare

Run the analysis with and without it. If the conclusion holds, report that.

4

Prefer robust

Rank-based tests and robust scaling before deleting data.

5

Record

The test, the cutoff, the samples removed, and why.

Never remove a point only because it spoils the p value.

Checklist

Tick as you go. Your ticks stay in this browser.

0 of 0

Before

During

After

Shape the next lesson

This series is built from the questions researchers actually ask. Tell us where you got stuck and what you want next. It takes under a minute.

Turn on JavaScript to send your questions, or write to wael@omvitalis.com.

References

DOIs were checked against Crossref. Check the details against the publisher page before you cite.

  1. The Grubbs testGrubbs FE. Procedures for detecting outlying observations in samples. Technometrics 1969;11:1-21.doi:10.1080/00401706.1969.10490657
  2. The distanceMahalanobis PC. On the generalised distance in statistics. Proc Natl Inst Sci India 1936;2:49-55. Original paper, no DOI.
  3. Robust covarianceRousseeuw PJ, Van Driessen K. A fast algorithm for the minimum covariance determinant estimator. Technometrics 1999;41:212-223.doi:10.1080/00401706.1999.10485670
  4. Multivariate outliersFilzmoser P, Garrett RG, Reimann C. Multivariate outlier detection in exploration geochemistry. Comput Geosci 2005;31:579-587.doi:10.1016/j.cageo.2004.11.013
  5. The MAD ruleLeys C, et al. Detecting outliers: do not use standard deviation around the mean, use absolute deviation around the median. J Exp Soc Psychol 2013;49:764-766.doi:10.1016/j.jesp.2013.03.013
  6. Box plot, skewed dataHubert M, Vandervieren E. An adjusted boxplot for skewed distributions. Comput Stat Data Anal 2008;52:5186-5201.doi:10.1016/j.csda.2007.11.008
  7. Handling outliersAguinis H, Gottfredson RK, Joo H. Best-practice recommendations for defining, identifying, and handling outliers. Organ Res Methods 2013;16:270-301.doi:10.1177/1094428112470848
  8. The 1.5 × IQR ruleTukey JW. Exploratory Data Analysis. Addison-Wesley; 1977. Book, no DOI.