OmVitalis Academy ▶  Watch Episode 4BAll guidesCourseEnroll free
Academy
Multi-omics, Demystified · Episode 4B

Normalization Methods Explained

What each normalization method does to your data, what it adjusts, what it leaves untouched, and which to choose for transcriptomics and metabolomics. Every section has a live lab: change the data and watch the method work, or fail.

Why normalize at all?

Every omics measurement mixes biology with the way the sample was handled and measured. Normalization removes the handling, so that what is left is the biology.

Live lab

Sequence one sample deeper

Two RNA samples, 400 genes. Of these, 40 truly differ (orange) and 360 do not. Sample B is sequenced more deeply than sample A. Raise the depth ratio and watch what happens to the genes that did not change.

unchanged genestruly changed genesdashed lines: 2-fold
Median log2 ratio, unchanged0
Unchanged but look 2-fold different0
Truly changed, still visible0

What each kind of variation looks like

Technical

Depth, input and dilution

Sequencing depth, how much RNA went into the library, how dilute a urine sample was, how much material was loaded on the column.

Technical

Drift and batch

Instrument sensitivity changing across a run, reagent lots, operators, days. Often shared by many features at once.

Biological

What you came for

Differences between groups, treatments and time points. Normalization must leave these alone, which is why every method makes an assumption about what stays the same.

Remember

No method rescues a bad design

If batch and group are the same thing, no normalization can separate them. Randomize and balance first, as in Episode 11.

LevelWhat it acts onQuestion it answersExamples in this guide
Sample levelOne factor per sample (a column)Is this sample simply bigger, deeper or more dilute?CPM, TPM, TMM, median of ratios, PQN, total sum, median, internal standards
Distribution levelThe whole shape of each sampleDo all samples have the same distribution of values?Quantile normalization, cyclic loess, median centring
Feature levelOne transformation per feature (a row) or for all valuesDoes the spread of a feature depend on its size? Do features have comparable weight?log2, arcsinh, VST, centring, autoscaling, Pareto
Batch and driftPosition in the run, batch labelDid the measurement change over time or between batches?QC-based LOESS, ComBat, removeBatchEffect, RUV

Counts, CPM, RPKM and TPM

Raw RNA-seq counts depend on depth and on gene length. CPM, RPKM, FPKM and TPM each remove a different part of that, and each leaves something behind.

Live lab

Change the unit, watch the totals

Eight genes of different lengths in two samples. Sample 2 was sequenced 1.6 times deeper. Choose a unit and compare the two samples gene by gene, and compare the totals.

Unit
Sample 1Sample 2
Sample 1, sum of values0
Sample 2, sum of values0

The formulas

UnitFormula for gene g in a sampleRemovesLeaves behind
CPMcount_g / total counts x 10^6Sequencing depthGene length, composition
RPKM, FPKMcount_g / (length in kb x total counts in millions)Depth and gene lengthSums differ between samples; composition
TPM(count_g / length in kb) / sum over genes of (count / length in kb) x 10^6Depth and gene lengthComposition. Every sample sums to one million

FPKM is RPKM for paired-end data, where a fragment is counted once instead of twice. For testing differences between groups, use raw counts in a count-based model (DESeq2, edgeR, limma-voom) and let the model handle depth. CPM, RPKM and TPM are for plots, filtering and descriptions.

Composition bias: TMM, median of ratios and upper quartile

Dividing by the total assumes the total amount of RNA is the same in every sample. When a few genes take a large share of the reads, every other gene looks lower, although nothing happened to it.

Live lab

Let six genes take over sample B

Two samples with the same number of reads. In sample B, six genes (orange) take a growing share of the reads. The other 494 genes are identical in both samples. Pick a method and see which one still centres them on zero.

Method
genes that did not changethe six dominant genes
Bias of the bulk, log20
Unchanged genes off by 1.5-fold0
Factor applied to B1

How each method finds its factor

MethodHow the factor is foundAssumptionWhere it is used
Total countLibrary sizeThe total RNA is the same in every sampleCPM, TPM, plots
Upper quartile75th percentile of the non-zero counts in each sampleThe upper quartile is not driven by changing genesedgeR (calcNormFactors), older pipelines
TMMWeighted mean of the log ratios against a reference sample, after trimming 30 percent of the ratios and 5 percent of the average expression at each endMost genes are not differentially expressededgeR, limma-voom
Median of ratiosDivide each gene by its geometric mean over samples, then take each sample's median ratioMost genes are not differentially expressed; needs genes with no zerosDESeq2 size factors

All four factors are used as scaling factors or offsets in the model, so the counts themselves stay integers. TMM and median of ratios are the usual choices for differential expression of bulk RNA-seq.

Quantile normalization

Quantile normalization forces every sample to have exactly the same distribution. It is powerful when most features do not change, and wrong when whole distributions truly differ.

Live lab

Force four samples onto one distribution

Four samples of 300 log2 intensities, each shifted and stretched by the measurement. Apply median centring or quantile normalization, then switch on a real biological shift in sample D to see what the method cannot tell apart.

Method
ABCD
Spread of medians0
Spread of IQRs0
D minus A, B, C0

The algorithm in four steps

  1. SortSort the values of each sample from smallest to largest.
  2. Average each rankAcross samples, average the smallest values, then the second smallest, and so on.
  3. ReplaceGive every value the average of its rank. All samples now share one distribution.
  4. Restore the orderPut each value back in its original feature. Ties are handled by averaging.

It was built for microarrays and is common in proteomics. It assumes most features do not change. If one condition really has more of everything, for example a transcription factor that raises most genes, quantile normalization removes the very effect you wanted to measure.

Metabolomics: sum, median, IQR, PQN and internal standards

In metabolomics the biggest sample-level problem is often how concentrated the sample was. Each method estimates a dilution factor in a different way.

Live lab

Recover the dilution factor

Fourteen urine-like samples (7 and 7), 60 metabolites. Each sample was diluted by a different amount, and group B is on average more dilute. Six metabolites truly rise in B. Choose a method and see whether it recovers the real dilution and what it does to the results.

Method
group Agroup Bdotted line: a perfect estimate
Unchanged flagged, p below 0.010
True changes found0
Factor against truth, r0

What each method divides by

MethodFactor for sample iStrengthWeakness
Total sumSum of all feature intensitiesSimple, widely used for NMR and MSOne abundant feature that changes drags all others the opposite way (closure)
MedianMedian intensity of the sampleNot moved by a few abundant featuresNeeds most features detected in all samples
IQRInterquartile range of the sampleRobust, uses the spread as well as the centrePoor when many values are missing or the distributions differ in shape
PQNMedian of the quotients of the sample against a reference spectrum, after sum normalizationDesigned for dilution, robust to large changes in a few signalsAssumes most signals do not change
Internal standardSignal of a compound added at a known amountCorrects extraction and injection variationDoes not correct biological dilution, because it is added after the sample is collected

Creatinine is sometimes used to normalize urine. It varies with age, sex, muscle mass and kidney function, so it is not a neutral reference. Check that your design allows it before using it.

Transformations: log, square root, arcsinh and variance stabilization

In counts and intensities the spread grows with the mean. A transformation makes the spread similar across the range, so low and high features can be compared fairly.

Live lab

Flatten the mean and spread relationship

300 simulated features, abundance from 1 to 100,000 counts, 10 samples each. Every point is a feature, with its standard deviation after the transformation. A good transformation makes the cloud flat.

Transformation
one featuretrend across abundance
SD range across abundance0
Verdict-

The family of transformations

TransformationHandles zerosWhat it doesTypical use
log2(x + c)With a pseudocountMakes fold changes additive; the pseudocount sets how low counts are treatedIntensities, log-CPM, plots
Square rootYesStabilizes Poisson noise only; weaker than logCounts with low overdispersion
Generalized log, arcsinhYesLinear near zero, logarithmic for large valuesMass spectrometry, cytometry, the basis of VSN
VST and rlog (DESeq2)YesUses the fitted mean and dispersion relationship of the dataPCA and clustering of RNA-seq
voom (limma)YesLog-CPM plus precision weights from the mean and variance trendDifferential expression with linear models

A transformation changes the scale on which a difference is measured. After a log2 transformation a difference of 1 means two-fold. Report results on the scale you used.

Feature scaling: centring, autoscaling, Pareto, range, vast and level

Multivariate methods such as PCA treat large variances as important. Scaling decides which features get a voice.

Live lab

Decide who dominates the PCA

Forty samples, 12 features. Three abundant features vary together because of the measurement (orange). Seven scarce features carry the real group difference (blue). Two scarce features are only noise (grey). Choose a scaling and watch the PCA.

Scaling

Samples on PC1 and PC2

group Agroup B

What drives PC1 (loadings)

abundant, measurementscarce, real signalscarce, noise
Variance on PC10
Group separation (best PC)0
Share of PC1 from abundant features0

The formulas (x is a value, mean and s are the feature mean and standard deviation)

ScalingFormulaEffectCaution
Centringx - meanMoves every feature to zero mean. Keeps the original scaleLarge features still dominate
Autoscaling (z-score)(x - mean) / sEvery feature has variance 1Inflates noise in features that are barely measured
Pareto(x - mean) / sqrt(s)Reduces the weight of large features without removing itA compromise, common in metabolomics
Range(x - mean) / (max - min)Every feature spans a similar rangeVery sensitive to outliers
Vast(x - mean) / s x (mean / s)Autoscaling, weighted toward stable features (low variation)Needs positive, reliable means
Level(x - mean) / meanShows relative changeNeeds positive means

Scaling changes the weight a feature has in PCA, clustering and machine learning. It does not change a univariate t-test on the same feature, because the group difference and its spread are scaled together.

Drift and batch: pooled QC samples, LOESS and batch correction

Instruments drift. Pooled quality control samples, measured at regular intervals, show how much, and let you correct for it.

Live lab

Correct the run with the pooled QCs

60 injections, every third a pooled QC (dark dots). Group B truly produces 12 percent more signal and was run later on average. Turn on the signal drift, then correct with a LOESS curve fitted to the QCs. Half of the QCs fit the curve and half are held out to check it.

group Agroup Bpooled QCLOESS fit
Held-out QC variation (RSD)0
Group B / A (truth 1.12)0
p value, B against A0

Which tool for which problem

ProblemToolNeedsRisk
Drift within a runQC-based LOESS (QC-RSC), SERRFPooled QCs spread across the runToo few QCs, or a span that is too small, fits noise
Batches that differ in mean or spreadComBat, removeBatchEffect, RUVBatch labels, or control featuresIf batch and group are the same, the group effect is removed too
Batch as part of the analysisInclude batch in the model (Episode 11)A balanced designOften the safest choice for testing

Correct drift with the QCs and test with the samples. Never judge a correction by the QCs that were used to fit it; hold some QCs back, as the lab does.

The method atlas

Every method in one table. Filter by omics, by the level it works at, or by what you need it to adjust.

Live lab

Find the methods that fit your problem

Pick a kind of data and a problem. The table keeps only the methods that apply, and says what each one changes and what it leaves untouched.

Data
Adjusts for

Choosing for your own study

There is no best normalization, only the one whose assumption your data satisfy. Describe your study and get a starting pipeline with the reason for every step.

Live lab

Build a pipeline

Choose the data, the goal and anything that applies. The steps follow the usual order: filter, correct, normalize, transform, scale. Episode 5 explains why the order matters.

Data
Goal

    Four rules that hold everywhere

    Rule 1

    Know the assumption

    Every method assumes something stays constant: the total, most features, the distribution, a spiked standard. Check yours.

    Rule 2

    Look before and after

    Plot the distributions, a PCA and the QCs before and after. If normalization added structure, stop.

    Rule 3

    Fit on training data only

    In prediction, learn scaling factors inside each cross-validation fold, so the test samples never influence them.

    Rule 4

    Write down the method

    Name the method, the software version and the order in your methods section, so someone else can repeat it.

    Checklist

    Tick as you go. Your ticks stay in this browser.

    0 of 0

    Before

    During

    After

    Shape the next lesson

    This series is built from the questions researchers actually ask. Tell us where you got stuck and what you want next. It takes under a minute.

    Turn on JavaScript to send your questions, or write to wael@omvitalis.com.

    References

    Primary sources for the methods in this guide. Check each DOI against the publisher page before you cite.

    1. TMMRobinson MD, Oshlack A. A scaling normalization method for differential expression analysis of RNA-seq data. Genome Biol 2010;11:R25.doi:10.1186/gb-2010-11-3-r25
    2. Median of ratiosAnders S, Huber W. Differential expression analysis for sequence count data. Genome Biol 2010;11:R106.doi:10.1186/gb-2010-11-10-r106
    3. DESeq2Love MI, Huber W, Anders S. Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biol 2014;15:550.doi:10.1186/s13059-014-0550-8
    4. Comparing methodsBullard JH, Purdom E, Hansen KD, Dudoit S. Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments. BMC Bioinformatics 2010;11:94.doi:10.1186/1471-2105-11-94
    5. Assumptions of each methodEvans C, Hardin J, Stoebel DM. Selecting between-sample RNA-Seq normalization methods from the perspective of their assumptions. Brief Bioinform 2018;19:776-792.doi:10.1093/bib/bbx008
    6. RPKMMortazavi A, Williams BA, McCue K, Schaeffer L, Wold B. Mapping and quantifying mammalian transcriptomes by RNA-Seq. Nat Methods 2008;5:621-628.doi:10.1038/nmeth.1226
    7. TPMWagner GP, Kin K, Lynch VJ. Measurement of mRNA abundance using RNA-seq data: RPKM measure is inconsistent among samples. Theory Biosci 2012;131:281-285.doi:10.1007/s12064-012-0162-3
    8. voom and limmaLaw CW, Chen Y, Shi W, Smyth GK. voom: precision weights unlock linear model analysis tools for RNA-seq read counts. Genome Biol 2014;15:R29.doi:10.1186/gb-2014-15-2-r29
    9. Quantile normalizationBolstad BM, Irizarry RA, Astrand M, Speed TP. A comparison of normalization methods for high density oligonucleotide array data based on variance and bias. Bioinformatics 2003;19:185-193.doi:10.1093/bioinformatics/19.2.185
    10. PQNDieterle F, Ross A, Schlotterbeck G, Senn H. Probabilistic quotient normalization as robust method to account for dilution of complex biological mixtures. Anal Chem 2006;78:4281-4290.doi:10.1021/ac051632c
    11. Internal standardsSysi-Aho M, Katajamaa M, Yetukuri L, Oresic M. Normalization method for metabolomics data using optimal selection of multiple internal standards. BMC Bioinformatics 2007;8:93.doi:10.1186/1471-2105-8-93
    12. Variance stabilizationHuber W, von Heydebreck A, Sultmann H, Poustka A, Vingron M. Variance stabilization applied to microarray data calibration and to the quantification of differential expression. Bioinformatics 2002;18(Suppl 1):S96-S104.doi:10.1093/bioinformatics/18.suppl_1.S96
    13. Centring and scalingvan den Berg RA, Hoefsloot HCJ, Westerhuis JA, Smilde AK, van der Werf MJ. Centering, scaling, and transformations: improving the biological information content of metabolomics data. BMC Genomics 2006;7:142.doi:10.1186/1471-2164-7-142
    14. QC-based drift correctionDunn WB, et al. Procedures for large-scale metabolic profiling of serum and plasma using gas chromatography and liquid chromatography coupled to mass spectrometry. Nat Protoc 2011;6:1060-1083.doi:10.1038/nprot.2011.335
    15. Batch correctionJohnson WE, Li C, Rabinovic A. Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics 2007;8:118-127.doi:10.1093/biostatistics/kxj037
    16. Single-cell size factorsLun ATL, Bach K, Marioni JC. Pooling across cells to normalize single-cell RNA sequencing data with many zero counts. Genome Biol 2016;17:75.doi:10.1186/s13059-016-0947-7