How to compare one feature between two groups. The t statistic as signal over noise, how fold change is calculated, how to read a box plot, paired versus unpaired designs, what a p value really says, and how to pick the test. With a checklist that remembers what you ticked.
Every two-group test asks one question: is the difference between the groups large compared with the spread inside them?
difference = mean B - mean AHow far apart the group averages sit.
SE = sqrt(s²A/nA + s²B/nB)The standard error of that difference. More spread or fewer samples make it larger.
t = difference / SEIn the video: 1.38 / 0.76 = 1.82 when the groups were noisy, and 3.55 for the real data. Same difference, different noise, different certainty.
The t distribution converts the statistic into the chance of seeing a value this large if there is no real difference.
How big is the difference, in a form people can read.
| Quantity | How to compute it | Example |
|---|---|---|
| log2 fold change | mean of group B minus mean of group A, both on the log2 scale | 21.26 - 19.89 = 1.38 |
| Fold change | 2 raised to the log2 fold change | 21.38 = 2.6 |
| Direction | Positive means higher in B, negative means lower in B | +1 is double, -1 is half |
On the ratio scale, doubling is +1.0 and halving is -0.5. On the log2 scale they are +1 and -1.
Averaging raw values first lets extreme samples pull the ratio. In our example it gave 3.15 instead of 2.6.
The fold change says how big. The p value says how sure. A feature needs both to be interesting.
For RNA-seq counts, packages such as DESeq2 and edgeR estimate fold change with count models. Episode 9 covers them.
A box plot summarises the middle of the data and flags what lies far from it.
| Part | What it shows |
|---|---|
| Line in the box | The median: half the samples above, half below. |
| Box | Q1 to Q3, the middle 50 percent of samples. Its height is the IQR. |
| Whiskers | The furthest sample within 1.5 × IQR of the box. |
| Dots beyond | Possible outliers by the 1.5 × IQR rule. |
| Not shown | The mean and the number of samples. Overlay the individual points. |
In our example the treated group has one dot far above its box (23.58). The box plot shows it, the t test is pulled by it, and a rank-based test barely notices it.
Ask whether each sample in one group has a partner in the other.
Use Welch's t test. It compares group averages against the spread within groups.
Before and after, tumour and adjacent normal, left and right. Test the one-line differences, so the subject to subject variation cancels.
Unpaired: t = 0.51, p = 0.619. Paired: t = 2.81, p = 0.020. Eight of the ten lines went up.
Pairing is a property of how the study was run, not a choice made after you see the p value.
The chance of a difference at least this large, if there were no real difference between the groups.
If the groups do not differ, the labels are arbitrary. Shuffle them thousands of times and record the difference each time.
The p value is the share of shuffles at least as extreme as the real difference. In the video, about 0.001 from 5000 shuffles.
It assumes no effect and asks how surprising the data are. It does not give the probability that the hypothesis is true.
Give the fold change, an interval and the p value. A permutation p value is never exactly zero: report it as at least 1 / (number of shuffles + 1).
Choose by the design and the shape of the data, before you look at the p value.
| Situation | Test | Why |
|---|---|---|
| Two independent groups, roughly bell shaped | Welch t test | Does not assume equal spread. A safe default. |
| The same subject in both conditions | Paired t test | Removes subject to subject variation. |
| Skewed data, heavy outliers, ordinal scores | Mann-Whitney (unpaired) or Wilcoxon signed rank (paired) | Compares ranks, so extreme values have little pull. |
| Question | Result |
|---|---|
| False positives at 5 percent, unequal spread and unequal group sizes: Student's t | 21.9 percent |
| Same setting: Welch's t | 5.2 percent |
| Power on skewed data: t test | 26.9 percent |
| Power on skewed data: Mann-Whitney | 45.0 percent |
Student's test assumes both groups have the same spread. When they do not, its error rate drifts away from 5 percent. Welch's version stays close to it.
Tick as you go. Your ticks stay in this browser.
This series is built from the questions researchers actually ask. Tell us where you got stuck and what you want next. It takes under a minute.
Turn on JavaScript to send your questions, or write to wael@omvitalis.com.
DOIs were checked against Crossref. Check the details against the publisher page before you cite.