What fitting a model means, how categorical and numerical variables enter it, what adjusting for a covariate does to your result and why, what collinearity does, fixed versus random effects, and a practical recipe. Every section has a live lab: change the model and watch the result move.
A model is a recipe for the expected value of a feature, plus noise. Fitting finds the coefficients that leave the least error.
24 people, one feature, and their ages. Lift and tilt the line to make the orange residual bars as short as you can. Then let least squares do it.
feature = baseline + slope x age + noiseThe baseline and the slope are coefficients. Each answers one question.
The gap between a measured value and the value the recipe predicts for that sample.
Choose the coefficients that make the sum of squared residuals as small as possible.
24 people. Overall mean only: squared residuals 10.2. Baseline 5.75 plus 0.048 per year: 2.1. Age explains 79 percent of the variation.
The type of a variable decides how it enters the model.
Six samples. Add variables to the model and watch each one become columns of numbers. Change a reference level and see which column flips.
| Type | Examples | How it enters | What its coefficient means |
|---|---|---|---|
| Categorical | Treatment, sex, batch | Dummy coding: one 0 and 1 column per non-reference level | Difference from the reference level |
| Numerical, continuous | Age, BMI, dose | As it is, one column | Change in the feature per extra unit |
| Numerical, discrete | Number of visits | As it is, one column | Change per extra count |
Put the columns side by side and you have the design matrix: one row per sample, one column per coefficient. The reference level of every categorical variable is the baseline, all zeros.
Adjusting for a covariate means comparing like with like. It can remove a false difference or sharpen a real one.
A simulated study with 30 cases, 30 controls and 500 features. The cases are older than the controls and most were run in the second batch. Only 40 features truly differ with disease.
Each dot is one feature. Dots above the dashed line pass FDR 0.05. Click any dot to inspect it.
This is simulated data built to show the mechanism, so we know which features are truly different. In real data you never see the colours, which is why the design and the model matter.
15 per group, treated about 56 years old, controls about 34, and the feature rises with age. Without age: difference +1.20, p < 0.001. With age in the model: -0.22, p = 0.52. The difference was age.
12 per group, same ages, but age adds noise. Without age: +0.49, standard error 0.38, p = 0.21. With age: +0.68, standard error 0.18, p = 0.001.
| Model | Up | Down | Real calls | False calls |
|---|---|---|---|---|
| Treatment only | 200 | 198 | 97 | 301 |
| Treatment + age | 19 | 20 | 36 | 3 |
Simulation: 1000 features, 20 vs 20, treated older, BH at 5 percent. Features that only respond to age slide back to the centre once age is in the model. Features that truly respond to treatment keep their direction and size but lose some certainty, because age and treatment overlap, so fewer cross the line. The volcano changes from a map of age into a map of treatment.
A confounder is linked to both the group and the feature. A covariate that only explains noise does not change the estimate much, but it shrinks the standard error.
Our simulation of 1000 features, 20 samples per group, treated older. 100 features respond to treatment, 300 respond only to age. Ordinary linear model per feature, BH at 5 percent.
Every sample gives you one degree of freedom. Every coefficient spends one. Add covariates, then decide how much noise they really explain.
| Model | Called changed | Real | False |
|---|---|---|---|
| Without age | 398 | 97 | 301 (76 percent) |
| With age | 39 | 36 | 3 |
| Extra covariates | Residual degrees of freedom | Power | False positive rate |
|---|---|---|---|
| 0 | 22 | 62 % | 4.4 % |
| 2 | 20 | 54 % | 5.9 % |
| 4 | 18 | 50 % | 5.0 % |
| 8 | 14 | 41 % | 4.5 % |
| 12 | 10 | 28 % | 3.8 % |
| 16 | 6 | 19 % | 4.4 % |
The extra covariates here are pure noise. The false positive rate stays near 5 percent, but power drops because every covariate uses a degree of freedom. Adjust for what matters, not for everything.
Two covariates that carry almost the same information. The model cannot tell whose credit is whose.
40 patients. The feature truly depends only on age. Slide the correlation between age and disease duration and watch the model lose its certainty.
40 patients. The feature truly depends only on age. Age alone: 0.49, p < 0.001. Duration alone: 0.47, p < 0.001, only because it travels with age.
Age 0.44, p = 0.39. Duration 0.06, p = 0.91. The variance inflation factor is about 18, so standard errors are about 4 times larger.
Over 200 bootstrap resamples the two coefficients moved in opposite directions (correlation -0.98) while their sum stayed steady.
Drop duration and the age estimate (95 percent interval) tightens about 4 times, the variance inflation factor falls from 18 to 1, and the resampled coefficients collapse from an elongated cloud onto a single stable value. Duration carried almost nothing that age did not already say.
All treated samples run in one batch: the treatment and batch columns are identical, and no model can separate them. Cross treatment and batch when you design the study.
What to do: check the correlation between covariates, keep one of a correlated pair or combine them, and read an unstable coefficient as a data limit, not a modelling error. The variance inflation factor is 1 / (1 - r squared) for two covariates.
8 subjects measured before and after treatment. Tell the model that the measurements come in pairs and watch what it can suddenly see.
Levels you chose and care about, and would reuse: treatment, sex, genotype. One coefficient per level.
Levels are a sample from a larger population: subjects, litters, labs, plates. You estimate how much they vary, not each one.
Each level estimate is pulled toward the overall mean, more when its data are noisy or few.
| Model | Treatment effect | p |
|---|---|---|
| Subject ignored | 1.3 | 0.31 |
| Subject in the model | 1.3 | 0.007 |
Estimated shrink factor 0.89 for subject baselines. With complete, balanced data, a random subject term gives the same answer as the paired test. Random effects pay off with many subjects, missing visits, and when conclusions should extend beyond these subjects. In limma, duplicateCorrelation handles repeated subjects. In lme4, the term is (1 | subject).
Describe your design and get the formula for each tool, with the reason for every term.
| Role | Handling |
|---|---|
| Variable of interest (treatment) | The term you test; report its adjusted fold change and adjusted p value |
| Covariates measured before treatment (age, sex, batch) | Adjust for them |
| Categorical covariate | Dummy columns |
| Numeric covariate | A slope |
| Repeated subjects, litters, labs | A block term or a random effect |
~ 0 + group + age + sex + batchThen contrasts between group columns.
~ batch + sex + age + conditionCovariates first, the variable of interest last.
Shows which variables drive the data before you choose the model.
That removes part of the effect you want to measure. Keep covariates few compared with samples.
Tick as you go. Your ticks stay in this browser.
This series is built from the questions researchers actually ask. Tell us where you got stuck and what you want next. It takes under a minute.
Turn on JavaScript to send your questions, or write to wael@omvitalis.com.
DOIs were checked against Crossref. Check the details against the publisher page before you cite.