Every meta-analysis rests on an implicit assumption: that no single study is so dominant that removing it would overturn the conclusions. Leave-one-out sensitivity analysis tests that assumption directly. By systematically omitting one study at a time and re-estimating the pooled effect, you produce a clear picture of how much each study drives your result. This procedure is one of the most requested sensitivity checks in peer review, and reviewers for journals indexed in PubMed, Cochrane, and PROSPERO routinely expect to see it reported.
When the pooled effect stays stable across all iterations, your conclusion is robust. When one omission shifts the estimate dramatically, you have an influential study that demands explanation. This guide walks through every aspect of leave-one-out analysis, from visual interpretation to advanced influence diagnostics, with a worked example you can replicate using our free robustness check tool or the metafor package in R.
What Makes a Study Influential
Three mechanisms create influential studies, and understanding them helps you explain your findings to reviewers rather than simply reporting numbers.
Precision leverage occurs when a study has an exceptionally small variance, giving it high statistical weight. A large randomized controlled trial in a pool of smaller observational studies will often dominate the pooled estimate simply because its confidence interval is narrow. Under a random-effects model, precision leverage is partially dampened by the between-study variance component (tau-squared), but it still matters.
Effect size discordance occurs when a study's point estimate is far from the other studies in the pool. A study reporting a standardized mean difference of 1.8 in a pool where the other nine studies range from 0.3 to 0.7 will pull the pooled estimate upward. When that study is removed, the pooled effect drops noticeably.
Heterogeneity contribution is more subtle: a study can inflate tau-squared, changing how all weights are distributed under a random-effects model. Removing such a study not only shifts the point estimate but also narrows the confidence interval for every remaining study because the between-study variance decreases. This is the mechanism most often missed by researchers who focus only on point estimates.
Try our free leave-one-out calculator to identify all three types of influential studies instantly.
How Leave-One-Out Analysis Works
For a meta-analysis with k studies: remove study 1, re-fit the model on k-1 studies, record the pooled effect, confidence interval, p-value, and heterogeneity statistics. Restore study 1, remove study 2. Continue until every study has been omitted once.
The output is a table or forest-style plot showing k rows of pooled estimates, each representing the analysis with one study excluded. In R, the metafor package (Viechtbauer, 2010) provides the leave1out() function that automates this procedure for any fitted rma() model object.
Running the Analysis with Our Free Tool
Navigate to the robustness check tool and enter your study labels, effect sizes, and standard errors. The tool runs the full leave-one-out procedure automatically and displays a leave-one-out forest plot with color coding for studies where exclusion changes significance.
The one-click export provides the equivalent R code using the leave1out() function from the metafor package.
How to Read a Leave-One-Out Forest Plot
A leave-one-out forest plot looks similar to a standard forest plot but communicates different information. Each row represents the pooled result after excluding one study, not the individual study's effect. Here is how to read it systematically.
The vertical reference line marks the overall pooled estimate from the full model (all k studies included). Each row's point estimate shows where the pooled effect lands when that specific study is removed. If all point estimates cluster tightly around the reference line, the meta-analysis is stable.
Confidence intervals on each row represent the precision of the reduced-k model. Watch for rows where the confidence interval crosses the null (zero for mean differences, one for odds ratios) when the full model's interval did not. That crossing signals a significance change, the most important finding in leave-one-out analysis.
Color coding in our online sensitivity analysis highlights rows where removing a study changes statistical significance. Red rows indicate that excluding that study makes the result non-significant. Green rows (less common) indicate that excluding a study strengthens significance by removing heterogeneity.
Use the interactive forest plot builder alongside the leave-one-out plot to compare each study's individual effect against the pooled estimate and visually identify which studies are discordant.
Interpreting Results
When Results Are Robust
If all k pooled estimates cluster tightly around the original estimate and confidence intervals consistently include or exclude zero across all iterations, your conclusion is robust. Report this explicitly. Reviewers value a clear statement such as: "Leave-one-out sensitivity analysis confirmed that no single study altered the direction, magnitude, or statistical significance of the pooled effect."
When One Study Changes Significance
Investigate the influential study along four dimensions: sample size and population, methodological quality, effect size and direction, and publication context.
After investigation, present both the full-model and leave-one-out estimates as co-primary results, or conduct a subgroup analysis separating the outlying study. Document whether the influential study differed in risk of bias, population characteristics, intervention dosage, or follow-up duration.
Use the forest plot software to visualize the study's position relative to the rest of the pool.
When Multiple Studies Are Influential
If removing several different studies each changes significance, the meta-analysis has a fragility problem. Report it honestly and recommend further primary research. Calculate the fragility index to quantify how many studies would need to change their event counts to reverse the result, and present this alongside the leave-one-out findings.
Significance Change Detection
Two criteria define a meaningful change: a clinically meaningful shift in the point estimate (a change exceeding the minimally important difference for the outcome), and a change in statistical significance (confidence interval changes from excluding to including the null, or vice versa).
The interactive funnel plot builder helps identify outliers before running leave-one-out analysis, giving you a visual preview of which studies sit far from the regression line.
When Leave-One-Out Analysis Is Insufficient
Leave-one-out analysis answers one question: what happens to the pooled effect when each study is removed individually? It does not capture jointly influential study pairs, and it does not quantify the specific mechanism of influence. For a deeper diagnosis, you need formal influence diagnostics.
Cook's Distance
Cook's distance measures how much all fitted values change when a study is removed. It combines leverage (how unusual a study's precision is) with residual magnitude (how far the study's effect is from the model prediction). A study with a high Cook's distance is influential through both mechanisms simultaneously. In the metafor package, this is available through the cooks.distance() method on a fitted rma() object.
DFBETAS
DFBETAS quantifies how much each model coefficient changes (in standard error units) when a study is deleted. For a simple random-effects meta-analysis with one pooled estimate, DFBETAS tells you whether a study shifts the intercept by more than a threshold (commonly 2 divided by the square root of k). Studies exceeding this threshold are considered influential.
Hat Values (Leverage)
Hat values measure a study's leverage, which is its potential to influence the pooled estimate based on precision alone. A study with a hat value much larger than the average (which equals the number of coefficients divided by k) has high leverage. High leverage combined with a large residual produces influence. Hat values are available through the hatvalues() method in metafor.
DFFITS
DFFITS measures the change in the fitted value for a study when it is deleted, standardized by the study's standard error. It is the individual-study analog of Cook's distance and follows a threshold of 2 times the square root of the number of coefficients divided by k.
Baujat Plot as a Complement to Leave-One-Out
The Baujat plot (Baujat et al., 2002) provides a two-dimensional view of study influence that leave-one-out analysis alone cannot offer. The x-axis shows each study's contribution to the overall heterogeneity statistic (Q), and the y-axis shows each study's influence on the pooled effect.
Studies in the upper-right quadrant are the most problematic: they contribute substantially to heterogeneity and pull the pooled estimate. Studies in the lower-right quadrant contribute to heterogeneity but do not shift the pooled result much, suggesting they add noise without changing the conclusion. Studies in the upper-left quadrant shift the result despite not contributing much to heterogeneity, indicating precision leverage.
Read our detailed guide on Galbraith plots and Baujat plots for heterogeneity diagnostics for a full walkthrough of visual diagnostic strategies, including when to use each plot type.
To generate a Baujat plot in R with metafor, call baujat(model) on your fitted rma() object. Our meta-analysis sensitivity calculator also displays Baujat-style information when you run the full diagnostics panel.
