A funnel plot is a scatter plot used in meta-analysis to assess whether the collection of studies included in a synthesis is likely affected by publication bias. Each point on the plot represents an individual study, positioned according to its effect size on the horizontal axis and its precision, typically the standard error, on the vertical axis. When no bias is present, the studies form a symmetric, inverted funnel shape centered on the pooled effect estimate.
Funnel plot interpretation is a foundational skill for any researcher conducting or appraising a meta-analysis. Publication bias, the tendency for studies with statistically significant or favorable results to be published more readily than those with null or negative findings, threatens the validity of every systematic review. If the studies entering your synthesis are a biased sample of all research conducted on a topic, your pooled estimate will be biased too. The funnel plot provides a simple, visual method for detecting this problem before it undermines your conclusions.
This guide covers the anatomy of funnel plots, how to assess symmetry and asymmetry, what causes asymmetric patterns, how to supplement visual inspection with formal statistical tests, and worked interpretation examples that show you exactly what to look for.
What Is a Funnel Plot
A funnel plot is a diagnostic graphic introduced by Light and Pillemer in 1984 and refined by Egger and colleagues in subsequent decades. Its purpose is to explore the relationship between study effect sizes and study precision in a meta-analysis. The underlying logic is straightforward: large studies with high precision should cluster tightly around the true effect, while smaller studies with lower precision should scatter more widely. If this expected pattern holds, wide scatter at the bottom, tight clustering at the top, the result is a symmetric, inverted funnel shape.
The name comes from this shape. When you look at a well-behaved funnel plot, the studies form a triangle that narrows toward the top, resembling an inverted funnel. The pooled effect estimate, usually shown as a vertical line, runs through the center. Studies scatter around this line, with the scatter increasing as you move downward toward less precise studies.
Publication bias disrupts this symmetry. If small studies with non-significant results are less likely to be published, the bottom of the funnel will be missing points on one side. This creates a visible asymmetry, an incomplete funnel with a gap where unpublished studies should have appeared.
The Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al., 2023) recommends funnel plots as a standard component of publication bias assessment in meta-analyses containing 10 or more studies. Fewer than 10 studies produce sparse plots where asymmetry cannot be reliably distinguished from random variation.
Anatomy of a Funnel Plot: Axes, Distribution, and Reference Lines
Understanding the axes and visual elements of a funnel plot is essential before attempting interpretation. Each component carries specific meaning.
X-axis: Effect size, The horizontal axis displays the effect size measure used in the meta-analysis. This might be a standardized mean difference (Cohen's d or Hedges' g), a log odds ratio, a log risk ratio, or any other effect metric. Each study's point estimate is plotted along this axis. The pooled effect estimate from the meta-analysis typically appears as a vertical reference line.
Y-axis: Standard error (inverted), The vertical axis displays the standard error of each study's effect estimate. Critically, this axis is inverted: zero (highest precision) is at the top, and larger standard errors (lower precision) are at the bottom. This inversion creates the funnel shape, precise studies cluster at the narrow top, while imprecise studies scatter across the wide bottom.
Some funnel plots use alternative precision measures on the y-axis, including sample size, inverse variance, or the reciprocal of the standard error. The standard error is the most common choice because it relates directly to the confidence interval width and produces the clearest funnel shape.
The inverted funnel shape, In the absence of bias, you expect studies to distribute symmetrically around the pooled effect. Large studies at the top will be very close to the vertical reference line because their estimates are precise. Smaller studies at the bottom will scatter more widely but should be equally likely to fall on either side of the line. The resulting pattern looks like a triangle with its apex at the top, the inverted funnel.
Reference lines, Most funnel plots include at least one reference line at the pooled effect estimate. Some add pseudo-95% confidence limits, diagonal lines extending from the apex of the funnel to define the region where 95% of studies would be expected to fall if there were no heterogeneity. Studies falling outside these limits may warrant investigation for heterogeneity or outlier status.
| Element | Represents | Interpretation |
|---|---|---|
| X-axis | Effect size (SMD, OR, RR) | Direction and magnitude of each study's result |
| Y-axis (inverted) | Standard error | Study precision, top is most precise |
| Vertical line | Pooled effect estimate | Central reference for symmetry assessment |
| Pseudo-CI lines | Expected scatter boundaries | Studies outside suggest heterogeneity |
| Individual dots | Individual studies | Distribution pattern reveals bias |
How to Interpret Funnel Plot Symmetry
The primary interpretive task when examining a funnel plot is assessing whether the distribution of studies is symmetric around the pooled effect estimate. This assessment involves examining the overall shape, looking for gaps, and evaluating whether the scatter is balanced.
Symmetric pattern, A symmetric funnel plot shows studies distributed evenly on both sides of the vertical reference line at every level of precision. The scatter widens gradually and proportionally as you move from precise studies at the top to imprecise studies at the bottom. There are no conspicuous gaps or clusters on one side. A symmetric pattern is consistent with no publication bias, though it does not prove its absence.
Asymmetric pattern, An asymmetric funnel plot shows an imbalance in the distribution of studies. The most common pattern associated with publication bias is a gap in the bottom-right or bottom-left corner of the funnel, where small studies with non-significant results would be expected. Instead of a balanced triangle, the funnel appears to be missing a section, leaning to one side.
When we present funnel plots to clients, the most common question is whether a mild degree of asymmetry should cause concern. The honest answer is that mild asymmetry in a plot with 10-15 studies is common even without bias. Sampling variability alone produces imperfect symmetry. The question is whether the asymmetry exceeds what you would expect from chance.
Assessing visually, Look at the plot from above, imagining folding it along the vertical reference line. If both halves would roughly overlap, the plot is symmetric. If one side clearly contains more studies, or if there is a visible gap on one side, the plot is asymmetric. Pay particular attention to the bottom portion of the plot, where small studies appear, this is where publication bias manifests most clearly because small studies with null results are the most likely to remain unpublished.
Interpretation Examples: Symmetric and Asymmetric Patterns
Concrete examples make interpretation principles tangible. Below are the patterns you will encounter when examining funnel plots in practice, along with what each pattern suggests about the evidence base.
Example 1: Symmetric funnel, no bias concern, Imagine a funnel plot with 20 studies. Three large studies cluster tightly near the pooled effect at the top. Below them, medium-sized studies scatter modestly, roughly equally on both sides. At the bottom, small studies are spread widely but without a clear gap on either side. The overall shape is a balanced inverted triangle. This pattern is consistent with no publication bias and is what you hope to see when assessing your meta-analysis.
Example 2: Asymmetric funnel, classic publication bias, Now imagine a similar plot, but the bottom-left corner is nearly empty. Small studies appear overwhelmingly on the right side of the pooled effect, meaning small studies with large positive effects were published while small studies with null or negative effects were not. The funnel shape is visibly lopsided. This pattern strongly suggests publication bias, specifically the selective publication of statistically significant results from small studies.
Example 3: Asymmetric funnel, bottom-heavy on one side, In some plots, a cluster of small studies appears far from the pooled effect on one side. These outlying small studies pull the pooled estimate in their direction. This pattern can indicate publication bias, but it can also reflect genuine heterogeneity, perhaps a subgroup of small studies examined a different population or used a different intervention variant.
Example 4: Hollow funnel, missing middle, Occasionally you see a funnel plot where large and small studies are present but medium-sized studies are sparse. This pattern is less about publication bias and more about the research landscape. It may reflect a field where studies tend to be either large trials or small pilot studies, with fewer moderate-sized investigations.
| Pattern | Visual Appearance | Most Likely Cause | Action Required |
|---|---|---|---|
| Symmetric funnel | Balanced inverted triangle | No bias evidence | Report as reassuring |
| Bottom-corner gap | Missing small studies on one side | Publication bias | Run Egger's test; consider trim-and-fill |
| One-sided cluster | Small studies clustered on one side | Bias or heterogeneity | Investigate subgroups; contour-enhanced plot |
| Hollow middle | Gap in medium-precision range | Research landscape artifact | Note in limitations |
| Extreme outliers | Points far outside funnel boundaries | Heterogeneity or data errors | Sensitivity analysis; verify data |
Causes of Funnel Plot Asymmetry Beyond Publication Bias
Funnel plot asymmetry is often reflexively attributed to publication bias, but the relationship is not that simple. Several mechanisms can produce asymmetric funnel plots, and distinguishing among them is critical for accurate interpretation.
Publication bias, The most discussed cause. Studies with statistically significant results are more likely to be submitted, accepted, and published. Studies with null findings languish in file drawers. Because significance depends partly on sample size, small null studies are the most likely to be missing, creating the classic asymmetric gap in the bottom corner of the funnel.
Small-study effects, Small studies may genuinely produce different effect estimates than large studies for reasons unrelated to publication bias. In clinical research, small studies may use more intensive interventions, recruit higher-risk patients, or be conducted in specialized centers. These methodological differences can produce larger effects in small studies without any selective publication.
Heterogeneity, When the true effect varies across studies due to differences in populations, interventions, or outcomes, the funnel shape will be distorted. Heterogeneity inflates scatter at all levels of precision and can create apparent asymmetry. High I-squared values (indicating substantial between-study variance) should prompt you to consider heterogeneity as the primary driver of any observed asymmetry.
Chance, With fewer than 20 studies, random variation alone can produce asymmetric-looking funnel plots. This is why the 10-study minimum is a floor, not an ideal. Even with 15 studies, moderate asymmetry may be within the range of chance expectations.
Reporting bias and outcome reporting bias, Selective reporting of outcomes within studies can create funnel plot asymmetry without classic publication bias. If researchers across multiple studies selectively report the outcome measure that produced the most favorable result, the meta-analysis will over-represent positive findings even if all studies were technically published.
Peters et al. (2008) introduced contour-enhanced funnel plots specifically to help distinguish publication bias from other causes of asymmetry. By overlaying contours of statistical significance (p = 0.01, 0.05, and 0.10), these enhanced plots reveal whether missing studies would have been non-significant, a pattern more consistent with publication bias, or whether the asymmetry falls in regions unrelated to significance thresholds.
Statistical Tests to Supplement Visual Inspection
Visual assessment of funnel plot symmetry is inherently subjective. Two researchers examining the same plot may reach different conclusions about whether it is symmetric. Formal statistical tests provide objectivity, and current best practice requires reporting at least one alongside your funnel plot.
Egger's regression test, Egger's test (Egger et al., 1997) regresses the standardized effect estimates (effect size divided by its standard error) against precision (the inverse of the standard error). A statistically significant intercept term indicates funnel plot asymmetry. This is the most widely used test for publication bias in meta-analysis and is recommended by the Cochrane Handbook. The test has adequate power when there are at least 10 studies, though its power increases with more studies.
Begg's rank correlation test, Begg and Mazumdar's test (1994) uses a rank correlation between the standardized effect estimates and their variances. It is conceptually simpler than Egger's test but generally has lower statistical power. Some researchers report both tests for completeness.
Peters' test, Peters' test is a variant of Egger's test designed for binary outcome measures (odds ratios and risk ratios). It regresses the effect estimate against the inverse of sample size rather than against the standard error, which reduces false positive rates when the effect measure is an odds ratio.
Trim-and-fill method, Trim-and-fill (Duval and Tweedie, 2000) is both a test and a correction method. It identifies studies contributing to funnel plot asymmetry, removes them ("trims"), uses the remaining symmetric funnel to estimate the true center, and then imputes the missing studies ("fills") needed to restore symmetry. The adjusted pooled estimate provides a sensitivity analysis showing what the result might be if the missing studies existed. While useful, trim-and-fill should not be treated as a definitive correction because it assumes that asymmetry is entirely due to publication bias.
| Test | What It Does | Best For | Minimum Studies |
|---|---|---|---|
| Egger's regression | Tests intercept of regression on precision | Continuous outcomes | 10+ |
| Begg's rank correlation | Rank correlation between effect and variance | General use (lower power) | 10+ |
| Peters' test | Regression on inverse sample size | Binary outcomes (OR, RR) | 10+ |
| Trim-and-fill | Imputes missing studies, recalculates estimate | Sensitivity analysis | 10+ |
| Harbord's test | Modified Egger's for log odds ratios | Rare events with ORs | 10+ |
When reporting, state both the visual assessment and the statistical test result. For example: "Visual inspection of the funnel plot suggested possible asymmetry, with fewer small studies reporting effects below the pooled estimate. Egger's regression test confirmed statistically significant asymmetry (bias coefficient = 2.31, p = 0.018), indicating potential publication bias." This dual reporting, visual plus statistical, is the standard expected by peer reviewers and guideline groups.
You can compute these tests in R using the metafor package (the regtest function for Egger's test, the trimfill function for trim-and-fill) or in Stata using the metabias and metatrim commands. For researchers who prefer a no-code approach, our funnel plot generator produces publication-ready funnel plots. You can also explore related tools such as the Bayes factor calculator for complementary analyses.
Common Mistakes in Funnel Plot Interpretation
Despite its apparent simplicity, the funnel plot is frequently misinterpreted. Recognizing common mistakes protects the credibility of your publication bias assessment.
Mistake 1: Interpreting asymmetry as proof of publication bias. Asymmetry raises concern but does not establish a cause. As discussed above, small-study effects, heterogeneity, and chance can all produce asymmetric funnels. Stating that "the funnel plot shows publication bias" is an overstatement. Instead, write that "the funnel plot suggests possible publication bias" and investigate alternative explanations.
Mistake 2: Using funnel plots with fewer than 10 studies. With 5 or 7 studies, a funnel plot will look sparse and irregular regardless of whether bias is present. Drawing conclusions from such plots is unreliable. If your meta-analysis has fewer than 10 studies, state this limitation and omit the funnel plot, or include it with an explicit caveat that it cannot be meaningfully interpreted.
Mistake 3: Ignoring heterogeneity as an explanation. If your meta-analysis has high I-squared (above 50-75%), the scatter in your funnel plot will be inflated by genuine between-study differences. Asymmetry in the context of high heterogeneity is more likely to reflect clinical or methodological diversity than selective publication. Always interpret the funnel plot in light of your heterogeneity statistics.
Mistake 4: Relying on visual inspection alone. Subjective visual assessment is necessary but insufficient. Different observers may disagree about whether a plot is symmetric, especially in borderline cases. Always supplement your visual interpretation with a formal statistical test. The combination of funnel plot plus Egger's test is the minimum standard.
Mistake 5: Treating trim-and-fill as a definitive correction. The trim-and-fill adjusted estimate is a useful sensitivity analysis, not a corrected truth. It assumes that all asymmetry is due to publication bias and that the symmetric funnel represents reality. If asymmetry is actually caused by heterogeneity, the trim-and-fill estimate will be biased in the opposite direction. Report trim-and-fill results as exploratory, not confirmatory.
Mistake 6: Forgetting to report the funnel plot at all. PRISMA 2020 and the Cochrane Handbook recommend publication bias assessment for meta-analyses with sufficient studies. Omitting funnel plots from qualifying meta-analyses is a common reviewer criticism. Include the funnel plot in your main text or supplementary materials, even when the result is reassuring.
Mistake 7: Confusing the y-axis orientation. The standard convention places standard error on the y-axis with the scale inverted (zero at top, increasing downward). Some software or older publications use non-inverted axes or alternative precision measures. Verify which convention your software uses and ensure your interpretation matches the actual axis orientation.
For a broader understanding of how funnel plot assessment fits into the complete synthesis workflow, see our guide on how to do a meta-analysis step by step. Researchers investigating the full range of approaches for identifying missing studies should also consult our overview of publication bias detection methods, which covers statistical tests, graphical methods, and sensitivity analyses in detail.
The funnel plot remains one of the most accessible and widely expected components of a well-conducted meta-analysis. Its simplicity is both its strength and its limitation: it communicates an intuitive message about the completeness of your evidence base, but it requires careful interpretation that accounts for alternative explanations. By understanding the anatomy, following a systematic approach to symmetry assessment, supplementing with formal tests, and avoiding the common mistakes outlined above, you will produce publication bias assessments that withstand peer review scrutiny and contribute to the transparency of your systematic review.