Narrative Synthesis in Systematic Reviews: A Guide

Narrative synthesis is a structured, transparent approach to combining findings from multiple studies in a systematic review when statistical pooling through deep dive into meta-analysis is not appropriate or not possible. Approximately half of all Cochrane systematic reviews use narrative synthesis for at least some outcomes, making it one of the most common synthesis methods in evidence-based research.

Narrative synthesis is not a fallback or lesser alternative to meta-analysis. It is the correct methodological choice when included studies are too heterogeneous in their populations, interventions, outcomes, or designs to justify combining them into a single pooled effect estimate. A well-conducted narrative synthesis identifies patterns across studies, explores relationships between study characteristics and findings, and draws transparent conclusions that inform practice and policy.

When Narrative Synthesis Is Appropriate

Use narrative synthesis instead of meta-analysis when any of the following conditions apply:

• Clinical heterogeneity. Studies examine sufficiently different interventions, populations, or settings that a single pooled estimate would be misleading
• Methodological heterogeneity. Studies use different designs (mixing randomized controlled trials with observational studies) or different outcome measurement instruments
• Statistical heterogeneity. Even when pooling is technically possible, very high I-squared values (above 75-80%) may indicate that a pooled estimate obscures important variation
• Insufficient data. Studies do not report enough quantitative data to calculate effect sizes for pooling
• Too few studies. With fewer than 3-5 studies addressing a specific comparison, meta-analysis provides limited additional information
• Incompatible outcome measures. Studies measure the same construct using different scales or metrics that cannot be standardized

The SWiM Reporting Guideline

The Synthesis Without Meta-analysis (SWiM) reporting guideline, published in 2020, provides a 9-item checklist for transparent reporting of narrative synthesis in systematic reviews. SWiM complements read about prisma 2020 and should be used alongside it.

Following SWiM transforms narrative synthesis from an unstructured description of individual studies into a rigorous, replicable analytical process that reviewers and editors can evaluate.

Structured Methods for Narrative Synthesis

Step 1: Group Studies for Synthesis

Organize your included studies into meaningful groups based on shared characteristics. Common grouping strategies include:

• By outcome. Group all studies measuring the same outcome together
• By intervention type. Group studies testing similar interventions
• By population. Group studies of similar patient populations
• By comparison. Group studies with the same comparator

Each group is then synthesized separately. This is analogous to conducting separate meta-analyses for different subgroups, except the synthesis is qualitative rather than quantitative.

Step 2: Tabulate Study Results

Create structured evidence tables that present key information from each study in a standardized format. Include study design, population, sample size, intervention details, outcome measures, main findings (with effect estimates and confidence intervals where available), and learn about risk of bias assessment results.

Step 3: Analyze Patterns Across Studies

Move beyond describing individual studies to identifying patterns:

• Consistency. Do studies generally point in the same direction (benefit, harm, or no effect)?
• Magnitude. Are effect sizes similar across studies, or do some show much larger effects?
• Dose-response. Do studies with higher doses or longer durations show larger effects?
• Quality-effect relationship. Do higher-quality studies show different results than lower-quality studies?

Step 4: Use Visual Displays

Visual tools can reveal patterns that are difficult to discern from tables alone:

• Harvest plots. Bar charts where each bar represents one study, positioned to show the direction of effect, with bar height indicating sample size and shading indicating quality
• Albatross plots. Scatter plots of p-values against sample sizes that allow visual assessment of effect magnitude without requiring standardized effect sizes
• Effect direction plots. Tables with directional arrows showing whether each study found a positive, negative, or null effect for each outcome

Need help synthesizing your systematic review evidence? Our methodologists select the optimal synthesis approach and produce publication-ready results with transparent reporting. get started with professional research support to discuss your project, or explore our narrative synthesis support.

Step 5: Apply GRADE for Certainty Assessment

The learn about grade framework can be applied to narrative synthesis, although the process requires judgment rather than statistical calculation. Rate the certainty of evidence for each outcome as high, moderate, low, or critically low, considering risk of bias, inconsistency, indirectness, imprecision, and publication bias across the body of evidence.

Quantifying Direction of Effect: A Worked Sign-Test Vote Count

The next section warns against naive vote counting, and rightly so. But the SWiM-endorsed alternative, direction-of-effect synthesis, has a formal statistical backbone that most narrative reviews omit: the binomial sign test. When you cannot pool effect sizes (different outcome scales, no usable variances), you can still ask a precise, testable question: if the intervention truly had no effect, how surprising is it that so many studies point the same way? This converts an impressionistic "most studies favored treatment" into a defensible probability, and it is the single most useful quantitative tool available when meta-analysis is off the table.

The logic. Under a true null of no effect, each study is equally likely to point in the favorable or unfavorable direction, so the count of favorable studies follows a binomial distribution with p = 0.5. The sign test computes the probability of observing at least as many favorable studies as you did, by chance alone. Ties (studies with effect exactly zero, or genuinely null) are conventionally dropped from the count, which reduces n.

A worked example. Suppose your review identifies 8 studies of an intervention, each reporting a usable direction of effect, and 7 of the 8 favor the intervention. The one-sided sign-test probability of 7 or more favorable out of 8 under the null p = 0.5 is:

P(X >= 7 | n = 8, p = 0.5) = C(8,7)(0.5^8) + C(8,8)(0.5^8)
                           = [8 + 1] / 256
                           = 9 / 256
                           = 0.035

So P = 0.035 one-sided. You can report this directly: "Seven of eight studies favored the intervention; a binomial sign test rejects the null of no consistent direction (one-sided P = 0.035)." That is a quantitative, reproducible statement a peer reviewer can check, and it is far stronger than "the majority of studies were positive." Thomson and Thomas (2013) formalized this approach for exactly the situation where effect-size pooling is impossible but direction is recoverable.

In R, the entire calculation is one line:

binom.test(7, 8, p = 0.5, alternative = "greater")
# number of successes = 7, number of trials = 8, p-value = 0.03516

Its limits, stated honestly. The sign test counts direction only; it ignores magnitude, precision, and sample size, which is precisely the information meta-analysis would use. A significant sign test tells you the direction is consistent, not that the effect is large or clinically meaningful. It also has low power with few studies: with 5 studies, even a perfect 5 of 5 gives P = 0.031, and 4 of 5 gives a non-significant P = 0.19, so absence of significance is not evidence of no effect. Report it as one structured strand of evidence alongside the harvest plot and GRADE rating, never as a substitute for the effect-size synthesis you would have run if the data had allowed it. For the threshold-based reasoning behind why direction alone is weaker than a pooled estimate, see our effect size calculation guide.

What to Avoid in Narrative Synthesis

Vote Counting by Statistical Significance

Simple vote counting (tallying how many studies found "statistically significant" results) is discouraged because it ignores effect magnitude, sample size, and precision. A large study finding a small but significant effect and a small study finding a large but non-significant effect contain very different information that vote counting treats identically.

Acceptable alternative: Direction-of-effect vote counting that considers the direction, magnitude, and confidence intervals of effects across studies. This approach, recommended by SWiM, acknowledges that studies can point in the same direction even if not all reach statistical significance.

Unstructured Study-by-Study Description

Simply describing each study in sequence ("Study A found X. Study B found Y.") without analysis of patterns, relationships, and the overall direction of evidence is not narrative synthesis. It is a study-by-study description that provides no added value over reading the original studies.

Combining Narrative Synthesis With Meta-Analysis

Many systematic reviews use both approaches. Meta-analysis is conducted for outcomes where studies are sufficiently homogeneous, while narrative synthesis is used for outcomes where pooling is not appropriate. This mixed approach is perfectly acceptable and often reflects the reality that some comparisons within a review are suitable for statistical pooling while others are not.

For our guide to forest plots of meta-analyzed outcomes and structured tables for narratively synthesized outcomes, use our free open-access forest plot generator and PRISMA flow diagram tool.

Narrative synthesis overlaps with but differs from qualitative synthesis approaches like meta-ethnography, which follow more formalized analytic frameworks.

When narrative synthesis is combined with statistical pooling, you may be integrating narrative and quantitative findings in mixed methods reviews.