An umbrella review is a form of evidence synthesis that exclusively includes systematic reviews and meta-analyses as its unit of analysis. Instead of synthesizing individual primary studies, an umbrella review synthesizes the findings of existing systematic reviews to provide a comprehensive overview of all available synthesized evidence on a broad topic. This makes it the highest level of evidence synthesis in the research hierarchy.
Umbrella reviews are also called "overviews of reviews," "reviews of reviews," or "meta-reviews." They are particularly valuable when multiple systematic reviews exist on overlapping clinical questions and decision-makers need a single, comprehensive summary that compares findings, assesses quality, and identifies gaps in the evidence base. The Joanna Briggs Institute and the Cochrane Collaboration both provide formal methodology for conducting umbrella reviews.
When to Conduct an Umbrella Review
An umbrella review is appropriate when several conditions are met:
- Multiple systematic reviews exist on related aspects of a broad topic. If only one or two systematic reviews exist, an umbrella review adds little value
- A comprehensive overview is needed for clinical guideline development, health technology assessment, or policy decisions
- Systematic reviews reach different conclusions and a structured comparison of their methods and findings is needed to understand why
- The evidence landscape needs mapping to identify which aspects of a topic have been well-synthesized and where gaps remain
Examples of appropriate umbrella review topics:
- All systematic reviews of interventions for fall prevention in older adults
- All systematic reviews examining the association between screen time and child development outcomes
- All systematic reviews of pharmacological treatments for chronic pain
If no systematic reviews exist on your topic, conduct a primary systematic review instead. If you need to compare a systematic review with a narrative review or other non-systematic synthesis, an umbrella review is not the right approach.
Umbrella Review vs Other Evidence Synthesis Types
| Feature | Umbrella Review | Systematic Review | Scoping Review | Rapid Review |
|---|---|---|---|---|
| Unit of analysis | Systematic reviews | Primary studies | Primary studies | Primary studies |
| Purpose | Overview of all synthesized evidence | Answer a specific question | Map the evidence landscape | Quick evidence summary |
| Quality assessment | AMSTAR 2 for included SRs | RoB 2, ROBINS-I, NOS | Optional | Abbreviated |
| Typical timeline | 4-8 months | 12-18 months | 6-12 months | 2-6 months |
| New meta-analysis | Re-analysis of existing meta-analysis results | Yes, from primary data | Rarely | Sometimes |
Step-by-Step Methodology
Step 1: Define the Research Question
Frame your question broadly enough to encompass multiple systematic reviews. Use the PICO framework but with wider parameters than a primary systematic review.
Example: "What is the effectiveness of exercise interventions for managing type 2 diabetes, as reported in published systematic reviews and meta-analyses?"
Step 2: Develop a Protocol
Register your protocol on PROSPERO or the JBI Systematic Review Register. Follow the PRISMA-P checklist for protocol reporting. Your protocol should specify:
- Eligibility criteria (include only systematic reviews and meta-analyses)
- Search strategy across relevant databases
- Quality assessment tool (AMSTAR 2)
- Data extraction plan
- Synthesis approach
Step 3: Search for Systematic Reviews
Search databases that index systematic reviews comprehensively. Essential databases include:
- Cochrane Database of Systematic Reviews (CDSR)
- MEDLINE (via PubMed, using the systematic review filter)
- Embase (with systematic review study type filter)
- JBI EBP Database
- Epistemonikos (a database specifically designed for evidence synthesis)
Use Boolean search strategies that include methodology terms ("systematic review" OR "meta-analysis" OR "evidence synthesis") combined with your topic terms. Document your complete search strategy for PRISMA compliance.
Step 4: Screen and Select Reviews
Apply your eligibility criteria using dual-reviewer screening at the title-abstract and full-text stages. Include only:
- Studies self-identified as systematic reviews or meta-analyses
- Studies that report a systematic search of at least two databases
- Studies with explicit inclusion/exclusion criteria
Exclude narrative reviews, scoping reviews, literature reviews, and evidence summaries that do not meet systematic review criteria. Document all exclusion reasons for your PRISMA flow diagram.
Step 5: Assess Quality Using AMSTAR 2
AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews, version 2) is the standard quality assessment instrument for umbrella reviews. It contains 16 items that evaluate critical methodological domains:
- Was the protocol registered before the review began?
- Was the literature search comprehensive?
- Were inclusion criteria clearly defined?
- Was risk of bias assessment performed?
- Were appropriate meta-analytical methods used?
- Was publication bias assessed?
AMSTAR 2 classifies overall confidence in each included systematic review as high, moderate, low, or critically low based on the number and type of critical and non-critical weaknesses identified. This quality grading helps readers understand the reliability of the evidence underpinning your umbrella review's conclusions.
Step 6: Extract Data
Extract key information from each included systematic review:
- Review characteristics: Authors, year, databases searched, date range, number of included primary studies
- PICO elements: Population, intervention, comparator, outcomes
- Findings: Main results, pooled effect estimates from meta-analyses, confidence intervals
- Quality: AMSTAR 2 overall rating and domain-level assessments
- Limitations: Key methodological weaknesses reported by the review authors
Need expert help conducting your umbrella review? Our team handles the entire process from search strategy through AMSTAR 2 quality assessment and synthesis. Get a free quote to discuss your project, or explore our evidence synthesis services.
Step 7: Synthesize Findings
Organize your synthesis by outcome, intervention, or population subgroup. For each outcome:
- Summarize the number and quality of systematic reviews that address it
- Compare effect estimates across reviews (do they agree or disagree?)
- Assess the overlap of primary studies across reviews (many reviews may include the same studies, inflating the apparent evidence base)
- Grade the certainty of evidence using GRADE applied at the umbrella review level
- Identify gaps where systematic reviews are needed but do not exist
Overlap assessment is a unique and important step in umbrella reviews. Calculate the corrected covered area (CCA) to quantify how much the primary studies overlap across included reviews. High overlap means the reviews are largely synthesizing the same evidence, while low overlap suggests different reviews have captured different bodies of evidence.
Step 8: Report Using PRISMA
Report your umbrella review following PRISMA 2020 guidelines. Include a flow diagram showing the number of systematic reviews identified, screened, assessed, and included. Present your AMSTAR 2 results in a summary table. Include a matrix showing the overlap of primary studies across included reviews.
Common Challenges in Umbrella Reviews
Dealing With Overlapping Primary Studies
Multiple systematic reviews on the same topic frequently include many of the same primary studies. This creates a risk of double-counting evidence. Address this by:
- Creating a matrix of primary studies included across reviews
- Calculating the CCA (corrected covered area) metric
- Presenting findings with awareness that apparent agreement across reviews may reflect shared evidence rather than independent confirmation
Handling Heterogeneous Review Methodologies
Included systematic reviews may use different search strategies, eligibility criteria, quality assessment tools, and synthesis methods. These methodological differences can explain conflicting findings. Explicitly compare the methods of included reviews and use AMSTAR 2 quality ratings to contextualize findings.
When Reviews Reach Different Conclusions
If included systematic reviews reach contradictory conclusions on the same question, your umbrella review should explore why. Common explanations include different eligibility criteria, different search dates, different quality assessment approaches, and different statistical methods for meta-analysis. Presenting these differences transparently is one of the key contributions of an umbrella review.
Frequently Asked Questions
The FAQ section below addresses the most common questions about conducting umbrella reviews.