PRISMA 2020 Item 11 requires you to describe the methods used to assess the risk of bias in each included study, including the tool used, the domains assessed, and the number of reviewers who independently applied the tool. The Cochrane Handbook (Higgins et al., 2023) recommends RoB 2 for randomized trials and ROBINS-I for non-randomized studies.
What this subsection must contain:
- The specific risk of bias tool used, with a citation
- The domains assessed by that tool (for example, RoB 2 assesses randomization, deviations from intended interventions, missing outcome data, measurement of outcome, and selection of reported result)
- The number of reviewers who independently assessed each study
- How disagreements in risk of bias judgments were resolved
- Whether risk of bias assessments were used in sensitivity analyses or to judge the certainty of evidence
Example paragraph:
Risk of bias in randomized controlled trials was assessed using the Cochrane Risk of Bias 2 (RoB 2) tool (Sterne et al., 2019). Each study was evaluated across five domains: bias arising from the randomization process, bias due to deviations from intended interventions, bias due to missing outcome data, bias in measurement of the outcome, and bias in selection of the reported result. Two reviewers independently assessed each study, and disagreements were resolved by discussion with a third reviewer. Overall risk of bias for each study was classified as low, some concerns, or high. Studies rated as high risk of bias were included in the primary analysis but examined separately in a sensitivity analysis.
Common reviewer criticisms:
- "You used the Newcastle-Ottawa Scale for randomized trials. NOS is designed for observational studies." Match the tool to the study design.
- "Risk of bias was assessed by only one reviewer." Independent dual assessment is required.
- "You did not describe how risk of bias judgments informed your synthesis." Explain whether high-risk studies were excluded from sensitivity analyses or downgraded in certainty assessment.
PRISMA 2020 Item 12 requires you to specify the effect measures used for each outcome (for example, risk ratio, odds ratio, mean difference, standardized mean difference, hazard ratio) and how they were selected.
What this subsection must contain:
- The primary effect measure for each outcome type (dichotomous, continuous, time-to-event)
- The rationale for choosing that measure
- Whether 95 percent confidence intervals were reported
- Any transformations applied (such as converting odds ratios to risk ratios, or using Hedges' g instead of Cohen's d for small samples)
Example paragraph:
For continuous outcomes, we calculated the mean difference when studies used the same measurement scale and the standardized mean difference (Hedges' g) when different scales were used. For dichotomous outcomes, we calculated risk ratios with 95 percent confidence intervals. Risk ratios were preferred over odds ratios because they are more interpretable when event rates are not rare. For time-to-event outcomes, we extracted or calculated hazard ratios with 95 percent confidence intervals.
Common reviewer criticisms:
- "Why did you use odds ratios when event rates exceed 20 percent? Odds ratios overestimate the effect in common outcomes." Justify the choice of effect measure relative to event frequency.
- "You did not specify whether you used Cohen's d or Hedges' g." Small-sample corrections matter; name the exact estimator.
- "Confidence intervals are missing for some outcomes." Always report both point estimates and intervals.
Struggling to draft a methods section that will satisfy peer reviewers? Research Gold's methodologists write and review methods sections for systematic reviews and meta-analyses across all clinical and social science disciplines. If you need expert feedback on your eligibility criteria, search strategy, or synthesis approach, get your project priced and tell us which sections you need help with. We also provide full Research Gold systematic review services from protocol to publication.
PRISMA 2020 Item 13 is the most detailed methods item. It requires you to describe every step of your data synthesis, whether narrative or quantitative. If you conducted a meta-analysis, this section must cover the statistical model, the method for estimating heterogeneity, the software used, and any additional analyses (subgroup, sensitivity, meta-regression).
What this subsection must contain for a meta-analysis:
- Whether a fixed-effect or random-effects model was used, and why
- The heterogeneity estimator (DerSimonian-Laird, restricted maximum likelihood, Paule-Mandel, or another method)
- The heterogeneity statistics reported (Cochran's Q, I-squared, tau-squared, prediction interval)
- The statistical software and version (such as R version 4.3.2 with the metafor package, or Review Manager 5.4, or Stata 18)
- Subgroup analyses planned a priori, including the variables tested and the minimum number of studies required per subgroup
- Sensitivity analyses conducted (leave-one-out, exclusion of high risk of bias studies, fixed-effect vs. random-effects comparison)
- Whether publication bias was assessed (see Item 15)
What this subsection must contain for a narrative synthesis:
- The framework used for narrative synthesis (such as Synthesis Without Meta-analysis, or SWiM, reporting guidelines)
- How studies were grouped for narrative comparison
- How effect direction and magnitude were described across studies
- Whether vote counting or harvest plots were used
Example paragraph (meta-analysis):
We conducted random-effects meta-analyses using the restricted maximum likelihood (REML) estimator to pool effect sizes. Statistical heterogeneity was assessed using the Cochran Q test (significance threshold p < 0.10) and the I-squared statistic. I-squared values of 25, 50, and 75 percent were interpreted as low, moderate, and high heterogeneity, respectively. Prediction intervals were calculated to estimate the range of true effects across settings. Subgroup analyses were planned a priori for intervention type (aerobic vs. resistance vs. combined exercise), intervention duration (less than 24 weeks vs. 24 weeks or more), and supervision status (supervised vs. unsupervised). Meta-regression was performed when 10 or more studies were available per covariate. Leave-one-out sensitivity analyses were conducted to assess the influence of individual studies. All analyses were performed in R version 4.3.2 using the metafor package (Viechtbauer, 2010).
Common reviewer criticisms:
- "You used DerSimonian-Laird but have fewer than 20 studies. REML or Paule-Mandel is more appropriate for small numbers of studies." The choice of heterogeneity estimator matters when the number of studies is small.
- "You reported I-squared but not the prediction interval. I-squared describes the proportion of variability, not the range of expected effects." Prediction intervals provide clinical context that I-squared alone cannot.
- "Your subgroup analyses were not pre-specified in the protocol." Post hoc subgroup analyses should be clearly labeled as exploratory.
- "You did not name the statistical software." Always report the software, version, and specific packages used.
PRISMA 2020 Item 15 requires you to describe the methods used to assess the risk of bias due to missing results in the synthesis (commonly called publication bias or reporting bias). This is distinct from the risk of bias within individual studies (Item 11).
What this subsection must contain:
- The methods used to assess publication bias (funnel plots, Egger's regression test, Begg's rank correlation test, or trim-and-fill analysis)
- The minimum number of studies required before these tests were applied (typically 10 studies for funnel plot-based methods)
- Whether selective reporting within studies was assessed (comparing published results with protocols or trial registrations)
Example paragraph:
For meta-analyses that included 10 or more studies, we assessed publication bias visually using funnel plots and statistically using Egger's regression test (Egger et al., 1997). Asymmetry was considered present when Egger's test yielded p < 0.10. For meta-analyses with fewer than 10 studies, we noted that statistical tests for publication bias have low power and results should be interpreted with caution. Selective outcome reporting was assessed by comparing published results with registered protocols on ClinicalTrials.gov or PROSPERO where available.
Common reviewer criticisms:
- "You have only 6 studies in this meta-analysis. Funnel plots and Egger's test are unreliable with fewer than 10 studies." Acknowledge this limitation explicitly.
- "You did not assess selective reporting bias." Comparing published outcomes with protocol registrations is now expected under PRISMA 2020.
- "Your funnel plot shows asymmetry but you did not discuss it." Funnel plot asymmetry requires interpretation: it may indicate publication bias, small-study effects, or genuine heterogeneity.
