Risk of Bias Assessment Tool

Every systematic review must evaluate the internal validity of its included studies, and the risk of bias tool developed by Cochrane is the gold standard for randomized controlled trials. The revised instrument, known as RoB 2 (Sterne et al., 2019), replaced the original Cochrane risk of bias tool to provide a more structured, domain-based framework that reduces assessor subjectivity. Researchers conducting an RoB 2 assessment online evaluate each trial across five distinct domains: bias arising from the randomization process, bias due to deviations from intended interventions, bias due to missing outcome data, bias in measurement of the outcome, and bias in selection of the reported result. Each domain uses a series of signaling questions whose answers map algorithmically to a judgment of Low risk, Some concerns, or High risk of bias. These RoB 2 signaling questions form a structured decision pathway that guides assessors through each domain systematically, reducing the subjectivity inherent in earlier versions of the tool. The framework also requires reviewers to specify whether they are assessing the effect of assignment to intervention (intention-to-treat) or the effect of adhering to intervention (per-protocol), as different signaling questions apply to each effect of interest.

The Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al., 2023) mandates risk of bias assessment as a core component of every Cochrane review, and PRISMA 2020 (Page et al., 2021) requires authors to present results of individual study bias assessments in both the text and supplementary materials. A Cochrane risk of bias generator streamlines this process by allowing reviewers to input domain-level judgments and produce publication-ready traffic light plots, domain summary rows, and robvis-style summary bar charts. These visual outputs have become the standard way to communicate bias findings. The traffic light plot displays each study as a row with colored indicators for each domain, enabling readers to quickly identify patterns of methodological concern across the evidence base. The summary bar chart complements the traffic light table by showing the percentage distribution of Low risk, Some concerns, and High risk judgments as stacked horizontal bars for each of the five domains plus the Overall judgment, giving reviewers and readers a single-glance overview of how bias is distributed across the entire evidence base. This tool generates both visualizations as downloadable high-resolution PNGs suitable for manuscript figures or supplementary materials. The robvis R package (McGuinness & Higgins, 2021) popularized this paired visualization approach, and our free online tool brings the same capability to researchers without requiring R or any coding knowledge. All assessment data is automatically saved to your browser via autosave, so you can close the tab and return later without losing any work.

Proper risk of bias assessment requires at least two independent reviewers. Each assessor completes the signaling questions for every domain independently, and discrepancies are resolved through discussion or arbitration by a third reviewer. This dual-independent approach, recommended by the Cochrane Handbook, improves the reliability and reproducibility of judgments. The overall risk of bias for each study follows a strict hierarchy: if any single domain is judged High risk, the overall assessment is High risk regardless of other domain judgments. This conservative rule ensures that a single critical methodological flaw is never masked by adequate performance in other domains. Adaptations of RoB 2 have been developed for specific trial designs, including crossover trials and cluster-randomized trials, which introduce additional signaling questions to address design-specific bias mechanisms such as carry-over effects and identification biases at the cluster level. For living systematic reviews that are continually updated as new evidence emerges, repeated RoB assessment must be integrated into each update cycle to maintain the currency and credibility of the review. Once all assessments are complete, reviewers should integrate the results into their evidence synthesis. For instance, they may conduct sensitivity analyses that restrict the meta-analysis to studies at low risk of bias, or by using the GRADE framework to downgrade certainty of evidence when a substantial proportion of the evidence carries high risk of bias.

It is important to select the correct bias assessment instrument for each study design. RoB 2 applies exclusively to randomized trials. For non-randomized studies of interventions such as cohort or case-control designs, Cochrane recommends the ROBINS-I tool for non-randomized study bias assessment, which evaluates seven domains including confounding, a bias source that randomization eliminates in RCTs. Observational studies may also be scored using the Newcastle-Ottawa Scale for cohort study quality assessment, which assigns stars across selection, comparability, and outcome categories. For reviews that include qualitative or cross-sectional evidence, the JBI critical appraisal checklists provide design-specific quality assessment frameworks endorsed by the Joanna Briggs Institute. Regardless of which tool you use, transparent reporting of the assessment process, including who assessed, how disagreements were resolved, and what proportion of studies fell into each risk category, strengthens the credibility and reproducibility of your systematic review.