Assess risk of bias in non-randomized studies of interventions using the ROBINS-I framework. Evaluate 7 bias domains, assign judgments per study, and export a publication-ready traffic-light summary table as a high-resolution PNG.
Add your studies and enter their names. Click each colored circle to cycle through judgments: + Low risk, ! Moderate risk, − Serious risk, × Critical risk, ? No information. The Overall column should reflect the most severe domain judgment. When done, export the table as a high-resolution PNG for your manuscript.
| Study | D1 Confounding | D2 Selection of participants | D3 Classification of interventions | D4 Deviations from interventions | D5 Missing data | D6 Measurement of outcomes | D7 Selection of reported result | Overall Overall | |
|---|---|---|---|---|---|---|---|---|---|
ROBINS-I (Sterne JAC et al., 2016) • Generated with Research Gold
Click 'Add Study' to add rows for each non-randomized study in your review. Enter the study identifier (e.g., Author Year) in each row.
Click the colored circles to cycle through judgments for each of the 7 ROBINS-I domains: Low, Moderate, Serious, Critical, or No information.
The Overall column should reflect the most severe domain judgment. ROBINS-I convention is that overall bias cannot be lower than the worst individual domain.
Click 'Export PNG' to download a high-resolution traffic-light summary table ready for your manuscript or supplementary materials.
ROBINS-I evaluates confounding, participant selection, intervention classification, deviations from intended interventions, missing data, outcome measurement, and selective reporting. Each domain has its own signaling questions that guide assessors to a structured judgment.
RoB 2 is for randomized trials (5 domains, 3 judgment levels). ROBINS-I is for non-randomized studies (7 domains, 4 judgment levels plus No information). The confounding domain in ROBINS-I is unique because randomization eliminates this bias source in RCTs.
ROBINS-I applies to cohort studies, case-control studies, controlled before-after designs, interrupted time series with a control group, and any other non-randomized design that compares interventions. It is not intended for single-arm studies or prevalence studies.
Each domain has 3-6 signaling questions answered as Yes, Probably yes, Probably no, No, or No information. The pattern of responses maps to a domain judgment, reducing subjectivity and improving reproducibility across reviewers.
Systematic reviews frequently include evidence from non-randomized study designs — cohort studies, case-control studies, controlled before-after designs, and interrupted time series — particularly when randomized trials are unavailable, unethical, or insufficient in number. The ROBINS-I tool online provides a structured framework for evaluating the risk of bias in these designs, developed by Sterne et al. (2016) under the auspices of Cochrane. Unlike tools designed for RCTs, ROBINS-I explicitly addresses the biases inherent to non-randomized study bias assessment, most notably confounding, which arises when the treatment groups differ systematically in ways that affect the outcome. The tool asks assessors to compare each study against a hypothetical target trial — an idealized randomized experiment that would answer the same question — and to judge how far the actual study departs from that ideal across seven domains. This target trial emulation framework, formalized by Hernán & Robins (2016), provides the conceptual foundation for ROBINS-I by establishing that valid causal inference from observational data requires explicit specification of the protocol that an ideal randomized trial would follow. Building on this approach, the ROBINS-E tool extends the framework to environmental and exposure studies where no deliberate intervention occurs, addressing the growing demand for rigorous bias assessment in environmental epidemiology.
Each of the seven ROBINS-I domains targets a distinct mechanism through which bias can distort study findings. Domain 1 (confounding) evaluates whether the study adequately controlled for known confounders through design or analysis. Reviewers are encouraged to use directed acyclic graphs (DAGs) to visualize the assumed causal structure and identify which confounders must be adjusted for, ensuring that the assessment is grounded in explicit causal reasoning rather than ad hoc variable selection. Common bias-adjustment strategies evaluated in this domain include propensity score methods (matching, weighting, or stratification) and instrumental variable approaches, each of which addresses confounding through different assumptions. Domain 2 (selection of participants) assesses whether the selection process introduced bias related to the intervention received. Domain 3 (classification of interventions) examines whether intervention status was defined unambiguously at the start of follow-up, with particular attention to immortal time bias — a concern common in observational pharmacoepidemiology where the period between cohort entry and treatment initiation is misclassified as exposed person-time. Domains 4 through 7 parallel the RoB 2 domains for randomized trials: deviations from intended interventions, missing data, measurement of outcomes, and selection of the reported result. The ROBINS-I template organizes signaling questions within each domain, and the pattern of answers maps to a judgment of Low, Moderate, Serious, or Critical risk of bias — a four-level scale that provides finer granularity than the three-level RoB 2 scale.
The Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al., 2023) recommends ROBINS-I as the preferred tool whenever a systematic review incorporates non-randomized studies of interventions, and PRISMA 2020 (Page et al., 2021) requires transparent reporting of all risk of bias assessments regardless of study design. Assessors should complete ROBINS-I independently in pairs, documenting their reasoning for each signaling question to facilitate consensus meetings. The overall judgment for each study follows the most severe domain-level rating: a single domain rated Critical renders the entire study Critical, regardless of the other six domains. This conservative rule ensures that serious methodological flaws are never obscured by satisfactory performance elsewhere. When studies rated Serious or Critical constitute a substantial share of the evidence, reviewers should consider using the GRADE certainty framework to downgrade the overall confidence in the pooled estimate.
Selecting the appropriate bias assessment tool depends entirely on the study design under evaluation. For randomized controlled trials, reviewers should use the Cochrane RoB 2 risk of bias tool with its five-domain framework and traffic light visualization. For observational studies where a simpler quality scoring approach is preferred, the Newcastle-Ottawa Scale calculator for cohort and case-control studies assigns stars across selection, comparability, and outcome categories. Qualitative and cross-sectional study designs are best appraised using the JBI critical appraisal checklists, which offer design-specific item sets endorsed by the Joanna Briggs Institute (Aromataris & Munn, 2020). Once all studies have been appraised, the bias assessment results should inform every downstream step — from structuring the data extraction template to interpreting the pooled effect estimates in your meta-analysis.
ROBINS-I (Risk Of Bias In Non-randomised Studies of Interventions) is a Cochrane tool for assessing bias in studies that did not use randomization. It is designed for cohort studies, case-control studies, and other non-randomized designs that compare health effects of two or more interventions. Use ROBINS-I when your systematic review includes non-randomized studies of interventions; for randomized trials, use RoB 2 instead.
ROBINS-I evaluates bias across seven domains: (1) confounding, (2) selection of participants into the study, (3) classification of interventions, (4) deviations from intended interventions, (5) missing data, (6) measurement of outcomes, and (7) selection of the reported result. Each domain addresses a distinct mechanism through which bias can affect the study result, and each is assessed using signaling questions.
RoB 2 is designed for randomized controlled trials and has 5 bias domains with judgments of Low, Some concerns, and High risk. ROBINS-I is for non-randomized studies, has 7 domains, and uses a finer-grained scale: Low, Moderate, Serious, and Critical risk of bias, plus No information. ROBINS-I includes a domain for confounding, which is not applicable to properly randomized trials.
The overall risk of bias judgment should reflect the most severe domain-level judgment. If any domain is rated Critical, the overall is Critical. If the worst domain is Serious, the overall is Serious, and so on. The overall judgment can never be lower than the worst individual domain. When No information is given for a domain, it typically maps to at least Moderate or Serious overall risk, depending on context.
Each ROBINS-I domain includes a set of signaling questions that guide assessors through the evaluation. These questions help identify specific bias mechanisms within each domain. Assessors answer Yes, Probably yes, Probably no, No, or No information for each signaling question. The pattern of answers determines the domain-level judgment. For example, Domain 1 asks whether the study controlled for important confounding variables and whether the methods used were appropriate.
ROBINS-I uses structured signaling questions across 7 bias domains with a defined algorithm, producing judgments of low, moderate, serious, or critical risk. NOS uses a simpler star-based scoring system (0–9 stars) across 3 categories. ROBINS-I is more detailed and rigorous but also more resource-intensive. The Cochrane Handbook recommends ROBINS-I for Cochrane reviews.
The target trial is a hypothetical randomized trial that the observational study is attempting to emulate. ROBINS-I requires reviewers to specify this target trial — including the eligible population, intervention, comparator, and outcome — before assessing bias. Each bias domain is then evaluated relative to what would have occurred in the target trial. This concept comes from Hernán and Robins’ target trial emulation framework.
Critical risk of bias means the study is too problematic to provide any useful evidence for the question at hand. The Cochrane Handbook recommends excluding studies with critical risk from the primary meta-analysis. This judgment typically results from fundamental confounding that cannot be adjusted for, or from severe selection bias that invalidates the comparison entirely.
Assessing randomized trials instead? Use our Cochrane RoB 2 assessment tool with 5 domains and traffic-light visualization. For cohort and case-control study quality, score methodological rigor with the Newcastle-Ottawa Scale scoring tool, which assigns stars across selection, comparability, and outcome categories. When you are ready to rate the overall certainty of your evidence, apply the GRADE certainty of evidence tool to generate summary-of-findings tables across outcomes.
Our team can conduct thorough ROBINS-I assessments with dual independent rating, signaling question documentation, and consensus resolution for your entire systematic review.