Diagnostic Test Accuracy Meta-Analysis: Bivariate + HSROC

Why diagnostic test accuracy meta-analysis is statistically different

A meta-analysis of intervention trials pools a single effect estimate per study (a risk ratio, an odds ratio, a mean difference). A diagnostic test accuracy review must pool two paired estimates per study at once: sensitivity and specificity. These two indices are negatively correlated within studies because they share an underlying threshold. Moving the cut-off that defines a positive test trades sensitivity for specificity. A standard univariate meta-analysis of either index alone ignores that correlation and produces summary estimates that are biased and over-precise.

The Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy resolves this with two equivalent statistical frameworks: the bivariate random-effects model (Reitsma 2005) and the hierarchical summary ROC model (Rutter and Gatsonis 2001). Both are recommended by the Cochrane Screening and Diagnostic Tests Methods Group. The 2010 chapter "Analysing and Presenting Results" by Macaskill, Gatsonis, Deeks, Harbord, and Takwoingi remains the methodological reference standard.

Worked Example: From Six 2x2 Tables to a Paired Forest Plot

Every bivariate or HSROC model starts from the same input: one 2x2 table per study. Suppose a review includes six studies of the same index test against the same reference standard (hypothetical counts). Sensitivity is true positives divided by everyone with the target condition; specificity is true negatives divided by everyone without it. The 95% confidence intervals are exact (Clopper-Pearson) binomial intervals.

Step by step for Study A

• Sensitivity = 45 / (45 + 5) = 0.90. Specificity = 90 / (90 + 10) = 0.90.
• Positive likelihood ratio = sensitivity / (1 - specificity) = 0.90 / 0.10 = 9.00. Its interval is built on the log scale: the standard error of ln(9.00) is √(1/45 - 1/50 + 1/10 - 1/100) = 0.304, so the 95% CI is exp(2.197 ± 1.96 × 0.304) = 4.96 to 16.32.
• Negative likelihood ratio = (1 - 0.90) / 0.90 = 0.11 (95% CI 0.05 to 0.26).
• Diagnostic odds ratio = (45 × 90) / (10 × 5) = 81.0. The standard error of ln(81.0) is √(1/45 + 1/10 + 1/5 + 1/90) = 0.577, giving a 95% CI of 26.1 to 251.2.
• At a pre-test probability of 20% (pre-test odds 0.25), a positive result gives post-test odds of 0.25 × 9.00 = 2.25, a post-test probability of 69.2%. A negative result lowers it to 2.7%.

What the paired forest plot shows

A paired forest plot puts sensitivity and specificity in two adjacent panels, one row per study. Studies E and F show why both panels matter. Study E has the highest specificity (0.96) and the lowest sensitivity (0.73); Study F has high sensitivity (0.89) and the lowest specificity (0.79). Their diagnostic odds ratios (66.0 and 31.7) hide that mirror image: E is the better test for ruling in (positive likelihood ratio 18.33), while F is better for ruling out (negative likelihood ratio 0.14 against 0.28).

Across the six studies, the Spearman correlation between sensitivity and specificity is -0.12 (p = 0.83). With six studies this descriptive check has almost no power, so threshold variation should be judged from the cut-offs each study reports and from the between-study correlation estimated by the bivariate model, not from this number alone. Pooling the six sensitivities and six specificities in separate univariate analyses would throw away exactly the pairing this plot displays.

The bivariate random-effects model in practice

The bivariate model assumes that the logit-transformed sensitivity and the logit-transformed specificity from each study are drawn from a bivariate normal distribution with a between-study covariance term. The model produces:

• A pooled summary sensitivity and a pooled summary specificity, with confidence intervals that account for the within-study and between-study variance components.
• A summary ROC curve that traces the joint posterior across the threshold space rather than connecting two point estimates.
• A 95% confidence region around the summary point and a 95% prediction region for the next study's expected sensitivity-specificity pair.
• The between-study correlation, which is typically negative when studies used different positivity thresholds.

The Reitsma model is fitted as a generalised linear mixed model with a logit link and study identifiers as random effects. The R package mada (Doebler) implements this via the reitsma() function. The R package metafor implements the bivariate model via rma.glmm() for binary outcomes. Stata users typically rely on metandi, midas, or the metan suite with custom random-effects extensions.

When the HSROC model is preferred

The hierarchical summary ROC model (Rutter and Gatsonis 2001) is mathematically equivalent to the bivariate model when no covariates are included. It expresses test performance with three parameters: an accuracy parameter (analogous to log diagnostic odds ratio), a threshold parameter, and a shape parameter that allows the SROC curve to be asymmetric. The HSROC parameterisation is preferred when:

• The included studies used heterogeneous threshold definitions and you want to model threshold variation explicitly.
• You need to compare two diagnostic tests within the same review using meta-regression of the accuracy parameter.
• The asymmetry of the SROC curve is clinically interpretable, for example when test performance degrades faster at higher specificity than at higher sensitivity.

For most reviews with a fixed test and a small number of threshold variants, the bivariate parameterisation is more interpretable and easier to communicate to clinical collaborators.

Threshold effects and what to do about them

A negative correlation between sensitivity and specificity across studies usually indicates that included studies used different positivity thresholds, either explicitly through different cut-off values or implicitly through different reader judgement criteria. Three pragmatic strategies address threshold variation:

1. Pool only studies that used the same threshold if the literature is large enough to support this restriction. The pooled estimates are then directly clinically interpretable.
2. Fit the HSROC model with the threshold parameter and report the SROC curve as the primary result, with a summary operating point only as a secondary estimate.
3. Report a clinically defensible operating point chosen prospectively in the protocol, for example the threshold maximising the Youden index or minimising a weighted combination of false positives and false negatives.

The Cochrane Handbook discourages the Q statistic* (the point on the SROC curve where sensitivity equals specificity) as a routine summary because it gives the wrong impression of accuracy when the SROC curve is asymmetric or when study points cluster away from the sensitivity-equals-specificity diagonal.

Risk of bias with QUADAS-2

QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies, version 2) is the recommended risk-of-bias tool for primary studies in a diagnostic test accuracy review. It evaluates four domains:

1. Patient selection (consecutive enrolment, inappropriate exclusions).
2. Index test (interpretation blinded to reference standard, threshold pre-specified).
3. Reference standard (correct classification of target condition, blinded interpretation).
4. Flow and timing (interval between index and reference, all participants receiving the same reference standard).

Each domain is rated for both risk of bias and applicability concerns. The PROBAST tool is used for prediction-model studies but is not a substitute for QUADAS-2 in diagnostic test accuracy reviews. Ratings from the risk of bias assessment tool should be reported alongside the paired forest plot of sensitivity and specificity, and a sensitivity analysis should test the effect of excluding high-risk studies.

Reporting under PRISMA-DTA

The PRISMA-DTA extension, published in JAMA in 2018, supplements PRISMA 2020 with 27 items specific to diagnostic accuracy reviews. The most consequential additions are:

• A 2x2 cross-classification table for each included study, recoverable from the published article or supplied by the original authors.
• A summary ROC curve with the bivariate or HSROC summary point, the 95% confidence region, and the 95% prediction region.
• Reporting of summary likelihood ratios and predictive values where the prevalence is meaningful for the clinical context.
• An explicit statement of the target condition, the reference standard, and the population in which the test was evaluated, since transportability to other populations cannot be assumed.

A Fagan nomogram or the equivalent post-test probability calculation is recommended when the review is intended to inform clinical decision-making at a stated pre-test probability.

Software for Diagnostic Test Accuracy meta-analysis

Mature options for fitting bivariate and HSROC models include:

• R mada (Doebler) for reitsma(), SROC curves with confidence and prediction regions, summary likelihood ratios, and forest plots of paired sensitivity-specificity.
• R metafor for rma.glmm() bivariate generalised linear mixed models and meta-regression with study-level covariates.
• R meta4diag for Bayesian bivariate models with informative priors using INLA.
• Stata metandi for bivariate maximum-likelihood estimation with SROC curve output.
• Stata midas by Dwamena for bivariate and HSROC models with publication-bias diagnostics, meta-regression, and Fagan nomograms.
• WinBUGS, JAGS, or Stan for Bayesian HSROC and latent-class bivariate models when the reference standard is imperfect.

Latent-class bivariate models (Chu et al. 2009, Dendukuri et al. 2012) are the appropriate framework when the reference standard itself is known to misclassify some participants. The mada and meta4diag packages document worked examples for these settings.

Common pitfalls in published Diagnostic Test Accuracy reviews

The most frequent statistical errors in published diagnostic test accuracy meta-analyses include:

• Pooling sensitivity and specificity separately with univariate methods, which ignores the within-study correlation and produces artificially narrow confidence intervals.
• Reporting only diagnostic odds ratio without sensitivity, specificity, and likelihood ratios, which strips clinical interpretability.
• Using the Q statistic as the primary summary*, which is discouraged by the Cochrane Handbook.
• Not reporting the 95% prediction region alongside the 95% confidence region on the SROC plot, which conflates the precision of the summary with the heterogeneity across studies.
• Failing to handle threshold variation explicitly, leaving the reader to interpret an SROC curve without knowing whether the spread reflects true performance variation or threshold heterogeneity.
• Using QUADAS-1 instead of QUADAS-2, which the Cochrane Methods Group has not recommended for over a decade.

When to commission a Diagnostic Test Accuracy meta-analysis

A diagnostic test accuracy review is the right design when the clinical question is about the accuracy of an index test against a defined reference standard, when the literature contains at least four primary studies reporting paired sensitivity and specificity, and when the review team has access to bivariate-modelling software. It is the wrong design for prognostic models (use TRIPOD and PROBAST), for predictive biomarkers without a defined cut-off, or for tests evaluated only against another imperfect index without latent-class identifiability. Teams who need a full statistical lead on a diagnostic test accuracy review can commission our PhD-led meta-analysis service, which delivers the bivariate or HSROC analysis, the SROC plot, and the PRISMA-DTA write-up in one project.