MOOSE Checklist: Meta-Analysis of Observational Studies

MOOSE is a 35-item reporting proposal for meta-analyses of observational studies, published by Stroup and colleagues in 2000. Its name stands for Meta-analysis Of Observational Studies in Epidemiology, and it came out of a workshop convened because the reporting guidance then available assumed randomised trials. Its items sit in six domains: the background, the search strategy, the methods, the results, the discussion and the conclusion.

Its standing today is worth stating plainly, because authors are often confused about it. Most journals now expect PRISMA 2020 as the primary reporting guideline for any systematic review, including one pooling observational evidence. MOOSE has not been superseded so much as repositioned: it is used alongside PRISMA to cover the reporting that observational synthesis specifically requires, above all around confounding and exposure measurement. Naming both, and saying what each covered, is the usual approach.

Why observational synthesis needs its own reporting attention

Pooling randomised trials and pooling cohort studies are not the same activity, and the difference is not a matter of degree. In a trial, randomisation is supposed to balance confounders, so the main threats a synthesis has to describe are how the trials were conducted and reported. In observational research, confounding is present by construction, and each study has dealt with it differently, using a different set of covariates, measured in different ways, with different residual error.

This produces a problem no statistical method fixes. A meta-analysis narrows a confidence interval by combining information; it does not identify or remove bias shared across the studies. If every included cohort failed to adjust for smoking, pooling them yields a tighter estimate of a confounded association. Greater precision around the wrong number is not progress, and it can look more convincing than any single study did.

MOOSE's practical contribution is to force that into the open, by asking explicitly about the assessment of confounding, the quality of exposure and outcome measurement, and the rationale for pooling at all.

The six domains in outline

Described in our own words, the domains ask for the following. The authoritative item wording sits with the original publication and is indexed on EQUATOR's index of reporting guidelines.

Reporting of background. The problem definition, the hypothesis being tested, the outcome and exposure of interest, the study designs eligible for inclusion, and the populations studied.

Reporting of search strategy. The qualifications and roles of the searchers, the databases and registries used, the search software and version with any special features, the use of hand searching, the inclusion and exclusion criteria with a rationale, the handling of non-English language publications, the treatment of unpublished studies and abstracts, and a description of any contact with authors.

Reporting of methods. How the included studies were checked for agreement with the hypothesis, how confounding was assessed, how study quality was assessed and whether assessors were blinded, how heterogeneity was assessed, a description of the statistical methods in enough detail for replication, and the provision of appropriate tables and graphics.

Reporting of results. A graph summarising individual study estimates and the pooled estimate, a table giving descriptive information for each included study, the results of any sensitivity analysis, and an indication of the statistical uncertainty of the findings.

Reporting of discussion. Quantitative assessment of bias, including publication bias; justification for the exclusion of any studies; and assessment of the quality of the included studies.

Reporting of conclusion. Consideration of alternative explanations for the observed results, generalisation of the conclusions given the data and the limits of the review, guidelines for future research, and disclosure of funding.

The confounding item does most of the work

If you take one thing from MOOSE into your methods, make it the confounding item, because it determines whether the synthesis is interpretable. Handling it properly means extracting, for every included study, which confounders were adjusted for and how they were measured, then making a decision about what to do with the variation.

There are three defensible routes and one common mistake. You can pool only comparable adjustment sets, restricting the primary analysis to studies that controlled for a minimum set of variables and treating the rest as a sensitivity analysis. You can pool the maximally adjusted estimate from each study and state clearly that adjustment sets varied, presenting the variation in the study characteristics table. Or you can stratify by adjustment, which turns the question into an empirical one and is often the most informative option.

The mistake is mixing crude and adjusted estimates in one pooled figure without comment. That produces a summary statistic with no consistent meaning, and it is the single most common fatal flaw in observational meta-analyses submitted for review.

Heterogeneity is a finding, not an obstacle

Observational syntheses typically show high heterogeneity, and the reflex response, fitting a random effects model and reporting a wide interval, treats a result as a nuisance. Our comparison of fixed-effect and random-effects models covers why the choice of model is not itself an explanation.

The informative move is to investigate. Subgroup analysis by study design, geography, exposure assessment method or adjustment set will often show that apparently inconsistent results are consistent within categories. Meta-regression does the same with continuous study-level characteristics, given enough studies to support it. Either way, prespecify the investigations in the protocol, because subgroup analyses invented after seeing the forest plot are exploratory whatever the paper says. PRISMA-P is where that prespecification belongs.

Quality assessment feeds the same investigation. For cohort and case-control studies the usual instruments are the Newcastle-Ottawa Scale and, for non-randomised studies of interventions, ROBINS-I. Note that neither is a reporting checklist, and that a study reported in full compliance with the STROBE reporting items can still score poorly on both.

Publication bias, with a caveat

MOOSE asks for a quantitative assessment of bias including publication bias, and the standard tools are a funnel plot with a formal test such as Egger's regression. Our funnel plot generator produces both.

The caveat matters in observational work. Funnel plot asymmetry has several possible causes, and publication bias is only one: genuine small study effects, differences in study quality correlated with size, and chance all produce the same picture. With fewer than about ten studies these tests have very little power, and reading asymmetry into a sparse plot is over-interpretation. Report the plot, report the test, and describe asymmetry as consistent with several explanations rather than as evidence of suppression.

Using MOOSE and PRISMA together

The practical arrangement most journals accept is this. Complete the PRISMA 2020 statement as the primary checklist, including the flow diagram, which our PRISMA flow diagram generator will produce from your screening counts. Then use MOOSE as a supplementary check for the observational-specific items, above all confounding assessment, exposure measurement quality and the justification for pooling. Name both in the methods and submit both completed checklists.

Certainty of the resulting body of evidence is a further, separate judgement. Observational evidence starts lower in the GRADE framework and can be upgraded on a large effect or a dose-response gradient, which is worth knowing before you write a conclusion the certainty rating will not support.