Mixed Methods Review: Integrate Findings Well

A mixed methods systematic review is a type of evidence synthesis that systematically identifies, appraises, and integrates both quantitative and qualitative research to answer a complex question that neither study type can address independently. Unlike traditional quantitative-only or qualitative-only reviews, a mixed methods approach combines numerical outcome data with experiential, contextual, and process-oriented findings. The JBI Mixed Methods Methodology group defines this synthesis type as one that "considers the complementarity of quantitative evidence of effectiveness and qualitative evidence of experience, feasibility, and meaningfulness." Hong et al. (2018) formalized a widely adopted framework distinguishing convergent, sequential, and multi-stage integration designs. This guide provides a practical, step-by-step approach for researchers conducting their first mixed methods systematic review, from formulating an appropriate research question through to final reporting with PRISMA 2020 compliance.

When a Mixed Methods Approach Is the Right Choice

Not every research question requires a mixed methods systematic review. Choosing this design adds complexity to your protocol, so understanding when it genuinely adds value prevents unnecessary methodological burden.

Complex interventions are the most common trigger for a mixed methods synthesis. When you need to evaluate both whether an intervention works (quantitative effectiveness data) and how or why it works (qualitative process data), a single-method review will only capture half the picture. Sandelowski et al. (2006) demonstrated this clearly in their landmark work integrating quantitative HIV adherence outcomes with qualitative patient experience findings, showing that the combined synthesis revealed barriers and facilitators invisible to either approach alone.

Implementation questions are another strong candidate. If your review asks not just "does this work?" but also "what helps or hinders implementation in real-world settings?", you need both outcome studies and qualitative implementation research. The Cochrane Qualitative and Implementation Methods Group (QIMG) has published guidance on exactly this scenario, recommending mixed methods synthesis when policy decisions require understanding context alongside effectiveness.

Health services research frequently benefits from the mixed methods approach because healthcare delivery involves human behavior, organizational systems, and clinical outcomes simultaneously. A review of a patient education program, for example, needs randomized trial data on clinical outcomes alongside interview-based studies exploring how patients experienced and interpreted the educational content.

Questions that are purely about treatment effect sizes, diagnostic accuracy, or prevalence estimates do not require mixed methods synthesis. If your question can be answered entirely with numerical data and statistical pooling, a standard quantitative Research Gold systematic review or meta-analysis is more efficient and methodologically cleaner.

Integration Designs: Convergent, Sequential, and Multi-Stage

The integration design determines how and when you combine your quantitative and qualitative streams. Hong et al. (2018) described three primary designs, each suited to different research questions and resource constraints.

Convergent Design

In a convergent design, you conduct the quantitative and qualitative synthesis streams simultaneously and integrate the findings at the interpretation stage. Both streams use the same search strategy and eligibility criteria, and you synthesize each stream independently before bringing them together in a final integration step. This is the most common design and works well when your quantitative and qualitative questions are closely related. Pluye et al. (2009) used a convergent approach to examine primary healthcare innovations, synthesizing effectiveness data and qualitative implementation evidence in parallel before mapping the combined findings into a unified framework.

The main advantage of convergent synthesis is efficiency. You run one search, screen once, and extract data from all included studies simultaneously. The challenge is that the integration step requires careful methodological planning. You must decide in advance how you will compare, contrast, and combine findings from two fundamentally different evidence types.

Sequential Design

In a sequential design, one synthesis stream informs the other. The most common sequence is quantitative-first: you conduct a meta-analysis or quantitative synthesis, identify gaps, unexplained heterogeneity, or unexpected findings, and then conduct a qualitative synthesis specifically designed to explore those gaps. Heyvaert et al. (2013) outlined the theoretical underpinnings of this approach, arguing that sequential designs are particularly valuable when quantitative results raise "why" questions that only qualitative evidence can answer.

The reverse sequence (qualitative-first) is less common but equally valid. You might start with a qualitative synthesis to identify relevant constructs, barriers, or facilitators, and then design your quantitative synthesis to test whether those constructs are reflected in outcome data.

Multi-Stage Design

A multi-stage design combines elements of both convergent and sequential approaches across multiple phases. This design is most appropriate for large-scale evidence syntheses addressing broad policy questions where the research landscape includes diverse study types and multiple sub-questions. Multi-stage designs are resource-intensive and typically require a dedicated review team with expertise in both quantitative and qualitative methods.

Quality Appraisal With the Mixed Methods Appraisal Tool

Appraising the quality of studies in a mixed methods systematic review is uniquely challenging because you must evaluate quantitative, qualitative, and mixed methods primary studies, each with different validity criteria.

The Mixed Methods Appraisal Tool (MMAT), developed by Pluye et al. (2009) and updated through multiple iterations, is the most widely used critical appraisal instrument for mixed methods reviews. The MMAT provides a single, integrated framework with five categories of study designs: qualitative research, randomized controlled trials, non-randomized studies, quantitative descriptive studies, and mixed methods studies. Each category includes specific appraisal criteria tailored to that design.

For qualitative studies, the MMAT assesses whether the qualitative approach is appropriate to the research question, whether data collection methods are adequate, whether findings are adequately derived from the data, whether the interpretation of results is sufficiently substantiated, and whether there is coherence between the data sources, collection, analysis, and interpretation.

For quantitative randomized trials, the MMAT evaluates randomization procedure, group comparability at baseline, outcome data completeness, blinding of outcome assessors, and whether participants adhered to the assigned intervention.

For mixed methods primary studies, the MMAT uniquely evaluates the rationale for using a mixed methods design, the effectiveness of integrating the qualitative and quantitative components, and whether the outputs of integration are adequately interpreted.

You can use the JBI critical appraisal tools alongside or instead of the MMAT for individual study designs, particularly when you want more granular assessment criteria for specific study types. Some review teams use JBI checklists for the quantitative and qualitative components and the MMAT specifically for appraising mixed methods primary studies.

A critical decision is whether to exclude studies based on quality scores. The MMAT developers explicitly advise against using overall quality scores to exclude studies, recommending instead that quality assessment results be used to inform the interpretation and weighting of evidence during synthesis.

Data Extraction Strategies for Mixed Studies

Data extraction in a mixed methods systematic review requires a framework that accommodates both numerical outcome data and textual qualitative findings within a unified system.

Design a single extraction form with parallel sections. Your form should include standard fields that apply to all studies (author, year, setting, population, sample size, study design) and then branch into quantitative data fields (effect sizes, confidence intervals, p-values, outcome measures) and qualitative data fields (themes, participant quotes, author interpretations, analytical approach). The extraction template builder can help you create a structured form that captures both data types systematically.

Extract quantitative data according to your planned synthesis method. If you plan to conduct a meta-analysis of the quantitative component, extract all data needed for effect size calculation: means and standard deviations for continuous outcomes, event counts and sample sizes for binary outcomes, and any adjusted estimates reported in multivariate analyses.

Extract qualitative data at the appropriate level of inference. You must decide whether to extract raw participant quotes (first-order constructs), author interpretations and themes (second-order constructs), or both. Sandelowski et al. (2006) recommended extracting at both levels when feasible, as raw quotes provide richer material for re-interpretation while author themes offer pre-synthesized analytical structure.

Code mixed methods primary studies carefully. When a primary study itself uses mixed methods, you need to extract both the quantitative results and the qualitative findings separately, and also note how the original authors integrated their own findings. This integration-level data from primary studies can inform your own integration strategy.

Pilot your extraction form on three to five studies representing different designs before full extraction. This pilot phase invariably reveals gaps in your form, ambiguous fields, or data types you did not anticipate. Revise the form based on pilot findings and re-extract the pilot studies using the final version.

Conducting a mixed methods systematic review and struggling with the complexity of dual data extraction? Research Gold's evidence synthesis service provides experienced methodologists who handle quantitative, qualitative, and mixed methods data extraction with validated frameworks. Our team has supported reviews integrating randomized trial data with interview findings, survey results with focus group themes, and cohort outcomes with ethnographic observations. request your free estimate and get a detailed proposal tailored to your mixed methods review.

Synthesis Approaches: From Thematic to Bayesian

The synthesis method determines how your quantitative and qualitative findings come together to produce integrated conclusions. Several approaches are available, and the best choice depends on your research question, your data, and the integration design you selected.

Thematic Synthesis

Thematic synthesis is the most accessible approach for researchers new to mixed methods reviews. You develop descriptive themes from both your quantitative findings (expressed as narrative statements about effect directions, sizes, and patterns) and your qualitative findings (expressed as interpretive themes). You then organize these themes into a matrix that maps where quantitative and qualitative evidence converge, complement, or contradict each other. This approach works well with convergent designs and does not require advanced statistical software.

Framework Synthesis

Framework synthesis uses a pre-existing theoretical or conceptual framework to organize and integrate findings from both evidence streams. You select a relevant framework (for example, the Consolidated Framework for Implementation Research for implementation questions) and map your quantitative and qualitative findings onto its constructs. This approach produces highly structured results that directly inform theory-driven practice recommendations.

Bayesian Synthesis

Bayesian synthesis represents the most statistically sophisticated approach to mixed methods integration. In a Bayesian framework, qualitative findings can be used to inform prior distributions for quantitative meta-analysis parameters, or quantitative results can be translated into probability statements that are then compared with qualitative evidence. Heyvaert et al. (2013) described how Bayesian methods can formally quantify the degree to which qualitative evidence supports or contradicts quantitative effect estimates. This approach requires statistical expertise in Bayesian methodology and is most appropriate for sequential designs where one evidence stream explicitly informs the analysis of the other.

Matrix Configuration

A joint display matrix or cross-tabulation is a practical integration tool that works across all synthesis approaches. You create a table where rows represent key themes, constructs, or outcomes and columns represent the evidence type (quantitative findings, qualitative findings, and integrated interpretation). This visual format makes convergences and divergences immediately apparent and is highly effective for communicating results to stakeholders and policymakers.

Reporting Your Mixed Methods Systematic Review With PRISMA

Transparent reporting is essential for the credibility and reproducibility of any systematic review, and mixed methods reviews have additional reporting requirements beyond the standard PRISMA 2020 checklist.

Use the PRISMA 2020 statement as your baseline. The 27-item checklist and the build a PRISMA flow diagram provide the structural foundation for reporting your search strategy, screening process, and included studies. Your flow diagram should clearly indicate how many studies were quantitative, qualitative, and mixed methods at each screening stage.

Report your integration design explicitly. State whether you used a convergent, sequential, or multi-stage design and explain the rationale for that choice. Readers and peer reviewers need to understand the architectural logic of your synthesis before they can evaluate the results.

Describe your quality appraisal process for each study type. If you used the MMAT, report which version you used, how many reviewers appraised each study, how disagreements were resolved, and whether quality ratings influenced your synthesis (for example, through sensitivity analysis excluding low-quality studies).

Present quantitative and qualitative results separately before presenting integrated findings. This three-part structure (quantitative results, qualitative results, integrated findings) allows readers to evaluate each evidence stream independently and then assess whether the integration is well-supported by the component syntheses.

Use supplementary materials generously. Mixed methods reviews generate extensive data that cannot fit into a standard journal article. Full extraction tables, complete MMAT ratings, detailed search strategies for each database, and the full joint display matrix should be provided as appendices or supplementary files.

Register your protocol prospectively. PROSPERO accepts mixed methods systematic review protocols, and prospective registration strengthens the methodological credibility of your review. Include your planned integration design, synthesis methods, and quality appraisal approach in the protocol.

Navigating Paradigm Tensions and Common Challenges

Mixed methods systematic reviews sit at the intersection of two research traditions with fundamentally different assumptions about knowledge, evidence, and validity. Navigating these tensions is as important as mastering the technical methods.

Paradigm incompatibility is the most frequently cited challenge. Quantitative research operates within a positivist framework emphasizing objectivity, measurement, and generalizability. Qualitative research often operates within interpretivist or constructivist frameworks emphasizing subjectivity, context, and meaning. Critics argue that combining these traditions within a single synthesis produces incoherent findings. Proponents counter that pragmatism, which prioritizes practical utility over philosophical purity, provides a coherent epistemological foundation for mixed methods synthesis. The JBI Mixed Methods Methodology group explicitly endorses a pragmatist stance.

Quality criteria differences create practical problems during appraisal. Concepts like internal validity, statistical power, and reproducibility apply to quantitative studies but not to qualitative research, which uses criteria like credibility, transferability, dependability, and confirmability. The MMAT addresses this by providing design-specific criteria, but review teams still need members who understand both sets of quality standards.

Unequal evidence volumes are common. You may find 30 randomized trials and only 5 qualitative studies on a given topic, or vice versa. This imbalance can skew your integrated findings toward the larger evidence base unless you deliberately structure your synthesis to give appropriate weight to both streams. Sandelowski et al. (2006) recommended treating the smaller evidence stream as equally important during integration, regardless of volume differences.

Reviewer expertise gaps pose a practical barrier. Few researchers are trained in both quantitative meta-analysis and qualitative synthesis. Building a review team with complementary expertise is essential. At minimum, your team needs one member experienced in systematic review methodology for the quantitative stream and one experienced in qualitative evidence synthesis for the qualitative stream.

Publication bias affects evidence streams differently. Quantitative studies are subject to well-documented publication bias favoring statistically significant results. Qualitative studies face a different form of selective reporting, where certain themes or perspectives may be overrepresented. Your protocol should describe how you plan to assess and address publication bias in each stream.

For the qualitative side of a mixed-methods review, our qualitative data analysis support handles coding, theme development, and integration with quantitative results.

Example Research Questions and Matched Designs

Understanding how to match a research question to the right integration design is one of the most practical skills in mixed methods synthesis. The following examples illustrate this matching process.

"What is the effectiveness of telehealth interventions for managing chronic pain, and how do patients experience these interventions?" This question has two clear components: effectiveness (quantitative) and experience (qualitative). A convergent design is appropriate because both components share the same population and intervention, and the answers are most useful when compared side by side.

"Does cognitive behavioral therapy reduce anxiety in adolescents, and what factors explain variation in treatment response?" This question starts with a quantifiable effectiveness question and follows with an exploratory question. A sequential design (quantitative-first) is ideal. Conduct the meta-analysis first, identify sources of heterogeneity in treatment effects, and then use qualitative evidence to explore those sources.

"How should schools implement mental health screening programs to maximize uptake and effectiveness?" This broad implementation question requires multiple synthesis stages: a qualitative synthesis of implementation barriers and facilitators, a quantitative synthesis of screening accuracy and intervention outcomes, and a final integration stage that combines both into implementation recommendations. A multi-stage design fits best.

"What is the impact of community health worker programs on maternal mortality in low-income countries, and how do socio-cultural factors influence program delivery?" A convergent design with a framework synthesis approach works well here, using a health systems framework to organize both the mortality data and the socio-cultural findings into a coherent set of policy recommendations.

These examples demonstrate the principle that the research question drives the design, not the other way around. Formulating a clear, two-part question before selecting your integration approach prevents methodological confusion later. For guidance on structuring different types of systematic reviews, including mixed methods designs, see our comprehensive overview.