Confirmatory Factor Analysis: How to Test a Measurement Model

Confirmatory factor analysis is a statistical method that tests whether a measurement model you specified in advance fits your data. You state, before looking at the results, exactly which observed items load on which latent factor, and the analysis tells you how well that hypothesized structure reproduces the relationships in your data. It is the standard way to validate that a questionnaire measures the constructs you claim it measures.

The defining feature is the word confirmatory. You are not asking the data to suggest a structure. You are testing a structure you already believe in, usually because theory or a previous study proposed it. This is what separates it from exploratory factor analysis, where the number of factors and the pattern of loadings are discovered rather than imposed.

When to use confirmatory factor analysis

Reach for confirmatory factor analysis when you have a clear hypothesis about your measurement structure and want to test it. The most common situations are validating an established scale in a new population, confirming a factor structure that an earlier exploratory analysis suggested, or establishing the measurement model before estimating relationships among constructs in a larger structural equation model.

If you do not yet know how many factors your items represent, confirmatory analysis is premature. Run an exploratory analysis first, then confirm the resulting structure on a fresh sample. Testing and confirming a structure on the same dataset that suggested it is circular and will not convince a careful reviewer.

What the model specifies

A confirmatory model fixes three things in advance. It fixes the number of factors, the assignment of each item to a factor, and which items are allowed to cross-load or share correlated errors. Every item is typically allowed to load on exactly one factor, and the residual variances, the part of each item the factor does not explain, are estimated as measurement error.

Because you impose this structure, the analysis can ask a sharp question: given that each set of items is supposed to measure one factor, does the pattern of correlations among all the items match what that structure predicts? The answer comes back as a set of factor loadings and a battery of fit statistics.

Reading the output

Two things matter most when you read the results. The first is the standardized factor loadings. Each loading is the strength of the link between an item and its factor, and loadings of roughly 0.50 or higher are generally considered meaningful, with higher being better. An item with a weak loading is not measuring its factor well and may need to be dropped or revised.

The second is model fit, summarized by the same indices used throughout latent variable modeling. A comparative fit index at or above 0.95, a root mean square error of approximation at or below 0.06, and a standardized root mean square residual at or below 0.08 are common targets, drawn from Hu and Bentler (1999). The chi-square test is reported but, because it is sensitive to sample size, is rarely used as the sole criterion. Report several indices together rather than the single most flattering one.

Beyond loadings and fit, researchers report construct reliability and convergent and discriminant validity evidence, showing that items within a factor cohere and that distinct factors are genuinely distinct. These quantities tie directly into the broader question of reliability and validity in measurement.

Sample size and assumptions

Confirmatory factor analysis is data-hungry. Stable estimates require enough cases relative to the number of items and factors, and weak loadings raise the requirement further. Rather than rely on a fixed ratio, estimate the needed sample for your specific model in advance, using the logic in our power analysis and sample size guide and checking feasibility with our sample size calculator.

The default estimator assumes continuous, roughly normally distributed indicators. Likert items with few categories often violate this, and treating them as continuous can bias loadings and fit, so an estimator built for ordinal data is frequently the correct choice. Selecting it is one of the decisions our biostatistics consulting team handles when a measurement model must hold up to review.

Common mistakes

1. Confirming on the same data that suggested the structure. This is circular. Confirm on an independent sample.
2. Adding correlated errors to chase fit. Freeing residual covariances without a theoretical reason produces a model that will not replicate.
3. Reporting only the chi-square or only one favorable index. Report a set of fit statistics and interpret them together.
4. Ignoring weak loadings. An item that barely loads on its factor is contributing noise, not measurement.
5. Treating ordinal Likert items as continuous. Use an estimator appropriate to the response scale.

Identification: the model has to be solvable before it can be tested

A confirmatory model only returns trustworthy estimates if it is identified, meaning the data contain enough information to solve for every free parameter uniquely. Two practical rules govern this. First, each latent factor has no natural scale, so you must set one, either by fixing one of its loadings to 1 (the marker-variable method) or by fixing the factor variance to 1 (the standardized method); the fit is identical, only the metric of the loadings changes. Second, the model's degrees of freedom, the number of non-redundant variances and covariances minus the number of free parameters, must be zero or positive, and a factor measured by only one or two items is usually underidentified unless its loadings are constrained. A related failure is the Heywood case: a negative estimated error variance or a standardized loading above one. It signals a misspecified model, too small a sample, or an empirically weak factor, and it must be diagnosed, not silently constrained away.

Estimating ordinal indicators the right way

The single most common technical error in applied confirmatory factor analysis is fitting Likert items with default maximum likelihood as though they were continuous. With ordered categories the indicators are not normal, and normal-theory maximum likelihood then biases the standardized loadings, deflates the standard errors, and inflates the chi-square. The correct approach treats each item as a coarse view of an underlying continuous trait and estimates the model from polychoric correlations using a diagonally weighted least squares estimator with mean- and variance-adjusted test statistics (the estimator most software labels WLSMV). For continuous but non-normal indicators, robust maximum likelihood with a Satorra-Bentler or Yuan-Bentler correction (often labelled MLR) is the analogue. Reporting which estimator you used, and why, is part of a defensible measurement paper.

Measurement invariance: the test you need before comparing groups

If you intend to compare factor means or associations across groups (men and women, countries, time points), equal scores must mean the same thing in each group, and that is an assumption you test rather than assume. Measurement invariance is established in a nested sequence: configural invariance (the same items load on the same factors in every group), metric or weak invariance (the loadings are equal, which licenses comparing covariances and regression slopes), scalar or strong invariance (the intercepts or thresholds are also equal, which is what licenses comparing latent means), and strict invariance (equal residual variances). You move up the ladder by constraining parameters and checking whether fit degrades meaningfully, with a change in the comparative fit index of about 0.01 a more reliable flag than the chi-square difference. When full scalar invariance fails, partial invariance (freeing one or two non-invariant intercepts while justifying it) can still support a cautious comparison. Skipping this step and comparing raw scale means across groups is a frequent and serious error.

Reliability and discriminant validity have formulas, not just thresholds

The reliability of a factor should not be reported as coefficient alpha, which assumes every item loads equally (tau-equivalence) and understates reliability when they do not. Use composite reliability, also called McDonald's omega, computed from the model's loadings and error variances, which is the quantity the measurement model actually justifies. For convergent validity, the average variance extracted is the mean squared standardized loading on a factor; a value of 0.50 or higher means the factor explains most of the variance in its items. For discriminant validity, the Fornell-Larcker criterion asks that the square root of a factor's average variance extracted exceed its correlation with every other factor, and the more sensitive heterotrait-monotrait ratio of correlations (with values below roughly 0.85 indicating distinct constructs) is now the preferred diagnostic when factors are strongly related.

Global fit can hide local misfit

A model can pass the comparative fit index and root mean square error of approximation while badly misrepresenting one corner of the data. Inspect local fit through the standardized residual covariances (large values flag pairs of items the model fails to reproduce) and the modification indices (which estimate how much the chi-square would drop if a fixed parameter were freed). The discipline is to treat a modification index as a hypothesis, not an instruction: free a correlated error or a cross-loading only when there is a substantive reason, such as near-identical item wording, and report that you did. Chasing fit by freeing parameters until the indices look good produces a model tuned to one sample's noise that will not replicate.

A worked confirmatory model in R

library(lavaan)
library(semTools)

model <- '
  anxiety  =~ a1 + a2 + a3 + a4
  depress  =~ d1 + d2 + d3 + d4
'

# Continuous indicators with a robust estimator; ordinal: add ordered = and WLSMV
fit <- cfa(model, data = d, estimator = 'MLR', std.lv = TRUE)
fit_ord <- cfa(model, data = d, ordered = c('a1','a2','a3','a4','d1','d2','d3','d4'))

summary(fit, fit.measures = TRUE, standardized = TRUE)
fitMeasures(fit, c('cfi','tli','rmsea','srmr'))

# Reliability (omega), and discriminant validity (HTMT)
reliability(fit)           # alpha, omega, average variance extracted
htmt(model, data = d)      # heterotrait-monotrait ratios

# Measurement invariance across a grouping variable
measEq.syntax(model, data = d, group = 'sex', group.equal = 'loadings')   # then 'intercepts'

Bringing it together

Confirmatory factor analysis is the disciplined test of a measurement model: state the structure in advance, fit it to data, and report loadings, fit indices, and validity evidence honestly. It is the foundation that every credible structural equation model and validated questionnaire rests on.

If you are validating a scale, confirming a factor structure, or building the measurement model for a larger analysis, our biostatistics consulting team specifies and fits the model in reproducible code. Request a quote and tell us what you are measuring.