The Laboratory Quality Briefing · Briefing II
A method-comparison report shows that the new analyser reads two units higher on average. The results also correlate closely. Before you sign off the change, ask what could happen to a single sample near a clinical decision limit. The average difference leaves that question open.
An important paper revisited · 1986
Bland and Altman’s 1986 Lancet paper asks how far apart two measurements on the same sample or person might be. Plot the difference between the paired results against their average, then look at where the differences sit and how widely they spread. The mean difference estimates the average bias between the methods. The spread tells you how much the difference varies from pair to pair.
High correlation is quite possible when samples cover a wide concentration range, even when the methods differ enough to change a decision. In the paper’s worked peak-flow example, the average difference was about −2 L/min, while the limits of agreement ran from roughly −80 to +76 L/min.
Same average bias, different agreement
Imagine two comparisons of a candidate method with the current method. Each has 60 independent pairs and a mean difference of +2 units, calculated as candidate minus current. The standard deviation of the differences is 3 units in comparison A and 8 in comparison B. Suppose differences of up to 10 units in either direction are acceptable.
Using the mean difference ± 1.96 times the standard deviation of the differences, A’s estimated limits of agreement are −3.9 to +7.9 units. B’s are −13.7 to +17.7 units. Both have the same average bias. Only A’s estimated limits sit within the acceptance band.
A test of the average bias can point you in the wrong direction here. B’s 95% confidence interval for the mean difference is about −0.1 to +4.1 units. It includes zero even though the differences between individual results are much more variable. A finding of “no statistically significant bias” answers a question about the average. For interchangeability, you also need to know how far apart individual results can be.
If you subtracted two units from every result in B, the average difference would move to zero. The individual differences would still have the same spread. That distinction matters when deciding whether to investigate a consistent offset, variation between paired results, or both.
What to check before you sign off
Start with the clinical decision the comparison is meant to support. Find the acceptance limits and their rationale, then inspect the differences near the concentrations that matter for that decision. If the plot fans out as concentration increases, a single pair of limits may hide how agreement changes across the range. Check whether each plotted point comes from single results or averages of replicates. Averaging can reduce variation, so limits based on averages may understate the spread you would see when reporting a single routine result.
For independent pairs with approximately normal differences and a stable spread, the estimated limits describe where roughly 95% of individual differences would be expected to lie. Those limits have their own sampling uncertainty. Check their confidence intervals before treating a limit just inside the acceptance band as evidence of agreement.
Read the paper. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–310. Author-hosted full text · Correction for repeated measurements.
On the website
Start with Does high correlation mean two laboratory methods agree? to see how a paired-result plot and a difference plot can tell different stories. If you’re planning a study, the guide to concentration range shows why it matters where your samples fall around clinical decision limits. The method verification guide puts the comparison alongside the other evidence you’ll need before introducing a method. You can also browse the full verification and method comparison guide collection.
In my Pathology in Practice series on method comparison, the article on difference plots follows on directly from Bland and Altman. Read it alongside the article on acceptability criteria to think through what size of difference would matter in your laboratory. The eight-part series also covers study design, statistical analysis, qualitative comparisons and writing the final report.
Further learning
The free email course What does “good enough” mean? is a next step if you want to think through how to set a performance requirement before collecting comparison data. It covers the role of bias and precision in five short emails.
Stephen MacDonald
Pathology Uncertainty