In 1995, Bland and Altman published "Comparing Methods of Measurement: Why Plotting Differences Against Standard Method Is Misleading" (Lancet 1995: 346:1085-1087) as a followup to their original paper on comparing two measuring devices.

When two methods of measurement are compared, it is sometimes common to see the differences between the measurements plotted against the measure that is considered to be the standard. This is often the result of a mistaken notion that standard is the same thing as truth. However, if the standard is subject to measurement error as most standards are, the differences will be correlated with the standard, no matter how golden the standard might be.

The paper is correct. However, the mathematical demonstration is presented in a way that masks much of what's going on. This note presents the same material in a different way.

Let each individual be characterized by a true underlying value U_i. Let the Us be distributed with mean and variance ²_U, that is

U~D(,²_U)

Suppose S and T are both unbiased estimates of U, that is,

S = U + , with ~D(0,²)
T = U + , with ~D(0,²)

This says S and T are both unbiased methods of measuring U with their own respective measurment errors, ² and ². Further, assume that all of the errors are uncorrelated, that is,

cov(U_i,U_j) = cov(_i,_j) = cov(_i,_j) = 0 , for all ij, and
cov(U_i,_j) = cov(_i,_j) = cov(_i,_j) = 0 , for all i,j.

• when the two measuring techniques have equal measurement error, their difference is uncorrelated with their mean;
• if one of the measurement techniques has no measurement error, the differences will be uncorrelated with it but will be correlated with the means;
• when the measurement error variances differ between the instruments, the differences in measurement will again be correlated with the mean. However, the correlation will be small unless the difference between the variances (|² - ²|) is large relative to the variability between individuals (²_U). (If the field of application is such that there is substantial likelihood that the difference between the variances of the measurement errors (|² - ²|) will be large relative to the variability between individuals (²_U), the "will be small" of the previous sentence might be changed to "will not be small unless"!)