The source of these data is lost to memory. It may have started out as a textbook exercise. They are not from an actual experiment. The next time around I'll change the labels to make the exercise more realistic, but for now it's easier to go with the data as they are currently constituted,

In this experiment four nurses use each of three methods for measuring blood pressure. Nurses and methods are crossed factors with a total of 12 combinations. Thirty-six subjects participate. Each subject is measured by only one combination of nurse and method, that is, three different subjects are measured for each combination of nurse and method. The research question is whether systolic blood pressure depends on the method use to measure it.

The analysis of variance table for the experiment is

Factors can either be fixed or random. A factor is fixed when the levels under study are the only levels of interest. A factor is random when the levels under study are a random sample from a larger population and the goal of the study is to make a statement regarding the larger population.

In this example, METHOD is a fixed factor. The purpose of this study is to examine these three methods of measuring blood pressure. There may be other methods, but they do not concern us here. When we are done, the hope is to make a statement comparing these three methods.

From the description of the study, the status of NURSE is less clear. If the investigator cares only about these four nurses, NURSE is a fixed factor. This might be the case where the study concerns the staff of a particular research unit and there is no goal of generalizing beyond the unit. Since only these four nurses matter, NURSE is a fixed factor. However, it might be that the point of the study is to generalize the results to all nurses. In that case, these four nurses might be viewed as a random sample of the population of all nurses, making NURSE a random factor.

One way to decide whether a factor is fixed or random is to ask what would happen if the study were repeated. If the same set of nurses would be used (as in the case of studying a particular research unit) the factor is fixed. If any set of nurses would do equally well, the factor is random.

There is an extra source of variability when factors random, so it should come as no surprise to learn that the analysis of METHOD changes according to whether NURSE is fixed or random. If NURSE is fixed, the analysis proceeds as usual. An F-ratio is constructed with the METHOD Mean Square in the numerator and the Residual Mean Square in the Denominator. The F-ratio is 4.97 [=339.6/68.3]. When it is compared to the percentiles of the F distribution with 2 numerator degrees of freedom and 24 denominator degrees of freedom, the resulting P value is 0.0156. Most statistical program packages produce this analysis by default. If NURSE is random, the F-ratio is still constructed with the METHOD Mean Square in the numerator, but the denominator is now the mean square for the METHOD*NURSE interaction. This F-ratio is 3.33 [=339.6/102.1]. When it is compared to the percentiles of the F distribution with 2 numerator degrees of freedom and 6 denominator degrees of freedom, the resulting P value is 0.1066.

When factors are fixed, the measure of underlying variability is the within cell standard deviation. Differences between methods are compared to the within cell standard deviation. When NURSES is random, methods are evaluated by seeing how much they differ on average relative to the way they differ from nurse to nurse. If two methods differ exactly the same way for all nurses, then that's the way they differ. However, if the differences between methods vary from nurse to nurse, many nurses must be examined to determine how the methods differ on average.

One critical consequence of NURSE being random is that the test for a METHOD effect depends on the number of nurses rather than the number of subjects measured by each nurse. This makes sense at the conceptual level because the determination of a METHOD effect is accomplished by seeing how methods differ from nurse to nurse. Therefore, the more nurses the better. Without going into too much detail, the measure of variability to which methods are compared when nurses are random behaves something like

where is an expression depending on the variability in individual subjects measured under the same conditions, n is the number of subjects per cell, _r is an expression depending on the variabilty between nurses, and r is the number of nurses. There is some advantage to be had by increasing n, but clearly the big gains are to be had by increasing r.

A faulty analysis?

If it is known that there is no interaction between method and nurse, a simpler model can be fitted.

Y_ijk = + _i + _j + _ij

In that case, the error term for testing the METHOD effect will be the Residual term from the simple model. Because this was a balanced experiment, the METHOD mean square will the same for both models while the Residual mean square for the simpler model will be obtained by adding the Residual and interaction sums of squares and degrees of freedom from the full model. That is, the Residual mean square will be 75.0 [=(612.4+163.9)/(6+24)]. The F-ratio is 4.53 with a corresponding P value of 0.0192.

While this is similar to pooling, I view it as different. With pooling, the error term remains the same while nonsignificant effects are added to it primarily to increase the error degrees of freedom. With the kind of model simplification described here, the error term changes. My own take on this kind of model simplification is that it usually represents an attempt to salvage a study that was not designed properly in the first place.

The issue of fixed and random factors is currently making itself felt in an area called group randomized trials. An example of a group randomized study is a comparison of teaching methods in which randomization is achieved by randomizing classes to methods. When CLASS is treated as a random factor, the unit of observation is effectively the class, not the student. The precision of estimates is governed by the expression above with CLASS in place of NURSE. It is not uncommon to see group randomized trials improperly analyzed by treating the grouping variable as fixed rather than random.

Multi-Center Trials

Sometimes it is known from the outset that sufficient numbers of subjects cannot be recruited from a single location. It is common for such studies to be carried out as multi-center trials where subjects are enrolled from many centers. Each center has its own randomization list to insure that each center has subjects on each treatment.

An important question is whether the factor CENTER should be treated as fixed or random. If it is treated as fixed (or, equivalently except for a few degrees of freedom, there is assumed to be no center-by- treatment interation), the sample size is effectively the number of subjects. If CENTER is treated as random, the sample size is effectively the number of centers, and the study is much less powerful.

Standard practice is to treat CENTER as fixed. The rationale is that the same protocol under the control of a single set of investigators is used at all centers. However, if a statistically significant center-by-method interaction is encountered, it must be explained fully.