Control charts have been used in manufacturing industry for many years as a way of monitoring quality. The basic idea is that intrinsic to any artificial process with a measurable outcome there will be variation in the quality of the product. Some of this variation is simply down to 'noise' in the system and does not necessarily reflect on how well the process is performing. Some larger variations may be due to external pressures on the process (either improving or worsening the outcome beyond the range of 'noise'.) A control chart is a graphical presentation of the outcomes of the process in a way that enables one to distinguish between random variation and variation that might suggest that the process is not performing well due to some extraneous factor, which one will want to identify and correct (or performing better than the usual process - which is something one may want to capture and spread as best practice).

In the context of audit we would like to be able to identify practices at both extremes to facilitate the process of quality improvement.

There are many types of control chart but the one above is typical of PCAG feedback. (By 'aspirin addressed' I mean either patients is on aspirin or has a recorded contraindication.) The mean performance for all practices is plotted as the blue line, with an upper and lower control limit in red (UCL, LCL), calculated as 3 standard deviations from this mean using a Binomial probability distribution (see below). The individual practice performance is then plotted. (The scales are not given because a mathematical transformation is needed to produce straight lines of the UCL and LCL to render the graph easier to read, making the scales relatively meaningless as a figures and the key to interpretation is the pattern of points rather than the figures anyway)

In this example, all the practices bar one are located within the UCL and LCL boundaries indicating that it is likely that the variation between them represents noise within the system rather than genuine performance differences. One practice falls just below the LCL, which suggests that another factor is impairing the performance and the practice should be able to improve its performance by actively identifying and correcting the reason(s) why. If the mean performance overall needs improving, this suggests that the PCT will need to look at the whole process of addressing aspirin in CHD in its practices, rather than merely exhorting practices to do better within the current process.

Traditionally practice performance has been compared using league tables. These suffer from several disadvantages.

1) They ignore the fact that variation between practices is partly a result of random fluctuation and that ranking them in this context is meaningless.

2) They do not take into account the fact that for smaller samples (say in smaller practices) a random fluctuation of an absolutely small number of patients meeting the audit criterion will have a disproportionately larger effect on percentage reporting than in a larger practice. Ie they ignore the effects of sample size

3) They do not provide a way of telling whether a variation from the mean is likely to be a random variation or due to a special cause and so do not allow one to be confident in targeting variations for action.

Control charts go some way to addressing all these issues. This is illustrated by the usual graphical league table presentation of the above data.

One might be tempted to say practices 27 and 50 both need intervention on the basis of the above graph. However, the smaller number of patients in practice 27 means that it is not significantly diverging from the mean (as judged by the control chart) and so its performance may just reflect random variation. However, if on repeated audits it fell close to the LCL, control chart theory would suggest that there is a special cause for this.

One might conclude (wrongly!) from the above that practice 56 has something to teach all the others but the control chart indicates that its performance is not significantly different. This is because practice 56 data is based on only 10 patients!

Control charts therefore add value to the interpretation of audit data in a fairly straightforward graphical manner.

NB The data here is illustrative only and does not reflect real practice performance.

Caveat: The mean performance line is currently calculated as the mean of the data for the practices in the data sample, rather than by reference to means achieved by practices where the process of care is known to be functioning well. So if the process of care is poor in all practices this will not show in the control chart. So in interpreting the charts if a few practices are well above the UCL, it is likely that there is a problem with the process in the rest, which usually means a data-recording problem.

Bristol, Shipman, and clinical governance: Shewhart’s forgotten Lessons

Mohammed A Mohammed, K K Cheng, Andrew Rouse, Tom Marshall

THE LANCET • Vol 357 • February 10, 2001

Understanding Statistical Process Control. Donald Wheeler. SPC Press. ISBN 0-945320-13-2

Improving Healthcare with Control Charts. Raymond Carey. ASQ Quality Press. ISBN 0-87389-562-2

1949g F. Mosteller and J. W. Tukey. The uses and usefulness of binomial probability paper, J. Amer. Statist. Assoc. 44,174-212

Deming, W. E. 1986. Out of the Crisis. Cambridge: Massachusetts Institute of Technology Center for Advanced Engineering Study

Treat the process of care as a process with a certain probability of delivering the audit criterion's standard. For N patients each has a probability p of achieving the standard.

Let y be the number achieving the standard and x be the number not, where x+y=N

So y=N.p and x=N.(1-p) and y_mean=x_mean.(p/(1-p))

By binomial probability theory three standard deviations for the control limits =

So the UCL is at y_ucl = N.p +

And x_ucl = N.(1-p) -

And the LCL is at y_lcl = N.p -

And x_lcl = N.(1-p) +

If after Tukey we plot y vs x as our control chart, but instead of plotting on probability paper (which uses a square root transformation) we transform the plot by multiplying by

1 /we get

y_ucl / = y_mean / +

x_ucl /= x_mean / -

This has the effect of making the mean line and UCL and LCL run in parallel and so we also transform the data by 1 / and plot y / vs x /.

If we calculate the value of y_ucl /at x_mean / , the distance y_ucl /- y_mean / is constant and will facilitate plotting the UCL and LCL.

y_ucl /- y_mean / = + (p/(1-p)).

=

ie the UCL and LCL are lines parallel to the mean line and vertical distance from it.