International Journal for Quality in Health Care 15:1-02 (2003)
© 2003 International Society for Quality in Health Care
Counterpoint |
Making the case for personal professional monitoring in health care
Department of Perioperative Medicine, Anaesthesia and Pain Management, Geelong Hospital, Geelong, Victoria, Australia
The use of databases to identify both good and bad (acceptable and unacceptable) trends in the performance of simple and complex procedures in medicine has a long history and was advocated by such important medical and nursing practitioners as Florence Nightingale, Ernest Codman, Lord Moynihan and others [1–7]. Unfortunately, not all of these committed clinicians have gained the respect of their local medical colleagues [2,8,9]. Thus the need for objectively endorsed systems to monitor the occurrence of adverse clinical outcomes in day-to-day practice is paramount [10–12]. In pursuit of this goal the article by Spiegelhalter et al. in this issue is a welcome contribution [13].
The demonstration by the authors that the adverse events occurring in the practice of both the paediatric cardiac surgeons in Bristol and the general practitioner Harold Shipman could have been detected by routine data monitoring confirms earlier work arising from the Kennedy Inquiry and the Dame Janet Smith Inquiry [14–16]. Unfortunately, in the Bristol case, similar information had already been presented to the surgeons, anaesthetists and managers involved in the care of these patients with little impact on the service [17,18]. In fact a Cusum graph based on the same criteria employed by the Great Ormond Street paediatric cardiac surgery group had been presented to paediatric cardiac anaesthetists and surgeons well before the lethal series of operations was suspended [5]. The problem of professional performance monitoring remains one of objective review coupled with acceptance of performance data. This issue was clearly identified by Professor Ian Kennedy when he concluded in his inquiry: ‘Bristol was awash with data. There was enough information from the late 1980s onwards to cause questions about mortality rates to be raised both in Bristol and elsewhere had the mindset to do so existed’ [19]. Unfortunately the service was not suspended until 1995, by which time as many as 40 children may have died unnecessarily [14–16]. Kennedy goes on to identify the need for objective external criteria by which clinical services can be assessed. ‘The clinicians in Bristol had no one to satisfy but themselves that the service which they provided was of appropriate quality. There was no systematic mechanism for monitoring the clinical performance of healthcare professionals or of hospitals. For the future there must be effective systems within hospitals to ensure that clinical performance is monitored. There must also be a system of independent external surveillance to review patterns of performance over time and to identify good and failing performance’ [19]. The article in this issue demonstrates the applicability of such mechanisms to detect poor performance and the relative ease of their introduction [13].
The issue of professional or specialist group acceptance of this type of performance monitoring is addressed by adjustment of the ‘acceptable’ and ‘unacceptable’ adverse event rates and the allowable type 1 error (
) and allowable type 2 error (ß) rates. Thus it would be reasonable to have a higher ‘acceptable’ adverse event rate for trainees than for specialists [20]. Exactly what these rates are will depend on clinical impressions, data from pilot studies, and early data collections [20]. The adverse event rates are likely to decline with monitoring as techniques, training and equipment improve [21]. However it should also be borne in mind that rates may increase as procedures become more difficult due to changing conditions such as age, obesity and illness severity [22].
Changing the ‘acceptable’ and ‘unacceptable’ adverse event rates has the effect of changing the distance between the horizontal lines on a Cusum plot. Similarly, changing the and ß values—that is, where is the risk of incorrectly assigning the practitioner to an ‘unacceptable’ and ß to an ‘acceptable’ performance level—will also change the vertical distance between the horizontal Cusum lines on the Cusum plot. Decreasing the value will increase the distance between the lines of the Cusum plot. This will make it more difficult for a clinician’s Cusum plot to span a boundary interval and therefore more difficult for that clinician’s performance to be declared acceptable.
In medical models the and ß limits have traditionally been set at 0.1 for ease of presentation of the data. If the and ß levels are the same, the horizontal Cusum lines are superimposed making interpretation of the graphs much simpler [5,20,23].
Why has such a mechanism not been introduced uniformly in clinical practice? Why are patients and their relatives still waiting for the widespread introduction of this type of monitoring process even though advocates of this process have made life-saving claims for its application [21]? One of the answers is clinician resistance, probably best enunciated in the early debate [22]. However, a degree of criticism must also be levelled at those health care administrators who have failed to ensure that such monitoring processes have been implemented. The necessary change in culture has sadly not occurred [7]. However, for the change in culture to be realised, it must be embraced by both clinicians and health care administrators for the benefit of all groups, including patients. One of the possible problems is the inability of clinical and administrative leaders in health care to believe that the type of monitoring proposed by Spiegelhalter et al. is possible and achievable [13]. This psychological block must be addressed if progress is to be made on this important issue in health care. The experience of the Geelong Personal Professional Monitoring Group—a group of anaesthetists at the Geelong Hospital in Australia committed to improving training and monitoring methods for all medical specialties—has been that cultural change in anaesthetic trainee attitudes to data collection and performance monitoring can be achieved provided data collection is made easy, feedback is rapid and informative, the training environment is supportive and the system is robust [6,7,23,24].
The optimistic message that the pilot study by Spiegelhalter et al. must convey to all the leaders of health care groups is that not only can personal professional monitoring be successfully and easily introduced into postgraduate training programmes but that it will also be enthusiastically and continuously employed by trainees in these programmes [6,7,23]. The exciting prospect is that these trainees will become the specialists of the future and will very rapidly help to create a changed culture and different role models for the medical students and trainees of the future [25]. When this has been achieved the spectres of Bristol and Harold Shipman will truly have been laid to rest. The challenge is to achieve widespread implementation of these processes in undergraduate and postgraduate training now.
References
- Monteiro LA. Florence Nightingale on public health nursing. Am J Public Health 1985; 75: 181–186.
[Abstract/Free Full Text] - Kaska SC, Weinstein JN. Historical perspective. Ernest Amory Codman, 1869–1940. A pioneer of evidence-based medicine: the end result idea. Spine 1998; 23: 629–633.[CrossRef][Medline]
- Chambler AF, Emery RJ. Lord Moynihan cuts Codman into audit. Ann R Coll Surg Engl 1997; 79 (4 Suppl): 174–176.[Medline]
- Hannan EL, O’Donnell JF, Kilburn H Jr, et al. Investigation of the relationship between volume and mortality for surgical procedures performed in New York State hospitals. J Am Med Assoc 1989; 262: 503–510.
[Abstract/Free Full Text] - de Leval MR, Francois K, Bull C, Brawn W, Spiegelhalter D. Analysis of a cluster of surgical failures: application to a series of neonatal arterial switch operations. J Thoracic Cardiovasc Surg 1994;107: 914–923.
[Abstract/Free Full Text] - Bolsin S. Routes to quality assurance: risk adjusted outcomes and personal professional monitoring. Int J Qual Health Care 2000; 12: 367–369.
[Free Full Text] - Bolsin SN. League tables and professionalism. Aus Health Rev 2001; 24: 1–4.
- Bolsin SN. Education & Debate: the Bristol cardiac disaster. Br Med J 1998; 317: 1579–1580.
[Free Full Text] - Bolsin SN. Personal Perspective. Professional misconduct: the Bristol Case. Med J Aus 1998; 169: 369–372.
- Horton R. UK medicine: what are we to do? Lancet 1998; 352: 1166.[CrossRef][Medline]
- Horton R. Doctors, the General Medical Council, and Bristol. Lancet 1998; 351: 1525–1526.[CrossRef][Medline]
- Smith R. All changed, changed utterly. British medicine will be transformed by the Bristol case [editorial]. Br. Med J 1998; 316: 1917–1918.
[Free Full Text] - Spiegelhalter D, Grigg O, Kinsman R, Treasure T. Risk-adjusted sequential probability ration tests: application to Bristol, Shipman and adult cardiac surgery. Int J Qual Health Care 2003; 15: 7–13.
[Abstract/Free Full Text] - Spiegelhalter DJ. Mortality and volume of cases in paediatric cardiac surgery: retrospective study based on routinely collected data. Br Med J 2002; 324: 261–262.
[Abstract/Free Full Text] - Dyer O. Shipman murdered more than 200 patients, inquiry finds. Br Med J 2002; 325: 181.
[Free Full Text] - Aylin P, Alves B, Best N, Cook A, Elliot P, Evans SJW et al. Comparison of UK paediatric cardiac surgical performance by analysis of routinely collected data 1984–96: was Bristol an outlier? Lancet 2001; 358: 181–187.[CrossRef][Web of Science][Medline]
- Bolsin S. Mortality and volume of cases in paediatric cardiac surgery. Br Med J 2002; 324: 1095.
[Free Full Text] - Bolsin SN, Patrick A. Comparison of UK paediatric cardiac surgical performance by analysis of routinely collected data 1984–96: was Bristol an outlier? Lancet 2001; 358: 2084.[Web of Science][Medline]
- Kennedy I. The Bristol Royal Infirmary Inquiry. 2001: http://www.bristol-inquiry.org.uk/final_report/report/Summary2.htm. Accessed October 2002.
- Bolsin SN, Colson M. Methodology Matters; CUSUM. Int J Qual Health Care 2000;12: 433–438.
[Abstract/Free Full Text] - Hannan EL, Kilburn H, Racz M, Shields E, Chassin MR. Improving the outcomes of coronary artery bypass surgery in New York State. J Am Med Assoc 1994; 271: 761–766.
[Abstract/Free Full Text] - Green J, Wintfield N. Report cards on surgeons. Assessing New York State’s approach. New Engl J Med 1995; 332: 1229–1232.
[Free Full Text] - Bent P, Creati B, Bolsin SN, Colson M, Patrick A. Professional monitoring and critical incident reporting using personal digital assistants. MJA 2002; 177: 496–499.
- Leung W-C, Diwakar V. Competency based medical training: review Commentary: The baby is thrown out with the bathwater. Br Med J 2002; 325: 693–696.
[Free Full Text] - Paice E, Heard S, Moss F. How important are role models in making good doctors? Br Med J 2002; 325: 707–710.
[Free Full Text]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  R. Wolfe, S. Bolsin, M. Colson, and P. Stow Monitoring the rate of re-exploration for excessive bleeding after cardiac surgery in adults Qual. Saf. Health Care, June 1, 2007; 16(3): 192 - 196. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Freestone, S. N. Bolsin, M. Colson, A. Patrick, and B. Creati Voluntary incident reporting by anaesthetic trainees in an Australian hospital Int. J. Qual. Health Care, December 1, 2006; 18(6): 452 - 457. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S Bolsin, T Faunce, and J Oakley Practical virtue ethics: healthcare whistleblowing and portable digital technology J. Med. Ethics, October 1, 2005; 31(10): 612 - 618. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Bolsin, A. Patrick, B. Creati, M. Colson, and L. Freestone Electronic incident reporting and professional monitoring transforms culture BMJ, July 3, 2004; 329(7456): 51 - 52. [Full Text]
|
|
|
|
|
|
T Faunce, S Bolsin, and W-P Chan Supporting whistleblowers in academic medicine: training and respecting the courage of professional conscience J. Med. Ethics, February 1, 2004; 30(1): 40 - 43. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Colson and S. Bolsin The use of statistical process control methods in monitoring clinical performance Int. J. Qual. Health Care, October 1, 2003; 15(5): 445 - 445. [Full Text] [PDF]
|
|
|
|
|
|
S Bolsin, R Solly, and A Patrick The value of personal professional monitoring performance data and open disclosure policies in anaesthetic practice: a case report Qual. Saf. Health Care, August 1, 2003; 12(4): 295 - 297. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. P. Morton The use of statistical process control methods in monitoring clinical performance Int. J. Qual. Health Care, August 1, 2003; 15(4): 361 - 362. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||