An examination of new methods of outcome determination
Mr Graham Copeland MB BS FRCS ChM Consultant General Surgeon
Mrs Christina Richardson Dip HE SHND Critical Care Nurse
Death following surgical intervention is widely used as an outcome measure and has been used for nearly four thousand years in various forms. The current paper examines some of the problems with using raw mortality rates and methodologies using low numbers of risk adjusted variables and some of the potential solutions using more refined mathematical models of outcome determination by the utilisation of fully automated monitoring systems. Fortunately, death is an infrequent consequence of surgical intervention and complications are much more common. The current paper also questions whether the time has come to move away from just using mortality rates and considers the use of risk adjusted complication rates and measures of potential patient harm as an additional or alternative form of outcome determination.
Over the past twenty years there has been increasing interest in the audit and investigation of the care of patients who die following surgery. This is not a new phenomenon. As early as 1750BC King Hammurabi of Babylon issued many edicts directed at surgeons. The most widely known are that ‘If a doctor inflicts a serious wound with his operation knife on a free man’s slave and kills him, the doctor must replace the slave with another. If a doctor has treated a free man but caused a serious injury from which the man dies, or if he opened an abscess and the man goes blind, the man is to cut off his hands’. While current day thinking may be somewhat different from that of four thousand years ago there is still a punitive approach to some aspects of surgical and medical care, and poor performance can still result in suspension and termination of a surgeon’s registration. Even more punitive approaches are often taken in nursing. How can this be examined in a more logical fashion?
Using mortality rates alone is widely recognised to be inappropriate due to the vagaries of case-mix. Many approaches have been taken to try and produce a methodology allowing risk-adjusted comparative surgical audit of outcomes usually involving purely mortality rates. A common approach using age, sex, social deprivation (postcode), mode of admission and a range of co-morbidity factors has become fairly widespread. However, the acute physiological disturbance caused by an acute vascular or abdominal pathology in general surgery and fractures in orthopaedics can have a major impact on a patient’s immediate physiological status which will not be assessed by such a low number of chronic disease state variable approach. Certainly in colorectal disease some researchers have shown that such an approach will under-estimate an individual patient’s risk1 and our own studies have shown that this underestimate may be up to 29% in 76% of patients (table 1). The most widely known system which avoids such a problem is the POSSUM2-5 surgical audit system which allows a true measure of individual patient outcome prediction and allows performance measures to be assessed for individual units and surgeons. This is the recommended system by NCEPOD and the Surgical Royal Colleges and is used in 38 countries worldwide. The previous major problem revolving around the need for manual data accrual has now been solved by its complete automation using complex but fully validated surrogates derived from the HES dataset which are highly accurate and comparable with manual data collection (Surgical CRAB [Copeland’s Risk Adjusting Barometer] CRAB Clinical Informatics).
The use of low number of variables methodologies, however, is still widespread and has resulted in a growing trend amongst surgeons to be risk averse causing a similar situation to that in Babylon four thousand years ago. Unlike Babylon we now possess a range of interventional radiological and endoscopic techniques which avoid open surgery at a much lower risk to the patient. Nevertheless, there is still a wide range of pathological processes which can only be resolved by operative intervention. From our international database of 10 million procedures collected over the past twenty years we have seen that following an initial increase in high risk patients undergoing surgery the actual number of patients undergoing high risk surgery is now declining at an accelerating rate as illustrated in our own unit (table 2).
This cannot be explained solely by the increase in interventional radiological and endoscopic techniques. It is possible that surgeons are depriving patients of potentially curative or remedial surgery because of the anxiety caused by the publication of mortality rates in the public domain. There is certainly anecdotal evidence to support this contention but its identification can be inordinately difficult particularly with the increasing use of DNAR orders (Do not resuscitate orders), the re-designation of patients as palliative (with the resultant exclusion from many methodologies which use a low number of risk adjusting variables such as Dr Foster) and the use of ceilings of treatment in critical care units. While these rationales may well be appropriate there does need to be an attempt at quality assurance for the group of patients who do not undergo operative intervention as the more simplistic approaches at assessment may not be suitable.
If one investigates hospital mortality rates in an average district general hospital there will be approximately 120 deaths per month of which 105 are medical (with an average patient age of 84 years at time of death) and 15 deaths will be surgical. Of these fifteen, seven will be true palliative patients with advanced malignancy of chronic disease states usually vascular in nature. Of the eight who undergo surgery most will have serious life threatening abdominal or vascular disease with an occasional trauma case. There will rarely be more than one or two patients per month where death may potentially have been avoided. However, during the same period, 15% of all surgical admissions will have experienced complications. Surely this group requires more investigation and such investigation has a far greater potential to improve outcomes, possibly avoid death, reduce complaint and litigation and, by improving patient knowledge of complications, improve patient satisfaction.
The only current system which can accurately predict complications is the POSSUM system which can also be used pre-operatively to counsel patients as to the risks of the intended surgery. A number of groups now provide a web- enabled method of assessing this risk although only a total morbidity risk is available6. The Surgical CRAB systems do allow a more individualised measure of individual complication risks.
Much governmental and media focus has been given recently to reducing thrombotic problems and wound infections post-operatively, in particular by reducing MRSA rates and encouraging thromboprophylaxis risk assessment. However, these complication rates as can be seen in table 3 are relatively small and in many units in which care is deteriorating it is the chest infection to wound infection ratio which is often outside the normal range of 0.85 to 1.22. More focus in this area would improve care significantly nationwide but the causation is often a complex relationship between early and regular mobilisation by nursing staff, inadequate physiotherapy pre- and post-operatively, poor pain control and many times, as a result of antecedent complications. It is often noted by intensivists and surgical practitioners alike that death rarely occurs suddenly postoperatively; it usually follows a whole range of preceding complications.
How best then to assess these aspects of care both in surgery and medicine? The universal trigger tool allows such an approach as it assesses aspects of care which could potentially have caused harm (table 4). The thirty two variables are divided into five groups: a general group, a surgical group, an intensive care group, a medication group and a laboratory test group. As can be readily seen from table 5 many of these may not be harm events but rather be related to the patient’s disease state. The trigger tool has been widely validated worldwide and is probably the most refined methodology for looking at overall medical and nursing care.
The methodology has been, however, poorly applied as the usual technique for assessment requires random sampling and intensive case-note review. This is very labour intensive and can lead to inaccuracies in interpretation if the sample size is too small or non-representative. These inaccuracies can be avoided if the universal trigger tool methodology is applied to the whole of a Trust’s activity. The CRAB medical system using validated and highly accurate surrogate algorithms, however, allows all patients to be assessed using the universal trigger tool in real time and then compare the trigger events with outcomes. For the first time it is possible to examine the process of care and compare this with outcomes both in medical and surgical patients and most importantly in the care of the non-operative surgical patient.
Examples of such an approach have been published as part of the recent Keogh review of 14 hospitals. In some Trusts, where anastomotic leakage occurred more commonly postoperatively, it was possible to identify high levels of escalation which mitigated the effects of this particular complication. In other Trusts where such events occurred an increase in ward- based care trigger events were identified which did not mitigate the effects of the complication. In addition, where some units had higher deep venous thrombosis and pulmonary embolism rates occurring in both medicine and surgery, thus indicating trust problems with thromboprophylaxis.
Death alone may not be a true quality indicator and the application of inaccurate methods of risk adjustment may be having a deleterious effect on the choices offered to patients in response to their disease condition. The application of more refined POSSUM and CRAB based Surgical and Universal Trigger Tool based methodologies may avoid this potential harm. It would appear that the time has come for us to move away from mortality measures alone.
Life, unfortunately, is a terminal illness and there is so much more to life than death. We should, as both medical and nursing professions, consider giving more credence to complications and the delivery of care than to death alone. The use of risk adjusted morbidity measures and the analysis of the complexities of overall care using trigger tool methodologies would seem to offer this opportunity. We should not measure our success on death rates alone.
- 28 Feb, 2014
- 0 Comments