An Assessment of the POSSUM System in Orthopaedic Surgery
Mohamed K FRCS*, Copeland GP ChM**, Boot DA FRCS***,
Casserley HC FRCS***, Shackleford IM FRCS*** Sherry PG FRCS***,
Stewart GJ FRCS***
*Orthopaedic Registrar, Warrington Hospital
** Consultant General Surgeon, Warrington Hospital
*** Consultant Orthopaedic Surgeon, Warrington Hospital
Key words: Orthopaedic surgery, Surgical audit, POSSUM scoring system
First revision 11.12.2001
Abstract
The present article describes the development and validation of a scoring system for orthopaedic surgical audit. The present system is a minor modification of the established POSSUM scoring system widely used in general surgery. The orthopaedic POSSUM system developed yields predictions for mortality and morbidity which correlate well with the observed mortality and morbidity rates in a sample of 2326 orthopaedic patients operated upon in a twelve month period.
Introduction
The assessment of outcome after surgical endeavour is not a new science. As early as 1750 BC King Hammurabi of Babylon issued a number of decrees related to surgeons and their surgery. His most infamous of these codices was that if a surgeon operated on a free man and the patient became blind or worse still died then the surgeon should have his operating hand cut off. While to many surgeons world wide this codex may still seem to be in operation, a number of groups have attempted to devise more reliable and robust methods for assessing the outcome from surgical intervention1-3.
Clearly the use of raw mortality and morbidity data will produce differences in outcome when comparing differing units, but these apparent differences may be explained merely by variations in case mix and the type of surgery4, 5. In general surgery methods have been devised to allow comparison between units by taking the physiological status of patients and the operative severity into account1, 6, 7.
In the general surgical setting the POSSUM1 and P-POSSUM8 systems have proved to be the most reliable and widely applicable of all the scoring systems so far devised, and they have both been found to be equally applicable in sub-specialities including vascular surgery, surgical gastroenterology and urology. Indeed the POSSUM system has been found to be applicable in many differing health care organisations world wide9-11.
A number of orthopaedic authors have drawn attention to the variability in outcome following orthopaedic surgery, in particular fractured neck of femur and have argued that this variability is more likely to be related to case mix than to hospital facilities or surgeons12. That mortality and morbidity can be high particularly with proximal femoral fractures is well recognised13. However more recent studies have cautioned that predicting the outcome of orthopaedic intervention in the elderly injured patient merely on the basis of the injury severity score can be hazardous, as host factors appear to be of greater importance14.
A system which is heavily weighted towards physiological status would appear, potentially, to be of benefit in assessing the outcome from orthopaedic surgical intervention. Clearly such a system should allow comparison based on the patient’s physiological status and an assessment of the magnitude of surgery and its timing.
During the period 1996-1998 we have utilised similar methodologies, to those described and used previously by ourselves in the development of the POSSUM system1, to design an orthopaedic operative severity score which would allow the general surgical logistic regression equation to be applied in an orthopaedic setting. In total 22 operative severity factors were assessed which were reduced by multivariate analysis to the minimum number necessary to produce an accurate estimate of mortality and morbidity. The resulting operative severity score was not surprising similar in many ways to the general surgical score although individual factors and weightings for these factors were dissimilar.
The present study attempts to validate the application of this new method of assessing the outcome after orthopaedic surgery.
Patients and Methods
All consecutive patients admitted to Warrington Hospital during a 12 month period in whom orthopaedic surgery was performed on a non day case basis were assessed using the new orthopaedic POSSUM system.
This includes a physiological assessment and an operative severity assessment. The physiological assessment included twelve variables each divided into four grades with an exponentially increasing score value (1,2,4 and 8) (see table 1). In the main all score variables were available for all patients, but where no score variable was available a score of 1 was allocated. The operative severity score included six score variables each divided into four grades with an exponentially increasing score value (see table 1). Definitions of operative magnitude are illustrated in table 2. Where a operation is not listed the most closely approximately operation grouping was chosen. The number of operations variable indicates the chronology of procedures occurring within thirty days of the preceding operation. All operative severity score variables were obtainable in all patients, although histological confirmation was necessary in some individuals to complete all score elements.
Outcome was assessed as 30 day morbidity and mortality. 30 day mortality and morbidity assessment was chosen to allow comparability with the general surgical system and in view of the Department of Health’s publication of 30 day mortality statistics. The presence of the following complications were recorded as morbidity:
- Infection- chest, urinary, wound, joint, bony, septicaemia, pyrexia of unknown origin, or deep collection
- Haemorrhage- deep or wound
- Other wound problems- seroma requiring drainage or dehiscence
- Thromboembolic complications- deep venous thrombosis, pulmonary embolus, myocardial infarction or cerebrovascular accident
- Cardiac- failure, hypotension, or abnormal rhythm
- Respiratory- failure, collapse, or pneumonia
- Renal- failure or retention
- Unanticipated dislocation of prosthetic implant
Exact definitions are as have been previously published1.
Mortality and morbidity predictions (R1 for mortality and R2 for morbidity) for individual patients were estimated using previously determined equations.
Loge R1/(1-R1) = -7.04 + (0.13 x physiological score) + (0.16 x operative severity score)
Loge R2/(1-R2) = -5.91 + (0.16 x physiological score) + (0.19 x operative severity score)
Results
During a twelve month period 2326 patients underwent orthopaedic surgery on a non-day case basis. 44% of these patients were operated on in the elective setting, the remainder, 56%, were operated on urgently or as emergencies.
The overall mortality rate was 2.2%, this being 0.2% for elective cases increasing to 3.8% for urgent and emergency surgery. The overall morbidity rate was 10.8%, being 4.4% for elective cases and 15.9% for patients undergoing urgent or emergency surgery. The complication profile is shown in table 3.
Using the POSSUM logistic regression equations yielded an overall predicted mortality rate of 53 patients (versus 51 observed) and a predicted morbidity rate of 254 patients (versus 252 observed). The risk spectrum for both mortality and morbidity are shown in table 4, and follow the expected exponential distribution.
The number of operations performed by each of the six surgeons contributing to the study is shown in table 5. As can be seen there are apparent differences in each surgeon’s overall mortality and morbidity rates.
However when corrected for case mix using the POSSUM system there would appear to be little difference between observed mortality and morbidity rates and those predicted by POSSUM (table 6).
The predictive accuracy of these equations was assessed by the determination of receiver operating characteristic curves (ROC curves), by determining classification matrices for different levels of predicted mortality and morbidity. The resultant ROC curves are illustrated in Figures 1 and 2, which show good correlation across the risk range.
Discussion
There has been increasing worldwide public and political interest in the assessment of quality of care with regard to surgical outcome. This is perhaps easier within the surgical specialities as death following surgery is a fairly obvious adverse outcome. This has led many lay and non-surgical clinicians to suggest that mortality rates may be a suitable indicator of surgical prowess. As is illustrated by King Hammurabi of Babylon who removed the hands of surgeons whose patients died this is not a new phenomenon. There is, however, increasing awareness that such ‘raw’ mortality data may be at best incorrect and at worst dangerous. In addition morbidity is often ignored. Death following surgery is a rare event as demonstrated in the present study and usually follows a number of antecedent complications. If improvements in care are to be made then reduction in complications may well proceed reduction in mortality.
The overall mortality and morbidity rates for the twelve month study period appear to be in keeping with previously published rates for non day case surgery in the district general hospital setting1,6. Indeed the risk profile of the study sample does show similarities to the general surgical distribution1, 6 suggesting that the present study group is comparable to the average district general orthopaedic case mix. In the general surgical setting however there is a tendency to have a greater number of patients whose risk is greater than 70% for mortality1, 4, 6, 7.
The present study shows an excellent correlation between the overall observed mortality and morbidity rates and the predictions derived from the POSSUM logistic regression equations. The physiological variables assessed were those shown by logistic regression analysis to be the most important in predicting mortality and morbidity. Other additional variables were not found to independently improve the predictive ability of the logistic regression equation. We have not compared the POSSUM predictions with P-POSSUM as previous authors have clearly demonstrated that if the correct mathematical model is applied there are no significant differences in the predictive ability of either method15.
There would appear at first sight to be significant differences in outcome between the six surgeons studied. Mortality rates varied between 1.1% and 3.0%, and morbidity rates between 4.6% and 13.4%. Similar variations have been demonstrated in the general surgical setting and can be explained on the basis of case mix and operative severity. The orthopaedic POSSUM predictions in the present analysis would lend weight to the argument that a similar explanation may also account for apparently marked differences in surgical outcome in orthopaedic surgery. This is well demonstrated by the close correlation between observed and predicted rates for both mortality and morbidity in individual surgeons.
The ROC curves would suggest that the orthopaedic POSSUM score is equally applicable across the spectrum of surgical risk and that the individual patient predictions may be of benefit when assessing at clinical audit an individual patient in who death or complication occurs. Such a system is already in operation within general surgery and may also be of benefit in orthopaedic surgery. We have previously drawn attention to the benefits of audit in patients who survive but in who a predicted risk of death exceeds 50%, as these patients often have more to teach us than those that die. Further studies are at present in progress to quantify the benefits of preoperative optimisation in the high risk orthopaedic patient, as recently Wilson et al16 have demonstrated potentially significant improvements in overall care are possible by optimisation in this particular group.
Clearly no regression equation for risk assessment should remain static over time, although no significant change has been needed over the past ten years. However should dramatic changes occur in the future the equation could be easily updated without the need to alter the score variables and comparison over time would still be possible. Should this occur historically scored patients would be assessed with the present equation and new patients assessed with an updated equation.
The present study indicates that orthopaedic POSSUM can be used as an audit aid to assess the quality of orthopaedic care. It is possible a quality measure similar to that used in general surgery, the O/E ratio (the ratio of observed adverse events to predicted adverse events) could be used within orthopaedic surgery as a more reliable quality measure than simple mortality and morbidity rates. Using such a method a ratio of 1.00 indicates average care. A ratio greater than 1.00 worse care and ratio less than 1.00 better care. Further studies are in progress in other units to assess its more wide spread usage and application.
References
- Copeland GP, Jones D, Walters M. POSSUM: a scoring system for surgical audit. Br J Surg 1991;78:355-360
- Jones HJS, de Cossart L. Risk scoring in surgical patients. Br J Surg 1999;86:149-157
- Pillai SB, van Rij AM, Williams S, Thomson IA, Putterill MJ, Greig S. Complexity and risk-adjusted model for measuring surgical outcome. Br J Surg 1999;86:1567-1572
- Copeland GP, Jones D, Wilcox A, Harris PL. Comparative vascular audit using the POSSUM scoring system. Ann R Coll Surg Engl 1993;75:175-177
- Sagar PM, Hartley MN, MacFie J, Taylor BA, Copeland GP. Comparison of individual surgeon’s performance. Dis Colon Rectum 1996;38:654-658
- Copeland GP, Sagar P, Brennan J, Roberts G, Ward J, Cornford P et al. Risk adjusted analysis of surgeon performance. Br J Surg 1995;82:408-411
- Copeland GP. Assessing the surgeon: 10 years experience with the POSSUM system. J Clin Excell 2000;2:187-190
- Prytherch D, Whiteley MS, Higgins B, Weaver PC, Prout WG, Powell SJ. POSSUM and Portsmouth POSSUM for predicting mortality. Br J Surg 1998;85:1217-1220
- Brunelli A, Fianchini A, Xiume F, Gesuita R, Mattei A, Carle F. Evaluation of the POSSUM scoring system in lung surgery. Thorac Cardiovasc Surg 1998;46:141-146
- Gotohda N, Iwagaki H, Itano S. Can POSSUM, a scoring system for perioperative surgical risk, predict postoperative clinical course. Acta Medica Okayama 1998;52:325-329
- Tekkis PP, Kocher HM, Bentley AJ, Cullen PT, South LM, Trotter GA, Ellul JP. Operative mortality rates among surgeons: comparison of POSSUM and p-POSSUM scoring systems in gastrointestinal surgery. Dis Colon Rectum 2000;43:1528-1532
- Withey C, Morris R, Beech R, Backhouse A. Outcome following fractured neck of femur- variation in acute hospital care or case mix? J Public Health Med 1995;17:429-437
- Thomas M, Eastwood H. Re-evaluation of two simple prognostic scores of outcome after proximal femoral fractures. Injury 1996;27:111-115
- Van der Sluis CK, Timmer HW, Eisma WH, ten Duis HJ. Outcome in elderly injured patients: injury severity versus host factors. Injury 1997;28:588-592
- Wijesinghe LD, Mahmood T, Scott DLA, Berridge DC, Kent PJ, Kester RC. Comparison of POSSUM and the Portsmouth predictor equation for predicting death following vascular surgery. Br J Surg 1998;85:209-212
- Wilson J, Woods I, Fawcett J, Whall R, Dibb W, Morris C, McManus E. Reducing the risk of major elective surgery: randomised controlled trial of preoperative optimisation of oxygen delivery. BMJ 1999;318:1099-1103
Tables
Table 1: POSSUM sheet for physiological score
| SCORE | 1 | 2 | 4 | 8 |
| Age | <60 | 61 - 70 | >71 | |
| (yrs) | ||||
| Cardiac | On Cardiac | Oedema | JVP | |
| signs | drugs or | Warfarin | ||
| Normal | steroid | |||
| CXR | Borderline | Cardio | ||
| cardiomegaly | megaly | |||
| Resp. signs | SOB | SOB | SOB rest | |
| Exertion | stairs | |||
| Normal | Any other | |||
| CXR | Mild COAD | Mod COAD | change | |
| SYSTOLIC | 110 - 130 | 131 - 170 | >171 | <89 |
| BP | 100 - 109 | 90 - 99 | ||
| (mmHg) | ||||
| Pulse | 50 - 80 | 81 - 100 | 101 - 120 | >121 |
| (/min) | 40 - 49 | <39 | ||
| Coma Score | 15 | 12 - 14 | 9 - 11 | <8 |
| Urea | <7.5 | 7.6 - 10 | 10.1 - 15 | >15.1 |
| (mmol/l) | ||||
| Na | >136 | 131 - 135 | 126 - 130 | <125 |
| (mmol/l) | ||||
| K | 3.5 - 5 | 3.2 - 3.4 | 2.9 - 3.1 | <2.8 |
| (mmol/l) | 5.1 - 5.3 | 5.4 - 5.9 | >6 | |
| Hb | 13 - 16 | 11.5 - 12.9 | 10 - 11.4 | <9.9 |
| g/100ml | 16.1 - 17 | 17.1 - 18 | >18.1 | |
| WCC | 4 - 10 | 10.1 - 20 | >20.1 | |
| x1012/l | 3.1 - 3.9 | <3 | ||
| ECG | Normal | AF (60 - 90) | Any other | |
| change |
Table 2: POSSUM sheet for operative severity score
| SCORE | 1 | 2 | 4 | 8 |
| Op magnitude | Minor | Inter | Major | Major + |
| Number of op | 1 | 2 | >2 | |
| within 30days | ||||
| Blood loss | <100 | 101 - 500 | 501 - 999 | >1000 |
| per operation | ||||
| Contamination | No | Incised | Minor | Gross |
| wound | contam or | contam or | ||
| ie stab | necrotic | necrotic | ||
| tissue | tissue | |||
| Presence of | No | 1o | Node | Distant |
| malignancy | metastases | metastases | ||
| Timing of | Elec. | Emerg. | Emerg. | |
| operation | resus poss | immed | ||
| <48hrs | <6hrs |
Table 3:Number of patients experiencing complications postoperatively.
(A number of patients experienced multiple complications)
| Complications | Number |
|---|---|
| Haemorrhage | 14 |
| Infection | |
| Chest | 59 |
| Urinary | 34 |
| Wound | 73 |
| Septicaemia | 7 |
| PUO | 5 |
| Respiratory Failure | 17 |
| Cardiac | |
| Hypotension | 29 |
| Cardiac Failure | 23 |
| Arrhythmia | 6 |
| Thrombotic | |
| Deep venous thrombosis | 12 |
| Pulmonary embolus | 9 |
| Myocardial infarction | 17 |
| Cerebrovascular infarction | 7 |
| Limb occlusion | 2 |
| Other vascular complications | 4 |
| Renal Failure | 15 |
| Urinary retention | 32 |
| Other wound problems | 10 |
| Prosthetic problems | 7 |
| Miscellaneous | 26 |
Table 4: Risk spectrums for both morbidity and mortality.
| Risk Band | Mortality (number of patients) |
Morbidity (number of patients) |
|---|---|---|
| <10% | 2086 | 1042 |
| 10-20% | 140 | 600 |
| 20-30% | 51 | 230 |
| 30-40% | 23 | 124 |
| 40-50% | 11 | 99 |
| 50-60% | 3 | 72 |
| 60-70% | 6 | 48 |
| 70-80% | 3 | 49 |
| 80-90% | 1 | 37 |
| >90% | 2 | 25 |
Table 5: The variability in surgeon workload, mortality and morbidity rates.
| Surgeon | Number of Procedures | Mortality | Morbidity |
|---|---|---|---|
| 1 | 495 | 3.0% | 12.9% |
| 2 | 474 | 1.1% | 10.5% |
| 3 | 372 | 2.4% | 11.0% |
| 4 | 216 | 1.8% | 4.6% |
| 5 | 426 | 2.1% | 9.6% |
| 6 | 343 | 2.6% | 13.4% |
Table 6: Comparisons between observed and predicted mortality and morbidity rates for individual surgeons during the study period
| Surgeon | Mortality | Morbidity | ||
|---|---|---|---|---|
| Observed | Predicted | Observed | Predicted | |
| 1 | 15 | 15 | 62 | 64 |
| 2 | 5 | 6 | 52 | 50 |
| 3 | 9 | 10 | 41 | 43 |
| 4 | 4 | 4 | 10 | 10 |
| 5 | 9 | 9 | 41 | 40 |
| 6 | 9 | 9 | 46 | 47 |
Legends
Fig 1. Receiver operating characteristic curve for mortality. A curve approaching the linear line indicates no predictive ability for the assessing system. The further from the linear line the better the predictive ability.
Fig 2. Receiver operating characteristic curve for morbidity.
