Commentary |
Falls risk-prediction tools for hospital inpatients. Time to put them to bed?
School of Health and Social Care and Institute of Health Sciences, University of Reading, Bulmershe Campus, Reading, Berks, UK
Address correspondence to: David Oliver. Email: D.oliver{at}reading.ac.uk
Accidental falls are the commonest safety incidents affecting hospital inpatients and care-home residents [1, 2]. The recent National Patient Safety Agency (NPSA) report, ‘Slips Trips and Falls in Hospital’ [2], identified over 200,000 reported falls incidents from acute, community and mental health trusts in England and Wales from 2004/2005 alone, (some 32% of all incidents in all age groups) though we know from this and other sources that such incidents are under-reported [3]. Such falls are associated with a range of adverse outcomes including injury, impaired confidence and function, increased length of stay, institutionalisation anxiety and guilt for staff and relatives, complaint and litigation. They should, therefore, be a major risk management priority for hospitals and care homes (where around 50% of residents fall at least once a year) [4], and have recently been made a main focus for examining older patients' care by the Healthcare Commission [5].
There is a growing body of evidence on interventions to prevent falls and falls-related injuries in hospital [1, 6, 7]. One component of many research interventions, and a common feature of ‘real-life’ falls policies in hospitals, is the use of falls risk-prediction tools. By this, I do not mean ‘checklists’ of common risk factors to prompt specific action by staff, which might, in turn, reduce falls. Such factors might include environmental and equipment safety, medication, hypotension, visual impairment, muscle weakness or postural instability, cognitive impairment, restlessness or agitation, all of which, amongst others, have been targeted in successful falls intervention programmes [8]. I have no argument with the use of such tools, which, in effect, prompt good comprehensive geriatric assessment and care-planning. My concern is over scoring tools, which purport simplistically to classify patients as having a ‘high’ or ‘low’ risk of falling so that interventions can be targeted to ‘high-risk’ patients.
This approach can work well in other fields, for instance, with diagnostic screening tests such as troponin for acute coronary syndrome, or d-dimer for suspected pulmonary embolism (both of which are good ‘true negative’ tests with high specificity). It is also possible for clinical prediction tools with continuous scoring to be used to calculate overall probability of an adverse event. Examples of these include the Kings Fund ‘Patient at Risk of Readmission Scores’ (PARR), or Critical Care Early Warning Scores, both of which correlate tightly with the probability of the adverse event, (i.e. hospital admission or cardiac arrest/ICU admission) [9, 10]. Falls risk-assessment tools in hospital have rarely been used in this probabilistic way, rather, they place patients categorically as either at ‘high’ or ‘low’ risk. Though most work has been done in hospitals, my comments on the use of such tools in hospitals could apply equally to their use in long-term care or community settings.
There have been a succession of systematic reviews on validation studies for hospital falls prediction tools in recent years [8, 11–13] and several more new validation studies are in press. In this issue of Age and Ageing, Vassallo et al. [14] report a prospective validation study comparing two tools (STRATIFY and Downton Score) with nurses' clinical judgement (largely based on wandering behaviour) on a cohort of 200 geriatric patients. Meanwhile, Ashburn et al. [15] looked at 122 consecutive patients discharged from a stroke ward, following up on them to record further falls at home over 12 months. Sixty-three experienced one or more further falls. Before discharge, they collected a variety of structured clinical data, a score based on ‘near-falls’ in hospital, and poor upper limb function on retrospective fitting predicted falls with 70% specificity and 60% sensitivity. In Vassallo et al., the single item of ‘wandering’ identified by nurses conferred better predictive accuracy than either of the formal scores, but significantly lower sensitivity (though this might not be the case for populations where wandering is infrequent). These papers raise some interesting questions around the practical utility in predicting and preventing falls, the ‘trade-off’ between the various elements of predictive validity, their validity in other (quite different) populations and settings. A detailed exploration of these and allied issues is not possible here, but is explored in recent reviews. However, even as one of the authors of the most widely validated tool for use in hospital (STRATIFY) [16]—still used in many hospitals [1, 2]—I am happy to recant. I do not believe that STRATIFY, or any other tool, is good enough at its job. In the words of the commercial, they do not do ‘what it says on the tin’. Here is why:
- In order to be useful [8, 13, 17, 18], a prediction tool should have ease and speed of completion, a small number of items (not requiring specialist assessment technology or skills), transparent, simple and evidence-based scoring and good inter-rater reliability. Most importantly, its predictive ability should have been prospectively validated in the population (or something similar) in which it is to be used. Good ‘internal validity’ does not mean that the tool has ‘external validity’ for a totally different population. Yet, many prediction tools in use (as the NPSA and other reports have shown [2, 19]) have been ‘home made’ and un-validated, or ‘modified’ from validated tools, with extra items thrown in because they have ‘face validity’ to staff (in effect, mixing a falls risk factor checklist with a prediction tool) and confusing the factors which can predict falls with the full range of those that can cause them or can be modified to prevent them. One or two tools such as STRATIFY or the Morse Falls Scale do fulfil most of these criteria. So why not use them?
- To answer as to why we should not use them, we need first to consider the elements of predictive validity. These have been set out by Wyatt and Altman [17, 18] and in the STARD guidelines on the use of diagnostic tools [20]. These include sensitivity (i.e. ‘what percentage of patients who have fallen were predicted as ‘high-risk’), specificity (i.e. what percentage of people who did not fall were classified as ‘low-risk’), Positive Predictive Value (PPV) (i.e. what percentage of people identified as ‘high-risk’ go on to fall?) and Negative Predictive Value (NPV) (i.e. ‘what percentage of patients identified as ‘low-risk’ go on not to fall?). We can also look at Total Predictive Accuracy (or correct classification of fallers and non-fallers) and at ROC curves (plotting sensitivity against 1 minus specificity) to determine the optimum predictive cut-off score. Both NPV and PPV depend on the prevalence of the reference event (falls) in the population. However, even if ‘headline figures’ around, say, specificity or NPV are high—meaning that reassurance can be given about low-risk patients if the PPV or total predictive accuracy are low (which they have tended to be in most validation studies) then staff will simply end up targeting most of the patients on the unit as ‘high-risk’—rendering the use of a risk-prediction tool pointless and a poor use of staff time. If NPV and specificity are low then staff may be falsely reassured that they no longer have to worry about falls risks in those patients. In reality, no tools consistently perform well enough across all elements of operational predictive validity to be of much practical use.
- For any screening or risk-assessment tool, it must ‘value-add’ either by being better than the routine clinical judgement of staff, or by increasing the percentage of patients in whom risk is assessed. This seems to work well with say, nutritional screening instruments [21], but as Vassallo et al. [14] have shown here, this is not necessarily the case with falls risk-assessment tools.
- The nature of frail or acutely ill older people in hospital is that their falls risk is often variable over time as they develop and recover from inter-current illness, postural instability, delirium, hypotension etc. Yet, falls risk-assessment tools have tended to be applied as ‘one-off’ measurements, which do not reflect this reality [8, 12].
- Merely ‘ticking the box’ to say a ‘falls risk assessment’ (even one which is well validated) has been completed does not mean that anything concrete has been done to prevent falls. Identifying someone at high risk has to lead to action to modify that risk or it is meaningless, and the completion of the score merely gives a false reassurance that now ‘something is being done’. But it is not! The NPSA report [2] surveyed many hospitals where often ‘made-up’ risk-assessment tools led to ‘a limited repertoire of stereotyped actions’ which were unlikely to have any meaningful effect on falls prevention.
- Following from this, if we accept that the purpose of risk-assessment tools is ultimately to prevent falls, some of the better-quality positive studies on falls prevention in hospitals eschewed the use of risk-prediction tools altogether, but still succeeded in reducing falls rates. It is by no means certain that they are an essential element of effective fall prevention. And as most interventions have been multi-factorial, even when they have been employed, it is hard to be certain of the relative contribution of risk assessment to the overall effect.
All these considerations and caveats apply equally to care-home residents or older people dwelling in their own homes. In both cases, fewer data on predictive validity exist and the index event—falls—is so common as to include most of the population anyway, with evidence-based interventions often not employing the use of risk-prediction tools.
Often, when I advocate that we should abandon the use of falls prediction tools, staff struggling to tackle the pressing problem of falls prevention become vexed. A prominent fellow researcher in this field labelled my advice as ‘unethical’ [22] suggesting that this meant simply allowing patients to fall. However, if we look at the evidence for fall prevention in hospitals, especially from RCTs, it is contested [7, 8, 23]. Also, many of those interventions which have produced sustained reductions in falls rates have employed approaches which focus on identifying falls risk factors, using each new fall as an indicator to prompt a reassessment, putting in care plans to modify each one; learning from incident reporting as part of governance; education and training for staff; and making the physical environment safer for all [24].
I can easily understand why there is an attachment to importing or devising ‘a tool for everything’. I can understand that using a tool may be seen to focus the minds of staff on the problem, and to demonstrate to inspectors or risk managers that at least ‘now we are doing something about falls’. However, unless we have an understanding of the limitations of such tools and the evidence for their use, this is a fool's paradise. If we look after all older people in hospital better, it is likely they will fall less.
 |
References |
|---|
 Top References |
|---|
- Australian Commission on Safety and Quality in Healthcare. Preventing Falls and Harm from Falls in Older People. (2005) accessed 31/01/07. Best practice guidelines for Australian hospitals and residential aged care facilities, Canberra, Australian Commission. Available at http://www.safetyandquality.org/internet/safety/publishing.nsf/Content/falls.
- National Patient Safety Agency. Slips Trips and Falls in Hospital (2007) Accessed 4/3/08. London: NPSA. Available at www.npsa.nhs.uk/EasySiteWeb/GatewayLink.aspx?alId=5533.
- Saari AB, Sheldon TA, Cracknell A, et al. Sensitivity of reporting system for routine patient safety incidents in an NHS hospital. BMJ (2007) 334:79.
[Abstract/Free Full Text] - Masud T, Morris R. Epidemiology of falls. Age Ageing (2001) 30(Supp. 1):3–7.
[Free Full Text] - Healthcare Commission. Incorporating Inpatient Falls Prevention into the Core Standards Assessment for DH Core Standard 1a (2008) London: Healthcare Commission. In press.
- Coussement J, De Paepe L, Schwendimann R, et al. Interventions for preventing falls in acute and chronic care hospitals: a systematic review and meta-analysis. J Am Geriatr Soc (2008) 56:29–37.[Web of Science][Medline]
- Oliver D, Connelly J, Victor C, et al. Strategies to prevent falls and fractures in hospitals and care homes and effects of cognitive impairment: systematic review and meta-analyses. BMJ (2007) 334:82–7.
[Abstract/Free Full Text] - Oliver D, McMurdo M, Daly F, et al. Risk factors and risk assessment tools for falls in hospital inpatients. A systematic review. Age Ageing (2004) 33:122–30.
[Abstract/Free Full Text] - Goldhill DR, Worthington L, Mulcahy AJ, et al. The patient-at-risk team: identifying and managing seriously ill ward patients. Anaesthesia (1999) 54:853–60.[CrossRef][Web of Science][Medline]
- Billings J, Dixon J, Mijanovich T, et al. Case-finding for patients at high risk of readmission to hospital. Development of algorithm to identify high-risk patients. BMJ (2006) 333:327.
[Abstract/Free Full Text] - Haines TP, Hill K, Walshe W, et al. Design-related bias in hospital falls risk screening tool predictive accuracy evaluations: Systematic Review and Meta-Analysis. J Gerontol (2007) 62A:664–72.[Web of Science]
- Myers H. Hospital falls risk assessment tools: a critique of the literature. Int J Nurs Pract (2003) 9:233–5.
- Scott V, Votova K, Scanlan A, et al. Multifactorial and functional mobility assessment tools for falls risk among older adults in the community, home-support, long-term care and acute settings. Age Ageing (2007) 36:130–40.
[Abstract/Free Full Text] - Vassallo M, Poynter L, Sharma J, et al. Fall risk assessment tools compared with clinical judgement: an evaluation on a rehabilitation ward. Age Ageing (2008) 37:277–81.
[Abstract/Free Full Text] - Ashburn A, Hyndman D, Pickering R, et al. Predicting people with stroke at risk of falls. Age Ageing (2008) 37:270–76.
[Abstract/Free Full Text] - Oliver D, Britton M, Seed P, et al. Development and evaluation of an evidence-based risk assessment tool (STRATIFY) to predict which elderly patients will fall: case control and cohort studies. Br Med J (1997) 315:1049–51.
[Abstract/Free Full Text] - Wyatt J, Altman D. Prognostic models: clinically useful or quickly forgotten? BMJ (1995) 311:539–41.
- Bowers D, House A, Owens D, eds. Measuring the characteristics of measures. In: Understanding Clinical Papers (2003) Chichester: John Wiley and Sons. 81–5. Chapter 14.
- Morse JM. The safety of safety research: the case of patient falls. Can J Nurs Res (2006) 38:72–86.
- Meyer JG. Guidelines for reporting information in studies of diagnostic test accuracy. The STARD initiative. J Pers Assess (2003) 81:191–3.[CrossRef][Web of Science][Medline]
- Lean M, Wiseman M. Malnutrition in hospitals. BMJ (2008) 336:290.
[Free Full Text] - Morse JM. The safety of patient safety research: the case of patient falls. Can J Nurs Res (2006) 38(2):72–8.
- Cameron ID, Kurrle S. Preventing falls in elderly people living in hospitals and care homes. BMJ (2007) 334:53–4.
[Free Full Text] - Oliver D. Preventing falls and falls-injuries in hospitals and long-term care facilities. Rev Clin Gerontol (2008) in press.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  D. Oliver, A. Papaioannou, L. Giangregorio, L. Thabane, K. Reizgys, and G. Foster A systematic review and meta-analysis of studies using the STRATIFY tool for prediction of falls in hospital patients: how well does it work? Age Ageing, November 1, 2008; 37(6): 621 - 627. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||