Safer Human-Computer Interaction for Healthcare

Introduction

Summary of the impact: Our research into the Human-Computer Interaction (HCI) issues behind medical error has enabled hospitals to procure safer devices and is strategically changing attitudes. Approximately 10% of deaths due to preventable errors in hospitals are computational errors; in absolute numbers, this is higher than road fatality rates. Corresponding increased hospital stays, etc, are estimated to cost the NHS >£600Mpa. Our formal analyses and laboratory research with clinicians show error rates can be reliably reduced dramatically by software and HCI improvements. Swansea’s research has significant international reach across decision-makers, clinicians, manufacturers, and regulators. The US Food and Drug Administration (the FDA, the leading international regulator) say research-based improvements will take decades and have therefore started to co-author papers with us to improve impact (all healthcare manufacturers watch the FDA closely).

Underpinning research

Our original research discovered that, and analysed how, interactive medical devices (e.g., drug delivery systems) ignore certain classes of critical user error, and that by managing error better, patient harm can be reduced. Clinicians make data entry slips (about 4% of keystrokes entered by nurses cause unnoticed errors), and errors are parsed badly and misinterpreted on many devices. If neither user nor device notices errors, this can lead to adverse incidents. 

We developed rigorous (e.g., using theorem proving systems) and novel (e.g., using eye tracking with nurses) methods for analysis and evaluation that have transformed the understanding of user interface safety for healthcare. Our approaches refine theories and design techniques to assure safety, and ensure systems are more resilient to human error. Our analyses of systematic design variations led to techniques to reduce severe errors by factors up to 25 compared to devices in current use in hospitals. We thus develop evidence-based principles, which have been used, for example, in NHS procurement to purchase safer devices.

Impact is viewed in the context of preventable hospital deaths: 17% are due to numerical/calculation errors alone (under-reported, since few clinicians notice them). We found paediatric intensive care patients have a 20% chance of avoidable drug calculation errors per week. Exact figures are contentious; the British Medical Journal estimates UK acute hospital preventable fatality rates of 14,000pa — but it is a mistake to focus too much on fatalities: extended hospitals stays and litigation costs the NHS >£600Mpa, and there are social and financial impacts on patients, relatives, and on clinicians too. 

Continued research has led to international best paper prizes, at ACM CHI and ACM EICS (Engineering Interactive Computer Systems) etc. 3 PhD students have won best paper prizes for both doctoral work and for mainstream refereed papers. 

Details of the impact

Healthcare suffers because of preventable errors. IT is a solution but also a current problem. The impact of our research can be described in the following ways:

Policy debate stimulated and informed by research. We conceived of and ran “Tully Meetings” (named after Prof Colin Tully, one of the founders of the NHS23 group) to bring top computer scientists and clinicians. 

Potential losses mitigated by improved risk assessment by health service. Collaborating with the NHS we showed in the largest ever study poor design of infusion pumps induces wastage of ~£1,000/pump/year. 

Public discourse stimulated and informed. We have given 120+ presentations (in REF period) at medical conferences, industrial seminars and workshops at international conferences such as ACM CHI (2010; 2011; 2013), ACM EICS (2011; 2012; 2013), BCS HCI (every year), as well as presentations to CTOs of many organizations and hospital trusts. 

Improved quality, efficiency & productivity of professional service. Drug doses rely on complex calculations. We developed calculators that eliminate some sources of error completely. A prototype was exhibited at a Royal Society Summer Science exhibition in 2005 and in TECHFEST, in Mumbai, India, etc. 

Defined best practice for professional bodies and learned societies. The Royal College of Physicians, the Royal College of Anaesthetists, the Royal College of Physicians Edinburgh, the Royal College of Pharmacists, Guild of Healthcare Pharmacists, Scottish Intensive Care Society, Central Sterilising Committee, Scottish Clinical Skills Network, etc, have had invited lectures from us. We have undertaken confidential assessments of medical devices for the NHS in respect of criminal and professional investigations.

Significance Our impact is increasing, but even if applied unchanged today we could prevent ~7,000 UK deaths/year, with additional social benefits in finance and wellbeing, and additional impact for the “second victims,” the clinicians who have to operate this equipment.

References to the research

  • A. Cox, P. Oladimeji & H. Thimbleby, “Number Entry Interfaces and their Effects on Errors and Number Perception,” Proc IFIP Conf on Human-Computer Interaction — Interact 2011, IV:178–185, Springer Verlag, 2011. Peer reviewed, full paper.

Unique laboratory experiments show skilled nurse drug dosing errors can be reduced by a factor of over 6 by improved design of user interfaces; also used eye tracking to support rationales for similar error reductions in other applications.

  • P. Lee, H. Thimbleby & F. Thompson, “Analysis of Infusion Pump Error Logs and Their Significance for Healthcare,” British Journal of Nursing, 21(8):S12–S22, 2012. Peer reviewed, full paper.

The largest survey and analysis of infusion pump log data (approximately 500,000 hours); shows a high proportion of staff time is wasted in user interface design problems.

  • P. Masci, R. Ruksenas, P. Oladimeji, A. Cauchi, A. Gimblett, Y. Li, P. Curzon & H. Thimbleby, “The benefits of formalising design guidelines: A case study on the predictability of drug infusion pumps,” Journal of Innovations in Systems and Software Engineering. http://dx.doi.org/10.1007/s11334-013-0200-4, 2013. Peer reviewed, full paper.

Formal analysis can completely eliminate certain classes of use error.

  • H. Thimbleby, “Interaction Walkthrough: Evaluation of Safety Critical Interactive Systems,” Proc XIII International Workshop on Design, Specification and Verification of Interactive Systems — DSVIS 2006, Lecture Notes in Computer Science, 4323:52–66, Springer Verlag, 2007. http://dx.doi.org/10.1007/978-3-540-69554-7_5. Peer reviewed, full paper.

Paper underpinning the EPSRC Programme Grant — showed there was a serious, but soluble problem in health IT.

  • H. Thimbleby & P. Cairns, “Reducing Number Entry Errors: Solving a Widespread, Serious Problem,” Journal Royal Society Interface, 7(51):1429–1439, 2010. http://dx.doi.org/10.1098/rsif.2010.0112. Peer reviewed, full paper.

Laboratory experiments and simulation show improved user interfaces can reduce out by ten errors by a factor of two.

  • H. Thimbleby & D. Williams, “Using Nomograms to Reduce Harm from Clinical Calculations,” Proc IEEE International Conf on Healthcare Informatics — ICHI, 2013 [in press; publication date September 2013]. Peer reviewed, full paper.

Alternatives to conventional calculators can reduce error rates and their relative magnitude, in our experiments completely eliminating errors out by more than 50%.


Our work has led to many EPSRC grants, including EP/G003971/1, EP/G059063/1, EP/F059116/1 & EP/J020834/1, and a Royal Society-Leverhulme Senior Research Fellowship.