Prescribers’ interactions with medication alerts at the point of prescribing: A multi-method, in situ investigation of the human–computer interaction

https://doi.org/10.1016/j.ijmedinf.2012.01.002Get rights and content

Abstract

Purpose

Few studies have examined prescribers’ interactions with medication alerts at the point of prescribing. We conducted an in situ, human factors investigation of outpatient prescribing to uncover factors that influence the prescriber–alert interaction and identify strategies to improve alert design.

Methods

Field observations and interviews were conducted with outpatient prescribers at a major Veterans Affairs Medical Center. Physicians, clinical pharmacists, and nurse practitioners were recruited across five primary care clinics and eight specialty clinics. Prescribers were observed in situ as they ordered medications for patients and resolved alerts. Researchers collected 351 pages of typed notes across 102 hours of observations and interviews. An interdisciplinary team identified emergent themes via inductive qualitative analysis.

Results

Altogether, 320 alerts were observed among 30 prescribers and their interactions with 146 patients. Qualitative analysis uncovered 44 emergent themes and 9 overarching factors, which were organized into a framework that describes the prescriber–alert interaction. Prescribers’ ability to act on alerts was impeded by the alert interface, which did not adequately support all prescriber types.

Conclusions

This empiric study produced a novel framework for understanding the prescriber–alert interaction. Results revealed key components of the alert interface that influence prescribers and indicate a need for more universal design. Actionable design recommendations are presented and may be used to enhance alert design and patient safety.

Highlights

► A richer understanding of prescribers’ interactions with alerts during patient care. ► Actionable recommendations to improve alert design. ► A framework that describes prescribers’ interactions with alerts. ► Evidence that alert designs more closely match clinical pharmacist mental-models. ► Alert interface components that should be enhanced to support non-pharmacists.

Introduction

The Institute of Medicine estimates at least 1.5 million preventable adverse drug events (ADEs) occur annually in the United States [1]. Automated, computerized alerts can warn prescribers about potential problems during medication ordering (e.g., drug interactions, drug-allergy warnings) and are intended to reduce ADEs. Proponents of alert systems hope these warnings will mitigate harmful orders before medications are dispensed to patients [2], [3].

However, alert systems have not reached their full potential for supporting prescribers [4], [5]. Prescribers are overwhelmed by the number of alerts [6], [7], [8], and studies suggest that alert designs do not fully support prescriber decision-making [6], [7]. In one survey, 41% of clinicians indicated that ‘insufficient information’ was a barrier to alert use [9]. Enhanced medication alert designs could benefit patients, prescribers, and healthcare organizations.

Researchers have conducted surveys of providers [10], [11], [12], examined alert frequencies [13], [14], and quantified alert override rates [7], [15], [16]; only a handful of studies have collected data on alerts via focus groups or interviews [17], [18], [19]. Directly observing prescriber behavior may provide a robust picture of the prescriber–alert interaction and reveal additional factors that influence alert success [15]. One study conducted disguised observations with six residents as they resolved alerts [20]. Medication alerts occurred for one-third of orders on the internal medicine wards, and override rate frequencies were high for several types of alerts. Another study examined alert resolution under simulated working conditions to assess prescriber accuracy [21]. Most alerts were resolved accurately, but incorrect rules or reasoning was used to justify responses to 36% of alerts. A systematic review of computerized alerts found substantial gaps in the literature with respect to human factors issues and how to display alerts at the point of prescribing [22]. Human factors principles may inform alert design [23], but information on prescribers’ actual interactions with medication alerts is lacking. The objective of this investigation was to observe prescribers during their work, conduct inductive, qualitative analysis to uncover factors that influence prescriber–alert interactions, and identify strategies to enhance alert design. To our knowledge, this is the first in situ investigation of the human–computer interaction between prescribers and naturally-occurring medication alerts.

Section snippets

Setting

This study was conducted at a large, academic Midwestern VA Medical Center (VAMC). The VA's electronic health record (EHR), the Computerized Patient Record System (CPRS), includes computerized provider order entry (CPOE) with automated medication alerts [6], [24]. VA medication alerts, more formally known as ‘order checks’, appear real-time in pop-up windows during the medication ordering process (Fig. 1). In general, the alert system we studied does not incorporate parameters such as patient

Data collection

Researchers observed 320 naturally-occurring alerts among 30 prescribers and 146 patients. Thirty-seven prescribers were invited to participate, and we successfully recruited 20 primary care and 10 specialty clinic prescribers (18 physicians, 7 nurse practitioners, and 5 clinical pharmacists). Researchers recruited both men (14) and women (16). Participants’ average VA experience was 10 years (range less than 1 year to 24 years); and their average age was 42 years (range 27–63 years).

Discussion

To our knowledge, this is the first in situ study to specifically investigate the prescriber–alert interaction. This study provides a more in-depth understanding of prescribers’ interactions with medication alerts at the point of prescribing. Data were qualitatively analyzed by a diverse team of professionals representing clinical, pharmaceutical, and human factors engineering expertise. This strengthened the rigor and breadth of the findings.

Author contributions

All authors contributed to the study design. AR and MM collected the data. AR, AZ, MM, and JS conducted the qualitative analysis. AR drafted the initial manuscript, and all authors contributed to and approved the final version of the manuscript.

Conflict of interest statement

The authors report there are no conflicts of interest.

Summary points

What was already known on the topic

  • Few alerts lead to medication changes, and alert fatigue poses a substantial barrier to alert effectiveness.

  • Prescribers are often unaware of important medication conflicts, but alert designs have not reached their full potential for aiding decision-making.

  • For all types of software systems, the interface plays a vital role in the human–computer interaction and can be challenging to design.

What

Acknowledgements

We would like to express our gratitude to the study participants. It has been an honor to observe their work. We also wish to thank the many local and national VA informatics specialists who helped answer our questions about the alert system. Thanks to Richard Frankel, PhD, for input on qualitative methods. Ms. Diana Lunsford assisted with Fig. 2 graphics. This research was supported by the VA HSR&D Center of Excellence on Implementing Evidence-Based Practice, Center grant #HFP 04-148. Dr. Russ

References (59)

  • R. Tamblyn et al.

    The development and evaluation of an integrated electronic prescribing and drug management system for primary care

    J. Am. Med. Inform. Assoc.

    (2006)
  • E.M. Campbell et al.

    Types of unintended consequences related to computerized provider order entry

    J. Am. Med. Inform. Assoc.

    (2006)
  • L. Slaughter et al.

    A framework for capturing the interactions between laypersons’ understanding of disease, information gathering behaviors, and actions taken during an epidemic

    J. Biomed. Inform.

    (2005)
  • H. Van der Sijs et al.

    Functionality test for drug safety alerting in computerized physician order entry systems

    Int. J. Med. Inform.

    (2010)
  • National Academy of Sciences, Preventing medication errors. Report brief from the Institute of Medicine, July 2006, pp....
  • E.N. Van Roon et al.

    Clinical relevance of drug–drug interactions: a structured assessment procedure

    Drug Saf.

    (2005)
  • R. Tamblyn et al.

    The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care

    Can. Med. Assoc. J.

    (2003)
  • A.C. Feldstein et al.

    Reducing warfarin medication interactions: an interrupted time series evaluation

    Arch. Intern. Med.

    (2006)
  • A.J. Grizzle et al.

    Reasons provided by prescribers when overriding drug–drug interaction alerts

    Am. J. Manage. Care

    (2007)
  • S.N. Weingart et al.

    Physicians’ decisions to override computerized drug alerts in primary care

    Arch. Intern. Med.

    (2003)
  • D. Magnus et al.

    GPs’ views on computerized drug interaction alerts: questionnaire survey

    J. Clin. Pharm. Ther.

    (2002)
  • P.A. Glassman et al.

    Exposure to automated drug alerts over time: effects on clinicians’ knowledge and perceptions

    Med. Care

    (2006)
  • L.K. Taylor, Y. Kawasumi, G. Bartlett, R. Tamblyn, Inappropriate prescribing practices: the challenge and opportunity...
  • T. Isaac et al.

    Overrides of medication alerts in ambulatory care

    Arch. Intern. Med.

    (2009)
  • K.L. Lapane et al.

    A mixed method study of the merits of e-prescribing drug alerts in primary care

    J. Gen. Intern. Med.

    (2008)
  • M.A. Krall, D.F. Sittig, Clinician's assessments of outpatient electronic medical record alert and reminder usability...
  • H. Van der Sijs et al.

    Drug safety alert generation and overriding in a large Dutch university medical centre

    Pharmacoepidemiol. Drug Saf.

    (2009)
  • S. Phansalkar et al.

    A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems

    J. Am. Med. Inform. Assoc.

    (2010)
  • K.W. Trettin, VA computer system drug interaction pop-up boxes: are they critical or significant? Journal [serial on...
  • Cited by (68)

    • A user-centered evaluation of medication therapy management alerts for community pharmacists: Recommendations to improve usability and usefulness

      2021, Research in Social and Administrative Pharmacy
      Citation Excerpt :

      Finally, Russ' model describes 9 factors that influence how prescribers perceive and respond to medication alerts. These factors are: 1) alert system logic, 2) alert system redundancy, 3) alert content, 4) alert display, 5) cognitive factors, 6) pharmaceutical knowledge, 7) medication management, 8) workflow, and 9) alert system reliability.25 As described below, this framework was applied during the collection and analysis of interview and usability testing data, and triangulation of findings across data sources.

    • A cognitive systems engineering design approach to improve the usability of electronic order forms for medical consultation

      2018, Journal of Biomedical Informatics
      Citation Excerpt :

      To better understand consultation requests within clinical workflow, individual data collectors observed health care providers during clinical encounters. The observations were conducted in clinicians’ clinics as half-day sessions lasting approximately four hours [44]. At the beginning of encounters, written assent was obtained from patients.

    View all citing articles on Scopus
    View full text