Providers’ Reactions to an Automated Health Maintenance Reminder System Incorporated into the Patient’s Electronic Medical Record
=====================================================================================================================================

* Kenneth G. Schellhase
* Thomas D. Koepsell
* Thomas E. Norris

## Abstract

*Background:* Automated health maintenance reminder (HMR) systems embedded in electronic medical records systems have been found to improve utilization of preventive services, but underuse persists. Our goal was to learn how to make HMRs more effective by measuring clinicians’ self-reported use of HMRs and attitudes toward an HMR system embedded in an electronic medical record.

*Methods:* We surveyed 43 clinicians using an electronic medical record with an automated HMR system that prompted the provision of preventive or screening interventions. We measured general attitudes toward computers and the HMR, attitudes toward health maintenance, reactions to key features of the HMR system, and use of information provided by the HMR system; and we asked open-ended responses on how to improve the system.

*Results:* Seventy-five percent of clinicians reported not observing or paying attention to the HMR flashing reminder icon when reviewing a chart, and 62.8% reported they either ignored or forgot to address an alert when it appeared. Only 20% reported regularly reviewing health maintenance needs of the patient before the clinical encounter, and 56% reported seldom or never acting on HMR information during an encounter that was not health maintenance.

*Conclusions:* This HMR system embedded in an electronic medical record was underused by clinicians, causing lost opportunities for provision of preventive care. As electronic medical records become more common, we need to find practical ways that are acceptable to clinicians to use the new capabilities the systems provide.

Physicians underuse health maintenance knowledge. Numerous preventive and screening interventions that are known and have proved to be effective, as well as lead to improved patient outcomes, are chronically underused.1,2 One approach to increasing appropriate use is through systems which remind or prompt a physician that a service is due for the patient being seen. Automated health maintenance reminder systems aimed at increasing intervention use have been shown to improve relative rates, although absolute rates are still disappointingly low.3–6 Two investigators have reported an overall improvement of 13 percentage points,7 and 15 percentage points.5 Two other investigators have reported specific improvements for different interventions, ranging from 18 to 21 percentage points4 and from to 10 to 31 points.8 Some investigators have studied prompts for a single preventive intervention, such as cholesterol screening (improved 12 to 17 percentage points),9 or influenza immunizations (improved 20 percentage points).10

When applied to large populations, the outcomes from improvements of this magnitude could be significant. Other studies and two meta-analyses have also found improvement, although results were reported qualitatively or in terms of odds ratios, making determination of effect size difficult.3,6,11 Furthermore, reminders aimed specifically at clinicians who can offer the services have been shown more effective than those directed at reminding patients to avail themselves of the services.11,12 In summary, we know that reminders work, and we know that it works better to remind clinicians to offer these services. Little is known, however, about clinician acceptance of and reaction to such computerized reminder systems.

Electronic medical record systems offer a unique opportunity to improve use of appropriate screening and preventive services. As electronic medical records become more widespread, one approach to increasing use of health maintenance and prevention services is to incorporate an automated reminder system into the electronic medical record to prompt the clinician or patient when a preventive or health maintenance service is due. Electronic medical record systems can integrate detailed patient-level data on test ordering, results, patient history, progress notes, and demographics—providing a substrate of data that can be programmed using decision-support tools to indicate whether a patient is eligible and due for a screening or preventive intervention. Thus electronic medical records have the potential to provide an excellent foundation on which to add an automated health maintenance reminder (HMR) system. In fact, one Institute of Medicine rationale for recommending the universal use of such systems is based on the opportunity to provide better prevention and maintenance services.13

Underuse of preventive care continues, even in systems that use HMR systems. To begin to understand this problem and to establish ways to improve the effectiveness of such systems, we sought to determine clinicians’ use of and reactions toward an automated HMR system that is embedded in an electronic medical record. We hope that determining specific barriers will lead to wider and more consistent use of this emerging technology.

## Methods

### Setting

The study was performed in May 1999 at a not-for-profit, primary care network affiliated with the University of Washington Academic Medical Center. At that time the network consisted of nine clinics in the Puget Sound region of Washington and was known as the University of Washington Physicians Network (UWPN). The first UWPN clinic began operation in 1996, and other clinics were opened successively during the next 2 years. From the time these sites opened, each clinic and every care provider working in them used the comprehensive electronic medical record system EpicCare (Epic Systems Corporation, Madison, Wisc) for all patient encounters. EpicCare and its related modules integrate progress notes, patient history, laboratory results, ordering, billing, and scheduling. In addition, the program contains an automated HMR system that can be configured to prompt or remind the clinician to offer needed preventive and health maintenance services. Functionally, this prompt is a flashing reminder icon in the corner of the clinical encounter screen for a given patient.

UWPN has chosen to use this reminder function, in accordance with the evidence-based US Preventive Services Task Force (USPSTF) guidelines,14 for Papanicolaou smears, mammograms, adult and child immunizations, and tests for colorectal cancer and hyperlipidemia. Although the HMR content could not be customized for each provider, the provider group using the program was able to choose which reminders were seen, which protocols were used, and which patients were eligible for screening reminders. In addition, UWPN providers participated in the decisions to use the HMR system and the USPSTF guidelines and periodically reviewed and revised HMR content. To determine whether an eligible patient was due for a service, the clinical database was automatically queried by the HMR function, and, if recommended services had not been performed, the flashing icon appeared in the corner of the screen as a reminder for the provider.

### Participants

All 51 primary care providers currently in the clinic network were surveyed; mental health providers were not included. There were 24 family physicians, 14 internists, 7 pediatricians, and 6 midlevel providers (physician assistants and nurse practitioners). Because the same reminder methodology was used for all clinicians, all were given the same questionnaire. Not all the providers would use all the reminders. For example, the pediatricians might not use the Papanicolaou smear or mammogram reminders. By virtue of their employment, all clinicians were provided with their own desktop computer in the clinic, as well as with networked computers in each examination room and at the nursing stations, on which they used EpicCare and had Internet access and e-mail.

### Measurements

A 30-item written questionnaire was developed and administered to participating providers. Policies regarding the protection of human subjects at the University of Washington were followed, and the project received Institutional Review Board approval. The survey was designed in such a fashion that each question fell into one of the following four domains. (1) provider demographics, training, experience, and use of computers; (2) general attitudes about use of the electronic medical record (EpicCare); (3) general attitudes toward health maintenance and clinical guidelines; and (4) use of and reactions toward the use of the HMR function in the electronic medical record.

Taking advantage of the highly computerized nature of this clinic network, the questionnaire was administered through a unique worldwide Web-based program called U Wired Catalyst, available at the University of Washington.15 The questionnaire was created in the format of a Web page and could be answered on-line. The questionnaire did not undergo any special testing for validity or reliability, but it was reviewed by several faculty who had considerable experience in questionnaire development. Questions were administered and a 5-point Likert scale was used for the responses. Participants received an initial e-mail explaining the study, and 1 to 2 days later received another e-mail formally soliciting their participation in the survey. This latter message contained instructions on how to fill out the questionnaire, as well as a link to the Internet address of the survey Web page—participants needed only click on the address to go directly to the questionnaire. To generate high levels of participation, nonresponders received one follow-up e-mail solicitation 1 week after the initial request. Participants were also given the option of using a paper questionnaire, if desired (3 requested paper questionnaires). Questionnaires (either paper or electronic) were selected to accommodate both the culture (frequent use of electronic practice tools) and the decentralized nature of the group of clinicians.

## Results

All 51 providers of primary health care were invited to participate in the survey. The first solicitation resulted in a 41% response rate; after the second e-mail solicitation, total participants increased to 43 providers for a final response rate of 84%. Responders were 23 family physicians, 9 internists, 6 pediatricians, and 5 midlevel clinicians (physician’s assistants and nurse practitioners). Three women and 5 men did not respond, a group comprising 3 internists, 3 family physicians, 1 pediatrician, and 1 midlevel clinician. Women made up 56% of survey respondents. The amount of time spent with the UWPN system reflected the newness of the clinic system: 33% had 2 years’ experience with UWPN, 44% between 1 and 2 years, and 23% had less than a year’s experience.

The general perception of using the electronic medical record (EpicCare) was positive, with 74% of the respondents reporting that they either liked or loved using it, and 16% reported they disliked using it. Although 79% of the responding providers believed they were either slightly or much less productive using the electronic medical record, compared with traditional paper documentation methods, 79% believed the quality of the medical record with the electronic medical record was at least somewhat better than traditional documentation. Furthermore, 49% of respondents agreed that the electronic medical record would allow them to provide better quality care.

Responding providers’ attitudes toward health maintenance and clinical guidelines were generally favorable, with 77% expressing some degree of favor toward using guidelines. Ninety-three percent believed that health maintenance was at least important in providing high-quality health care. Of this 93%, 23% believed health maintenance was of paramount importance for high-quality health care.

Regarding specific attitudes about the electronic medical record HMR system, survey results indicated that more than 86% or respondents disagreed or strongly disagreed that the HMR system intrudes on their decision-making autonomy, while 79% agreed or strongly agreed that the basic HMR recommendations (namely, the particular tests recommended, and the target populations and periodicity of those tests) were appropriate.

Table 1 details responses to several key survey questions. In characterizing the ability of the HMR function to attract the clinicians’ attention, 42% of the responding providers thought that the blinking icon was either too unobtrusive or unnoticeable. In describing whether they deliberately looked for an HMR system alert, only 7% of providers reported that they often or always deliberately looked for the HMR alert icon when they selected a patient on the scheduling screen. Fifty-one percent claimed to never look for such an alert. In characterizing their most common response to an HMR alert, more than one half of respondents stated that they ignored it, and only 16% attempted to address the health maintenance issue at the current clinical encounter.

View this table:
[Table 1.](http://www.jabfm.org/content/16/4/312/T1)

Table 1. 
Responses to Key Survey Questions Regarding Use of the Health Maintenance Reminder Function (HMR) in the Electronic Medical Record.

Incidental use of the HMR system (that is, the use of the system to remind a provider of a patient’s health maintenance needs when the patient is in for an unrelated problem) is described in Table 2. It appears that most (56%) providers seldom or never act on HMR alerts during unrelated visits, and only 21% always or often use the HMR system to review health maintenance needs before a encounter not related to health maintenance.

View this table:
[Table 2.](http://www.jabfm.org/content/16/4/312/T2)

Table 2. 
Incidental Use of the Health Maintenance Reminder (HMR) System.

Two open-ended questions were included in the questionnaire. In an informal review of the responses, several themes emerge. In response to, “What is most useful or helpful about the HMR function?” participants had the following recurring responses (number of responses follows in parentheses): 

1.  It is an easy reminder of needed interventions (19).

2.  It provides access to prevention guidelines (3).

3.  It is not helpful (3).

4.  It becomes background noise because too many patients have flashing reminders (2).

The question, “What would make the HMR function more useful/helpful?” generated the following responses (number of responses follows in parentheses): 

1.  Make it more user-friendly (more easily adaptable and tailored to specific patients, able to turn it on or off for a particular patient or a specific provider) (16).

2.  Use the HMR system to send reminders directly to patients (not just providers) (5).

3.  Make the alerts more noticeable (for example, use pop-ups on the screen) (5).

4.  Change the system so that support staff, not the clinician, do the work of maintaining and using the data (3).

## Discussion

The goal of this study was to gain insight into primary care clinicians’ use of and attitudes toward an automated HMR system embedded in an electronic medical record. Most clinicians reported that health maintenance and prevention guidelines are helpful in clinical care, implying that the group was positive in general about the idea of using guidelines in clinical care. Data from Tables 1 and 2, however, show that this automated HMR system was not used to its full potential. The HMR alert icon functions as the door into the reminder system itself; yet few clinicians regularly looked for the HMR alert, a substantial percentage indicated the alert icon was not sufficiently noticeable, and a majority commonly ignored an alert when they did notice it. Coupling these observations, it seems that, for one reason or another, many of these primary care clinicians did not make use of the HMR system—they did not open the door to the system. In this context, it is not surprising that few providers capitalized on incidental opportunities to provide needed preventive and screening interventions.

Some explanation for these self-reported behaviors might be found in the responses to the open-ended questions: (1) many providers apparently felt that the HMR system was not easy to use, was difficult to tailor to special needs, and would have benefited from being more user-friendly; (2) some would have preferred a more interactive (and more obtrusive) pop-up design; and (3) some would have delegated the administrative burden of tracking health maintenance to support staff. Others would have used the system to send reminders or recalls automatically to appropriate patients.

This study has several limitations. First, we studied a single EMR system, even though it is one of the most widely used systems in the United States. Thus, to the extent that the findings of this survey were dictated by the particular design of this specific HMR system, there could be limited generalizability of results. Even so, some findings, especially regarding incidental use of the system, might apply to different designs. Second, the UWPN clinic system was relatively new, and many providers were new to an electronic medical record system such as EpicCare. Given more time to adapt to such a system, it is possible more providers would make better use of features such as an HMR system. It must be noted, however, that most providers (88%) had at least 6 months’ experience with the system; it is doubtful that after this time users would be greatly improving their proficiency with EpicCare. Lastly, we studied self-reported behaviors, which might be different from the clinicians’ actual behaviors.

In conclusion, these clinicians believed in the importance of health maintenance and preventive care. They generally liked their computerized medical record system despite its limitations, and they liked the idea of having an automated reminder of when and for whom to provide needed health maintenance services. Unfortunately, their use of this system was less than optimal. Thus, there continue to be numerous lost opportunities to provide needed and beneficial preventive and screening care. Because improvement in the low rates of provision of such services is an ongoing quality improvement concern for governments and health plans, future studies should focus on why providers ignore HMRs, even when they are philosophically supportive of them. The results of such research might make HMR systems, especially those embedded in electronic medical records, more easily used and better accepted by clinicians.

## Acknowledgments

The authors would like to thank University of Washington Physicians Network for its help and cooperation in designing and executing this study.

## Notes

*   The Robert Wood Johnson Foundation Clinical Scholars Program supported Dr. Schellhase in this work.

*   Received for publication November 26, 2002.
*   Revision received November 26, 2002.

## References

1.  Paskett ED, McMahon K, Tatum C, et al. Clinic-based interventions to promote breast and cervical cancer screening. Prev Med 1998; 27: 120–8.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1006/pmed.1997.0254&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=9465362&link_type=MED&atom=%2Fjabfp%2F16%2F4%2F312.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000071888300015&link_type=ISI) 

2.  Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States? Milbank Q 1998; 76: 517–63, 509.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1111/1468-0009.00105&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=9879302&link_type=MED&atom=%2Fjabfp%2F16%2F4%2F312.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000077653800002&link_type=ISI) 

3.  Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 1998; 280: 1339–46.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1001/jama.280.15.1339&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=9794315&link_type=MED&atom=%2Fjabfp%2F16%2F4%2F312.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000076472500028&link_type=ISI) 

4.  Chambers CV, Balaban DJ, Carlson BL, Gasberger DM. The effect of microcomputer-generated reminders on influenza vaccination rates in a university-based family practice center. J Am Board Fam Pract 1991; 4: 19–26.
    
    

5.  Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW. Computer-based vs manual health maintenance tracking. A controlled trial. Arch Fam Med 1994; 3: 581–8.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1001/archfami.1994.01850200023011&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=7921293&link_type=MED&atom=%2Fjabfp%2F16%2F4%2F312.atom) 

6.  Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc 1996; 3: 399–409.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiamFtaW5mbyI7czo1OiJyZXNpZCI7czo3OiIzLzYvMzk5IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8xNi80LzMxMi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

7.  Balas EA, Weingarten S, Garb CT, Blumenthal D, Boren SA, Brown GD. Improving preventive care by prompting physicians. Arch Intern Med 2000; 160: 301–8.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1001/archinte.160.3.301&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=10668831&link_type=MED&atom=%2Fjabfp%2F16%2F4%2F312.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000085134000004&link_type=ISI) 

8.  McPhee SJ, Bird JA, Fordham D, Rodnick JE, Osborn EH. Promoting cancer prevention activities by primary care physicians. Results of a randomized, controlled trial. JAMA 1991; 266: 538–44.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1001/jama.1991.03470040102030&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=2061981&link_type=MED&atom=%2Fjabfp%2F16%2F4%2F312.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=A1991FX02100028&link_type=ISI) 

9.  Khoury AT, Wan GJ, Niedermaier ON, et al. Improved cholesterol management in coronary heart disease patients enrolled in an HMO. J Health Qual 2001; 23(2): 29–33.
    
    

10. Kleschen MZ, Holbrook J, Rothbaum AK, Stringer RA, McInerney MJ, Helgerson SD. Improving the pneumococcal immunization rate for patients with diabetes in a managed care population: a simple intervention with a rapid effect. Jt Comm J Qual Improv 2000; 26: 538–46.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=10983294&link_type=MED&atom=%2Fjabfp%2F16%2F4%2F312.atom) 

11. Turner BJ, Day SC, Borenstein B. A controlled trial to improve delivery of preventive care: physician or patient reminders? J Gen Intern Med 1989; 4: 403–9.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1007/BF02599691&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=2677271&link_type=MED&atom=%2Fjabfp%2F16%2F4%2F312.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=A1989AR19000007&link_type=ISI) 

12. Simon MS, Gimotty PA, Moncrease A, Dews P, Burack RC. The effect of patient reminders on the use of screening mammography in an urban health department primary care setting. Breast Cancer Res Treat 2001; 65: 63–70.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1023/A:1006410711370&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=11245341&link_type=MED&atom=%2Fjabfp%2F16%2F4%2F312.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000166616600008&link_type=ISI) 

13. Institute of Medicine, Dick RS, Steen EB, Detmer DE, editors. The computer-based patient record: an essential technology for health care. 2nd ed. Committee on Improving the Patient Record, Division of Health Care Services. Washington, DC: National Academy Press, 1997.
    
    

14. Guide to clinical preventive services: report of the US Preventive Services Task Force. Alexandria, Va: International Medical Publishing, 1996.
    
    

15. UWired Catalyst. Seattle: University of Washington, 1999.