A Randomized Controlled Trial of Two Interventions to Improve Medication Reconciliation
=======================================================================================

* Caroline M. Wolff
* Amy S. Nowacki
* Jun-Yen Yeh
* John M. Hickner

## Abstract

*Objective:* Medication errors can be caused by lack of agreement between what physicians believe patients are taking and what patients actually take. There has been little systematic research to find the best way to reconcile medication lists in primary care. The objective of this study was to assess the impact of 2 interventions on agreement between electronic medical record medication lists and what patients report actually taking.

*Methods:* This study was a factorial randomized trial that randomized 440 eligible patients (English-speaking, age 18 and older, taking at least 2 prescriptions) visiting 20 primary care physicians; 367 completed the study. Interventions included (1) providing patients a printed copy of their current medication list at check-in and (2) beginning the medication review with an open-ended question. Patients were randomized to receive no intervention, one or the other intervention, or both interventions. The outcome measure was agreement on all prescription and nonprescription medications, vitamins, and supplements between the list from the electronic medical record after the visit and a list based on patient report generated during a phone interview within a week of the office visit.

*Results:* Agreement rates between medication lists and patient report for the 4 study groups were: 67.4% in the no intervention group, 66.7% in the printed list only group, 58.1% in the open-ended question only group, and 75.6% in the combined intervention group. Both a printed list and beginning a medication discussion with an open-ended question were required before any significant increase in agreement was observed.

*Conclusions:* While neither intervention alone improved medication list agreement, these interventions may have value in a multistep protocol to improve the agreement of medication lists in primary care offices. Baseline agreement was much higher than expected, possibly reflecting a Hawthorne effect.

*   Medication Reconciliation
*   Patient Safety
*   Practice-based Research
*   Primary Health Care

Medication errors and adverse drug events are common in outpatient settings. A meta-analysis of 29 studies found an incidence of about 15 adverse drug events per 100 outpatients per year, of which 20% were judged as being preventable.1 In a systematic review of 15 studies, Winterstein et al2 estimated that 4.3% of US hospital admissions are due to preventable drug-related causes. One source of outpatient medication error is discrepancies between what physicians prescribe and believe patients are taking, as recorded in patients' medication lists, and what patients actually take. Of the medication errors reported in one primary care study, 23% were inaccuracies in the medication list.3 Therefore, one way to reduce medication errors and adverse drug reactions is to carefully reconcile a patient's medication list in the medical record against what the patient actually takes.4 A Veterans Affairs study found that only 21% of medical records had accurate documentation of all drug names, dosages, and directions for use compared with a structured review by a pharmacist.5

There has been little systematic research, however, to identify the best approach for continually updating and maintaining an accurate outpatient medication list in primary care practices—the venue for most medication management of patients with chronic diseases who take multiple medications. As of December 2013, 4 interventional studies to improve medication reconciliation in ambulatory care have been published; none were randomized trials.

One was conducted in an oncology practice6 and the other 3 in primary care practice settings.7⇓–9 In the study by Varkey et al,7 patients were asked to bring their medication lists to their appointments to verify and correct the most recent list in the electronic medical record (EMR) with their physicians. In doing this, the proportion of medication discrepancies was reduced from 89% to 49%, and the proportion of medication lists with a discrepancy was reduced from 89% to 66%. In another study, receptionists asked patients to bring updated medication lists or medication bottles to appointments.8 During the appointments, patients were asked to write down their medications. This list then was recorded by a medical assistant (MA) and reconciled by the physician. The proportion of discrepancies did not change dramatically after the intervention, and the proportion of lists with discrepancies actually increased from 76% to 82%.

Primary care offices in PeaceHealth instituted a 5-step protocol involving all office staff and patients in the process9: (1) All patients were asked to bring in a current list of medications, including over-the-counter (OTC) drugs and herbal supplements. (2) Clinical staff (nurses and MAs) reviewed the medications with the patient at the start of the visit. (3) The patient's medication list was reconciled with the list in the clinic's EMR, and any changes were documented by the physician/provider. (4) Any newly prescribed medications were checked for interactions with the updated, reconciled medication list in the EMR by the physician/provider. (5) The patient left the office with a paper copy of an updated, reconciled medication list. Three months after this protocol was implemented, the percentage of lists containing discrepancies dropped from 80% to 50%. The researchers did not test the individual steps, so it is uncertain to what extent each step improved reconciliation.

A 2011 observational study of medication reconciliation in Cleveland Clinic primary care practices found that only 15.1% of medication lists were fully concordant for prescription medications, herbals, and other over the counter medications, and 29.8% for prescription medications alone, a rate similar to that of the Veterans Affairs and PeaceHealth systems before interventions.10

We wanted to discover whether discrete interventions improved the agreement between EMR and patient-reported medication lists. In the 2011 Cleveland Clinic observational study, using an open-ended question such as, “Tell me about your medications” was associated with a 20% absolute increase in agreement between the EMR medication list and patient report. The PeaceHealth study found that presenting the patient with a paper copy of their current EMR medication list at check-in for review before the encounter with the physician improved medication reconciliation.9 Therefore, we conducted this study to evaluate the effectiveness of 2 interventions on the agreement of EMR medication lists and patient report: (1) providing a paper copy of their EMR medication list to patients at check-in for review, and (2) having MAs use an open-ended question to introduce the medication review.

## Methods

### Site Selection and Training

We identified 2 Cleveland Clinic family health centers that did not provide printed medication lists to patients before visits. At these sites, standard policy was to have MAs review the medication list before the doctor's visit, but there was no formal medication reconciliation protocol. Each physician worked almost exclusively with a particular MA for the majority of his or her clinics. Ten such physician/MA pairs (MD/MA pairs) from each center agreed to participate in the study. Although both sites serve as teaching clinics for medical students and residents, none were precepting with the MD/MA pairs while this research was being conducted. The study began in October 2011 and concluded in April 2012.

Before patient enrollment with each MD/MA pair began, the study coordinator (CMW) instructed the MAs individually in the design, which required varying the approach to medication reconciliation based on the randomization of each patient. The list-oriented approach consisted of the MA stating, “Let's review your medication list” and then listing each medication already in the record, with the patient generally saying “yes” or “no”. For the open-ended question approach, the script began with the statement, “Tell me about your medications”, allowing the patient to lead the review of medications. If the MA felt the list had not been adequately reviewed, she would use follow-up statements/questions to complete the discussion, such as, “Have there been any changes or additions since your last visit here?” and “Let's review your medication list.”

### Patient Enrollment, Randomization, and Sample Size Calculation

Each morning, the study coordinator identified eligible patients from physicians' schedules. Eligibility criteria were age ≥18 years, English speaking, and having at least 2 prescription medications on the EMR list before the visit. When an eligible patient arrived, the coordinator would escort the patient to the examination room, where informed consent and contact information were obtained.

The study coordinator randomized each consenting patient to 1 of the 4 groups (Figure 1) using a random number generator. Randomization was performed using a block approach with the MD/MA pairs, so that each set of 4 patients was assigned to 1 of the 4 groups in a random order.

![Figure 1.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/27/3/347/F1.medium.gif)

[Figure 1.](http://www.jabfm.org/content/27/3/347/F1)

Figure 1. 
Subjects were randomized to 1 of 4 study groups: no intervention (group A), printed medication list only (group B), open-ended question only (group C), and both interventions (group D).

As noted in a previous observational study, the prevalence of agreement between medication lists and patient report in Cleveland Clinic primary care practices was approximately 20% before the study.10 We estimated that each intervention would increase the agreement between the medication list and patient report by 20%. To detect a 20% absolute difference from 20% to 40%, 82 patients per group were needed to achieve 80% power at a 2-tailed significance level of .05. This was extrapolated to a 4-group factorial design, resulting in a total of 328 observations required; the possibility of this being underpowered for smaller interaction effects was acknowledged. Initially, a goal of 400 patient observations was selected after inflating the sample size to account for dropout rates observed in the previous observational study,10 which used the same pharmacist-conducted telephone interview process. Twenty patients were enrolled with each of the first 10 MD/MA pairs. We had a higher than anticipated rate of loss to follow-up, so 24 patients were enrolled for each of the remaining 10 MD/MA pairs to achieve the desired sample size and adequate power.

### Intervention

If the patient was randomized to a group receiving a printed copy of the medication list, the study coordinator handed the patient a printed medication list and a pen and said: “This is a current list of your medications. Please read it carefully, and use this pen to note anything that doesn't look right to you. Then please mention the differences when your medications are reviewed during the visit so that we can update the list.”

Before leaving the room, the study coordinator turned on a digital recorder; then the coordinator attached the appropriate script card to the outside of the door. The MA read the card before entering the room and began the interview as prescribed by the card (open-ended or list-oriented script). After the medication review was complete, the MA turned off the recorder. The remainder of the visit was conducted routinely.

### Data Collection

When the encounter was complete, the study coordinator entered the EMR medication list generated at the end of the visit into the study database. The study coordinator also listened to each recording to determine whether the MA asked the proper question and the duration of the medication review.

To document medications patients report taking, a pharmacist conducted a detailed telephone interview with the patient and/or caretaker within a week of the office visit using a prepared script. The pharmacist recorded the exact name, dosage, route, and instructions for use, as read from the bottle label by the patient, and asked the patient how she or he actually takes the medication, thus generating a list based on patient self-report. Medications were categorized as prescription, OTC-meds, or OTC-other. The OTC-meds group included medications such as acetaminophen, ibuprofen, and aspirin at doses available OTC. The OTC-other group included nonmedication items such as vitamins, herbals, and other supplements available OTC.

The pharmacist noted any discrepancies between the EMR and the patient's/caretaker's self-report and asked about reasons for the discrepancies. When discrepancies were identified during the phone interview, the pharmacist sent a message through the EMR to the physician advising them of the discrepancy. The pharmacist also asked the patients several other follow-up questions and assessed their health literacy.11

Study data were collected and managed using REDCap (Research Electronic Data Capture) tools hosted at the Cleveland Clinic.12 REDCap is a secure, web-based application designed to support data capture for research studies; it provides (1) an intuitive interface for validated data entry; (2) audit trails for tracking data manipulation and export procedures; (3) automated export procedures for seamless data downloads to common statistical packages; and (4) procedures for importing data from external sources.

### Data Analysis

Medication lists were considered in full agreement when the EMR medication list and patient report agreed on the 5 following elements: (1) name of medication, (2) dose, (3) frequency of administration, (4) route of administration, and (5) PRN status. Agreement between the EMR medication list and patient report was assessed using 2 measures: (1) a dichotomous categorical variable (perfect agreement or not between the prescription medications on the medication list gathered through the phone interview and the medication list recorded in the EMR) and (2) a numeric variable (the proportion of individual medications that had at least one discrepancy between the medication list from the phone interview and the medication list in the EMR). Sample size was estimated based on the first criterion.

Descriptive statistics (means, standard deviations, counts, and proportions) were used to summarize all data. The primary outcome was full agreement between the medication list and patient report. To assess whether the proportion of medication lists in agreement with patient report differed among the 4 study groups, a mixed model was used to account for the hierarchical data structure (patients clustered within the MD/MA pair). An unadjusted analysis was performed, then the data were analyzed again, adjusting for potential confounders as necessary. Covariates achieving a *P* value <.2 in unadjusted analyses were carried forward for possible inclusion in the multivariable mixed model. A backward stepwise approach to variable selection determined the final model. The use of randomization helped evenly disperse factors related to MAs, physicians, and sites since randomization was blocked within MA/MD pairs. The data from the audio files was used when analyzing the implementation of the interventions.

## Results

Of the 440 patients randomized, 73 were lost to follow-up and 367 were included in analysis—an 83.4% follow-up rate. Figure 2 shows the consort diagram of enrollment. Follow-up rate and patient variables, including age, sex, number of medications on list, and health literacy, were well distributed in all 4 groups, suggesting successful randomization (Table 1).

![Figure 2.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/27/3/347/F2.medium.gif)

[Figure 2.](http://www.jabfm.org/content/27/3/347/F2)

Figure 2. 
Consort diagram describing the number of patients enrolled, randomized, and lost to follow-up and the number of patients completing the trial by study group.

View this table:
[Table 1.](http://www.jabfm.org/content/27/3/347/T1)

Table 1. Patient Demographics, Average Number of Medications Prescribed, and Literacy Levels of Patients in the 4 Study Groups

There were no significant differences among groups in the time taken for the medication review by the MAs (range, of 1.3–1.5 minutes; *P* = .75), and the open-ended question intervention was easily learned and performed by the MAs (range, 94–96% correct implementation). The mean duration of the medication reconciliation performed by the pharmacists by telephone after the visit was 10.3 minutes (range, 5–35 minutes).

The primary outcome was list agreement. The overall agreement rate was 67%; 121 of 367 lists contained at least one discrepancy. Considering only prescription medications, the overall agreement rate was 82%. The overall agreement rates in each group are given in Figure 3. Compared with no intervention (67.4%), there was almost no change in agreement when patients were given a printed list only (66.7%), a decrease in agreement when the medication reconciliation began with an open-ended question only (58.1%), and an increase in agreement in the combined intervention group (75.6%). Although not the primary outcome, the proportions of individual medications in agreement were as follows: 0.92 in the no intervention group, 0.88 in the open-ended question group, 0.92 in the printed list group, and 0.95 in the combined intervention group.

![Figure 3.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/27/3/347/F3.medium.gif)

[Figure 3.](http://www.jabfm.org/content/27/3/347/F3)

Figure 3. 
Unadjusted medication list agreement for the 4 study groups showing the synergy between medical assistants' use of an open-ended question and patients' review of a printed medication list before seeing the physician.

Figure 4 displays the agreement rate for all medications among the 20 MD/MA pairs, which shows considerable variation, from 33% to 91%. Therefore, clustering patients within MD/MA pairs had to be taken into account in the analysis. This also illustrates why it was important to randomize within MD/MA pairs in the study design: to minimize the effects of MD/MA pair variability.

![Figure 4.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/27/3/347/F4.medium.gif)

[Figure 4.](http://www.jabfm.org/content/27/3/347/F4)

Figure 4. 
Variation in agreement rates of the 20 physician/medical assistant (MD/MA) pairs.

A significant interaction (*P* = .04) was identified, indicating that both a printed list and beginning with an open-ended question are required before any increase in agreement takes place. This can be seen visually in Figure 3, where the proportion of lists in full agreement is less than the control group for each intervention but is greater when both interventions are implemented together.

Adjusting for demographic factors (patient age, type of visit, total number of medications, and site) and taking into account the clustering of patients within each MD/MA pair, the interaction remained significant (*P* = .01). This means that the effect of each intervention on list agreement depends on the status of the other intervention. When the reconciliation does not begin with an open-ended question, agreement is not associated with whether the medication list is provided (odds ratio, 0.95; 95% confidence interval, 0.56–1.63). When the reconciliation does begin with an open-ended question, however, the probability of agreement increases when the medication list is provided (odds ratio, 2.70; 95% confidence interval, 1.73–4.22). This can be seen visually in Figure 3; with no open-ended question (dashed line) there is no impact of the printed list; with the open-ended question (solid line); there is, however, an increase in agreement with the printed list (positive slope).

The data can be further explored with regard to the types of discrepancies that occurred in each group. The categories were categorized broadly as 1 of 3 types: medications being taken but missing from list, medications on lists that are not being taken, and medications being taken incorrectly (Figure 5).

![Figure 5.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/27/3/347/F5.medium.gif)

[Figure 5.](http://www.jabfm.org/content/27/3/347/F5)

Figure 5. 
Number and types of discrepancies by study group.

## Discussion

Medication reconciliation is an important element of medication safety. There has been little systematic research to find the best way to maintain accurate outpatient medication lists. Presented here are the results of the first randomized trial to improve the agreement of medication lists in outpatient care.

In this study, neither intervention alone improved the agreement between patient-reported and EMR medication lists. There was, however, a significant interaction that persisted even after adjusting for potential confounders. This means that the effect of the printed list on agreement depends on whether an open-ended question was used and vice versa. The significant interaction of the 3 interventions suggests that efforts toward improving medication list agreement are more likely to succeed when they use a multistep approach. Indeed, this was demonstrated by the PeaceHealth improvement project.9

In this study, the combined intervention group increased the agreement rate by approximately 10% compared with the no intervention group. We believe that an increase of this magnitude, especially in the setting of an already high rate of agreement at baseline, should be considered clinically important. We suggest, therefore, that practices use a multistep intervention model for improvements in the medication reconciliation process.

The high baseline agreement and the failure of the single interventions to improve agreement demand an explanation. During the consent process, patients were told that the purpose of the study was to learn more about the way medication lists are updated. The MAs and physicians knew that they were being studied, which may have led them to increase their efforts during the medication list update. This is known as the Hawthorne effect13: a psychological response in which subjects in a research study change their behavior simply because they are subjects in a study, not because of the research treatment. Thus, the heightened awareness of the patients, MAs, and physicians about the significance of the medication review could have increased the baseline agreement rate. This effect, however, would affect all 4 intervention groups equally and thus would explain only the high overall baseline agreement rate, not the group differences observed.

During the study the Cleveland Clinic health system began a system-wide effort to improve medication reconciliation, although there had been no dissemination of educational materials to physicians or support staff before the conclusion of this study. However, it is possible that the MAs and physicians were aware of an increased focus on medication reconciliation and that this could have affected their reconciliation practices, again possibly increasing the baseline agreement rate.

Also, new prescribing technology was implemented just before the trial started; the technology automates certain features of prescribing. For example, one cannot prescribe a medication without entering the route of administration. These minor improvements to the electronic tools used for prescribing may have increased the completeness of the items listed in the EMR. However, in the observational study conducted at the Cleveland Clinic 1 year before this randomized controlled trial, only 5% (11 of 212) of the total discrepancies identified were due to differences in route of administration. This particular change, therefore, would not completely explain the high rate of agreement at baseline.

We expected that the open-ended question approach and the paper medication list would improve agreement between the EMR lists and patient report since they were highly correlated with medication list agreement in the prior observational study. Open-ended questions and statements can be powerful in clinical practice because patients may feel empowered to provide more information than they might otherwise share. However, if patients are not prepared to review their medications or do not know their medications well, the open-ended approach may not be used to its full advantage. When a list is provided but the communication approach is closed, patients may assume that the medications already listed are the only ones that should be included and that the information listed is more accurate than their memory. Patients may also hesitate to ask for or provide clarification when not prompted. More exploration into the types of discrepancies that occurred with each approach may provide insight into the causes of failure of each solo intervention.

The duration of the medication reconciliation during the office visit lasted only 1.3 to 1.5 minutes, without significant variation between intervention and no intervention groups. This means that the interventions had a minimal effect on the duration of the office visit, which is essential in a busy primary care setting. In contrast, the pharmacist-led reconciliation by phone lasted an average of 10.3 minutes. While such an in-depth review is ideal for medication reconciliation, such a time-intensive intervention is not feasible in most primary care practices on a routine basis.

There are some limitations to this study. First, as discussed, we suspect the Hawthorne effect was at play. Second, only one element of medication management is addressed by this study, and the lack of agreement between medication lists and patient report accounts for only about a quarter of medication errors in office practice. Third, because we only studied 2 practices, the results are not necessarily generalizable to all primary care sites. Fourth, there was substantial variation in list agreement by physician, suggesting that individual effort has a large effect on the accuracy of medication lists. Finally, one could argue that the MD/MA pair should have been randomized to preserve the accuracy of the intervention and maximize intervention effects. However, because we had noted in our prior study great variation in medication reconciliation performance among MD/MA pairs, a large number of MD/MA pairs would have been required to overcome this variability and obtain valid estimates, which was not feasible given our resources for the study. Therefore, we chose to randomize each patient, implement objective interventions (providing a list and scripted dialogue), and adjust for patient clustering within MD/MA pair in the statistical analyses, resulting in slightly attenuated but valid estimates.

## Conclusion

This study did not support the use of these 2 methods as single, discrete interventions to improve medication reconciliation in primary care. However, our results suggest that these interventions may have value as part of a more comprehensive process aimed at improving the agreement of medication lists with patient report. Therefore, we believe future efforts to improve medication reconciliation in primary care should focus on multistep protocols.

## Acknowledgments

The authors thank Kim Begany and Shannon Buie, the pharmacists who performed all follow-up phone interviews; Lisa Potts, PharmD, and Cari Cristiani, PharmD, who contributed to the training of pharmacists and developed the protocol for the phone interviews; Dr. Alexis Reedy, who completed the observational study on which this trial was based and served as consultant. The authors also thank the medical assistants and physicians of the Solon and Strongsville CCF Family Health Centers who participated in this study and played an integral role in the interventions; Kelly Nottingham, who provided research administration support; Laurie Evans, who provided administrative support; and Drs. David Van Duin and Robert Jones, the study review committee members. The authors thank the Cardinal Health Foundation for the grant that supported expenses related to presenting work at conferences and refreshments provided at training sessions, as well as the Cleveland Clinic Medicine Institute, which provided digital recorders and a secure space in which to store study-related materials.

## Notes

*   This article was externally peer reviewed.

*   *Funding:* Support was received from the Cardinal Health Foundation's E3 Grant Program (grant no. 35800, awarded to J-YY).

*   *Conflict of interest:* none declared.

*   Received for publication August 19, 2013.
*   Revision received January 12, 2014.
*   Accepted for publication January 27, 2014.

## References

1.  1. Thomsen LA, Winterstein AG, Søndergaard B, Haugbølle LS, Melander A. Systematic review of the incidence and characteristics of preventable adverse drug events in ambulatory care. Ann Pharmacother 2007;41:1411–26.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3Bhb3AiO3M6NToicmVzaWQiO3M6OToiNDEvOS8xNDExIjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8yNy8zLzM0Ny5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

2.  2. Winterstein AG, Sauer BC, Hepler CD, Poole C. Preventable drug related hospital admissions. Ann Pharmacother 2002;36:1238–48.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3Bhb3AiO3M6NToicmVzaWQiO3M6MTE6IjM2LzctOC8xMjM4IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8yNy8zLzM0Ny5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

3.  3. Hickner J, Zafar A, Kuo GM, et al. Field test results of a new ambulatory care Medication Error and Adverse Drug Event Reporting System—MEADERS. Ann Fam Med 2010;8:517–25.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6NzoiOC82LzUxNyI7czo0OiJhdG9tIjtzOjIwOiIvamFiZnAvMjcvMy8zNDcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

4.  4. Gandhi TK, Weingart S, Borus J, et al. Adverse drug events in ambulatory care. N Engl J Med 2003;348:1556–64.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1056/NEJMsa020703&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=12700376&link_type=MED&atom=%2Fjabfp%2F27%2F3%2F347.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000182248900007&link_type=ISI) 

5.  5. Pronovost P, Weast B, Schwartz M, et al. Medication reconciliation: a practice tool to reduce the risk of medication errors. J Crit Care 2003;18:201–5.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/j.jcrc.2003.10.001&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=14691892&link_type=MED&atom=%2Fjabfp%2F27%2F3%2F347.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000187717500001&link_type=ISI) 

6.  6. Weingart SN, Cleary A, Seger A, et al. Medication reconciliation in ambulatory oncology. Jt Comm J Qual Patient Saf 2007;33:750–7.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=18200900&link_type=MED&atom=%2Fjabfp%2F27%2F3%2F347.atom) 

7.  7. Varkey P, Cunningham J, Bisping D. Improving medication reconciliation in the outpatient setting. Jt Comm J Qual Patient Saf 2007;33:286–92.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=17503684&link_type=MED&atom=%2Fjabfp%2F27%2F3%2F347.atom) 

8.  8. Nassaralla C, Naessens J, Chaudhry R, Hansen M, Scheitel S. Implementation of a medication reconciliation process in an ambulatory internal medicine clinic. Qual Saf Health Care 2007;16:90–4.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjc6IjE2LzIvOTAiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI3LzMvMzQ3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

9.  9. Stock R, Scott J, Gurtel S. Using an electronic prescribing system to ensure accurate medication lists in a large multidisciplinary medical group. Jt Comm J Qual Patient Saf 2009;35:271–7.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=19480381&link_type=MED&atom=%2Fjabfp%2F27%2F3%2F347.atom) 

10. 10. Reedy A, Yeh J, Nowacki A, Hickner J. Identifying patient, physician, medical assistant, and office visit factors associated with medication list agreement. J Patient Saf. In press.
    
    

11. 11. Chew L, Griffin J, Partin M, Noorbaloochi S, Grill J. Validation of screening questions for limited health literacy in a large VA outpatient population. J Gen Intern Med 2008;23:561–6.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1007/s11606-008-0520-5&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=18335281&link_type=MED&atom=%2Fjabfp%2F27%2F3%2F347.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000255195300008&link_type=ISI) 

12. 12. Harris P, Taylor R, Thielke R, Payne J, Gonzalez N, Conde J. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/j.jbi.2008.08.010&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=18929686&link_type=MED&atom=%2Fjabfp%2F27%2F3%2F347.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000264958800018&link_type=ISI) 

13. 13. Landsberger H. Hawthorne revisited: management and the worker, its critics, and developments in human relations in industry. Ithaca, NY: Cornell University; 1958.