Abstract
Objective: Most electronic health record (EHR) systems have the capability of generating a printed after-visit summary (AVS), but there has been little research on optimal content. We conducted a qualitative study and a randomized trial to understand the effect of AVS content on patient recall and satisfaction.
Methods: Adult primary care patients (n = 272) with at least 1 chronic condition were randomly assigned to 4 AVS content conditions: minimum, intermediate, maximum, or standard AVS. Demographics and health literacy were measured at an index clinic visit. Recall and satisfaction were measured by telephone 2 days and 2 to 3 weeks after the clinic visit.
Results: Average age was 52 years; 75% of patients were female, 61% were Hispanic, and 21% were African American, and 64% had adequate health literacy. Average medication recall accuracy was 53% at 2 days and 52% at 3 weeks, with no significant difference among groups at either time. Satisfaction with AVS content was high and did not differ among groups. Recall of specific content categories was low and unrelated to group assignment. Health literacy was unrelated to recall and satisfaction.
Conclusion: Primary care patients like to receive an AVS, but the amount of information included does not affect content recall or satisfaction with the information.
Supplying patients with printed instructions and information when they leave a medical encounter has become common practice. Before the availability of electronic health records (EHRs), patient educational materials often took the form of handouts and pamphlets and sometimes videos or interactive computer programs. Most EHRs enable clinicians to supply patients with individualized information in the form of an after visit summary (AVS) based on data available in patients' medical records. Recent legislation has all but required this feature to be included in the EHR for eligibility for financial incentives by meeting the Centers for Medicare and Medicaid Services' (CMS) Meaningful Use (MU) guidelines for the AVS.1 Core Measure 13 describes the objective to provide to patients clinical summaries about each office visit. According to §495.6(d)(13)(ii), the expectation is for, “Clinical summaries provided to patients for more than 50% of all office visits within 3 business days.”2
The minimum set of elements recommended by CMS to achieve stage 1 of MU includes patient name, provider name, date and location of visit, reason(s) for visit, vitals (temperature, blood pressure, height, weight, BMI, exercise status in minutes/week), problem list/current conditions, medication list, medication allergies, diagnostic test/laboratory results, and patient instructions.
Additional information may include referrals, topics covered during the visit, immunizations or medications administered during visit, next recommended appointment, other appointments/testing the patient needs to schedule, appointments/testing already scheduled, medication instructions, personalized instructions/notes, patient decision aids recommended, links to (or copies of) relevant educational information, care gaps, preventive screenings due, and a personalized message/closing.
The EHR-generated AVS represents a potentially efficient and effective tool to support a variety of objectives for optimal patient outcomes in primary care. The majority of adult patients are followed for chronic conditions, such as hypertension and type 2 diabetes, which require long-term adherence to self-management behaviors. The concept of patient activation as a central factor in effective chronic disease self-management underlies many current behavioral interventions to improve outcomes.3,4 In this conceptualization, the activated patient is provided with useful information about his or her condition(s), uses this information to undertake the recommended treatment plan, and engages with the provider in shared decision making to achieve desired health outcomes. The AVS can be viewed as a communication channel by which the provider can transmit and reinforce information provided during an encounter, thus supporting one of the foundations of patient activation, that is, accurate and current information about their condition and treatment plan. The AVS has the additional feature of allowing tailoring and personalization, which may be more effective than generic information in encouraging behavior change.5
Despite its potential to improve shared decision making, increase adherence to provider instructions, and improve coordination of care across health settings, we were unable to identify any previously published discussions of the theoretical rationale for a choice of AVS format or content or any observational or experimental studies testing the effects of providing an AVS and/or altering its content. A recent systematic review of the effects of EHRs in office settings does not mention the potential role of the AVS in supporting desired care outcomes.6 The extant literature on the AVS is limited to a study of patient attitudes about the use of the AVS in clinical settings.7 In an editorial on the rapid development and adoption of computerized clinical decision support systems deployed within EHRs, Garg and Tonelli8 note the gap between the availability of electronic innovations and any accompanying scientifically developed information in relation to their effects on patient outcomes and cost.
In 2008 clinicians in our setting were tasked with advising health system administrators on the configuration and features of the new EHR system. They identified as a research priority the need for evidence regarding the best way to format and utilize the AVS feature. We therefore undertook a 2-phase study to (1) understand through qualitative research the design features that patients and clinicians viewed as important; and (2) to test in an individual-level randomized trial whether various salient features of the AVS identified during the qualitative phase affected important patient outcomes, including recall of the general AVS content, medication recall, satisfaction with the AVS, patient uses of the AVS, and self-reported adherence to physician instructions.
Methods
Study Setting
The study was conducted in 4 clinics that are part of the Southern Primary-care Urban Research Network (SPUR-Net) practice-based research network, housed in the Department of Family and Community Medicine, Baylor College of Medicine. We selected the 4 clinics to provide a diverse sample consisting of privately and publicly insured patients, English and Spanish speakers, and a mix of African Americans, Hispanics, and non-Hispanic whites. When the study was initiated in 2009, the Epic EHR (Epic Systems Corp., Verona, WI) had been operational in the study practices for 1 to 4 years. Each Epic system, however, was configured differently based on specifications requested by the administrators and health care providers in the respective health systems. At the time, routine printing and delivery of an AVS to patients after their clinic visit was inconsistent across clinics.
Qualitative Phase: Patient and Provider Preferences Regarding AVS Content
To have a basis for determining which features of the AVS should be varied to meet patient and provider expectations and preferences within the constraints of the existing EHR, we conducted individual semistructured interviews with 12 physicians and 48 of their patients in the participating clinics. Patients were recruited to represent the demographics of each clinic and the racial/ethnic identity and language groups of patients who would be participating in the quantitative phase of the study. Interview questions explored physician and patient experiences, attitudes, preferences, and recommendations for content and format of an AVS. For example, we expected that patients would have preferences regarding the overall appearance and formatting of the AVS, both the volume and type of information it contained, and the readability of the AVS content.
Randomized Trial of Different AVS Versions
While developing the experimental versions of the AVS we were guided by a number of factors, including a priori hypotheses regarding features that could affect specific outcomes and the constraints imposed by the EHR data capture and retrieval systems. We hypothesized that the overall volume of information would affect both patient satisfaction with and recall of the information. We also hypothesized that patients with lower literacy levels and Spanish speakers would have difficulty retaining information from AVSs with a large volume of information, such as that required by MU criteria. At the time of the study, the Epic systems in use in the study settings did not have the capability of translating EHR data recorded in English into Spanish. In addition, because the AVS was designed to reflect data generated during the current visit, there were limitations on the ability to include historical data, such as previously obtained laboratory values. Since the qualitative phase indicated that the amount of information was a salient variable for both patients and physicians, we tested whether variations in the amount of information included in the AVS would affect the specified outcomes. One experimental version included all elements necessary to meet the CMS MU requirements. The remaining 2 versions contained reduced amounts of information. We included a fourth study group that received the usual care AVS already in place at the given clinic. The program code necessary to generate the different AVS versions was written by one of the authors (J.T.G.) and installed into the Epic systems in the 4 clinics. To protect patient safety and satisfy minimum Joint Commission standards, we could not eliminate the medications or patient instructions sections from any of the experimental versions. The content of each AVS version is shown in Table 1.
Patient Recruitment and Randomization
We used purposive recruitment in the 4 clinics to obtain prespecified numbers of English and Spanish speakers from each setting. This approach was designed to yield a sample of 68 participants in each of the 4 experimental AVS groups with approximately equal numbers of English- and Spanish-speaking patients randomized into each condition. The study had a 99% power to detect a small to moderate effect size (≥.25) using a 1-way, 4-group analysis of variance, with α set at 05. Eligibility criteria included age between 21 and 75 years, one previous visit to the clinic, and at least one chronic health problem requiring medication recorded at the previous visit. The requirement of at least one chronic health problem ensured the medication recall test included as an outcome would be valid for all participants. Patients were excluded if they did not have a phone number that was verified as working at the time of recruitment, if they expected to be out of town during the follow-up period, or were otherwise unable to complete the study measurements (eg, because of dementia or blindness). The project was reviewed and approved by the institutional review board of Baylor College of Medicine. Patients were recruited in the clinic before a physician visit. After consent they completed a brief interview and literacy assessment and were randomized to 1 of the 4 study groups. After the visit, the research coordinator printed the assigned AVS version and gave it to the patient.
Measures
We selected a limited number of outcome variables that seemed most important for understanding both the potential and the limits of alterations in the AVS content. The primary outcomes were patient satisfaction and recall of AVS content (both general and specific medication details). Patient-reported satisfaction with the amount of information received is a key element in widely used quality improvement surveys, such as the Consumer Assessment of Health care Providers and Systems surveys developed by Agency for Healthcare Research and Quality.9 Content recall reflects the extent to which the AVS has been effective as an information transfer tool. The patients' ability to absorb and retain visit-related information could be strongly influenced by the amount of information presented.
A secondary outcome was self-reported adherence to physician advice. Adherence is the principal target of care delivery innovations in primary care.10,11 We hypothesized that variations in satisfaction and/or recall could affect patients' willingness or ability to follow their doctor's treatment recommendations. Health literacy and language preference were measured as potential mediating variables of any effects on the primary outcomes. Finally, to aid in the interpretation of any direct or mediating effects observed and in understanding the potential for the AVS as an information transfer and self-management tool, we collected data on how patients used the AVS after the office visit.
The outcome measures were obtained by telephone interview in the patient's preferred language, which had been documented during the in-clinic interview. Two interviews were conducted 1 to 3 days and 14 to 21 days after the clinic visit to measure short-term and long-term recall and uses of the AVS. Participants were asked at the time of enrollment to provide convenient times for the phone interview, and multiple call-back attempts within the time window were made to achieve adequate data completeness. Interviews were conducted by a single research coordinator who was fluent in both English and Spanish.
Data obtained during the in-office interview included demographic information, previous experience with the AVS in that setting, and a health literacy assessment using the Short Test of Functional Health Literacy in Adults (S-TOFHLA).12 The S-TOFHLA is a well-validated timed reading comprehension test that measures patients' ability to read and understand passages describing common medical instructions. It is available in English and Spanish versions. Patients are categorized as having adequate, marginal, or inadequate health literacy. Outcome measures collected during the telephone interviews 2 days and 2 to 3 weeks after the clinic visit.
General AVS Content Recall
The interviewer asked patients to list the items covered in the AVS they received at their last visit. Answers were classified into the appropriate AVS category. For example, “my blood pressure” would be classified under “vital signs.”
Medication Recall
Patients were asked to generate the list of medications prescribed and their dosing schedules. The patients' answers were recorded verbatim and then scored based on criteria that awarded a point for the correct name, a point for the correct indication, and a point for the correct dosing schedule. The score was the percentage of medication information recalled correctly averaged across the number of medications prescribed.
Patient Satisfaction
Finding no published measure of satisfaction with the AVS or related tool, we elected to adapt a measure used previously by our group to evaluate patient responses to educational materials. The measure had been used by both English- and Spanish-speaking patients in our setting and displayed good reliability.13 Response options to our modified 9-item scale ranged from “strongly agree” to “strongly disagree.”
Self-Reported Adherence to Treatment
We chose a general measure of adherence that summarizes information about the patient's tendency to adhere to medical recommendations, regardless of type of treatment recommended.14 The items in this measure are:
I had a hard time doing what the doctor suggested I do.
I found it easy to do the things my doctor suggested I do.
I was unable to do what was necessary to follow my doctor's treatment plans.
I followed my doctor's suggestions exactly.
Generally speaking, how often during the past 4 weeks were you able to do what the doctor told you?
Response options for each item range from “none of the time” to “all the time.” This scale has good internal reliability (approaching 0.80) and acceptable stability over time. Standard back-translation procedures were used to construct an equivalent Spanish version.
Analysis
For the qualitative analysis, we used Nvivo-8 software (QSR International, Doncaster, Victoria, Australia) to analyze interview transcripts for major content domains and emergent themes. To analyze the clinical trial outcomes, we created an AVS content summary score by summing the number of content area categories each patient freely recalled. We also evaluated the recall of the content categories that were common to all the AVSs distributed across the 4 treatment arms. After verifying that the potential confounders were evenly distributed among the treatment groups, we used the χ2 test to assess differences in recall of AVS content categories. The medication recall scores and satisfaction scale scores were compared using 1-way analysis of variance. When the analysis of variance test was significant, we conducted post hoc pair-wise comparisons to identify the group means that were significantly different. In addition to the analysis of prespecified outcomes, we conducted exploratory analyses to determine whether recall and satisfaction were influenced by health literacy.
Results
Qualitative analysis of the patient interviews revealed that patients were generally satisfied with the AVSs they had received from their clinic in recent visits, and they did not place great importance on font style and other formatting features of the document. When asked how the document could be improved, they indicated a desire for specific, individualized explanations of their health problems and personalized health goals. They also expressed the desire for the information on medications and problems lists to be up to date and accurate. Spanish speakers wished to receive the information in Spanish, although most said they were able to obtain translations from family members.
Physicians viewed the AVS as a potentially useful tool for patient care and education, but they expressed concerns about the lack of flexibility in tailoring content to the patient's reading level and language preference (eg, no ability to provide AVSs in Spanish) and the amount of time required of them to keep problem and medication lists current. In contrast with the patient group, which generally preferred inclusion of more information, most physicians suggested keeping the AVS brief. The findings from the qualitative analysis supported our expectation that the amount of information contained in the AVS was an important variable.
Recruitment for the clinical trial began in December 2010 and was completed in January 2012. As shown in Figure 1, a total of 300 patients were screened, 272 were randomized, 272 completed the first follow-up call, and 212 completed the final follow-up call.
CONSORT diagram. AVS, after visit summary; F/UP, follow-up.
The characteristics of study participants by group assignment are reported in Table 2. Randomization was successful in evenly distributing potential confounders across the groups, and there was no need to conduct an adjusted analysis of the effect of AVS type on the study outcomes. Of the participants, 75% were female, 64% had adequate health literacy, and the average number of prescribed medications was 5.8.
Free recall of the information content of the AVS is reported in Table 3. There was no significant difference across groups in the probability of recalling information in content categories that were common to all versions of the AVS, although the P value for recall of diagnosis was .06. The only information category mentioned by more than 50% of patients was medications. When the number of categories recalled by patients was expressed as a proportion of the total number of categories on their version of the AVS, the highest percentage was in the group that received the shortest AVS version. However, the percentage of categories recalled in this group was relatively low (32%). Average medication recall scores did not differ across groups at either time point.
When patients were asked whether they liked receiving printed information when they left their clinic encounter, 70% agreed and 24% strongly agreed. Responses to the follow-up items about satisfaction with the version of the AVS received at the clinic encounter are shown in Table 4. Only one individual satisfaction item reached the significance level of .05: respondents who were assigned to the longest version of the AVS (AVS1) reported less agreement that the AVS gave them “the right amount of information.” The summed satisfaction scale scores did not differ significantly across groups. The mean adherence score was not significantly affected by the AVS group assignment (P = .135; detailed data available upon request).
The average adherence score for the groups combined was 3.35 ± 0.57, with no significant differences associated with group assignment (F3,197 = 1.875; P = .135). The individual group scores ranged from a low of 3.23 ± 0.53 for the AVS2 group to 3.46 ± 0.54 for the AVS3 group. Response options are scored on a scale of 1 to 4, with 4 indicating high adherence. Thus, the average score reflects a high level of adherence.
There were no significant differences in reported uses of the AVS across the groups. At the time of the in-clinic interview, 30% of participants indicated they planned to save the AVS for the next primary care appointment, and less than 1% said they planned to show it to a different provider. During the first telephone follow-up, 12.5% of participants reported they had read the AVS at least once, and 51% said they had filed it with their other health records. Less than 2% of participants said they discarded or lost it. The most common disposition of the AVS by the 44% of individuals who did not choose one of the standard answers on the checklist was to leave it in their car or have it somewhere in their home or office to be filed.
Health literacy was strongly associated with language preference. Of the English speakers, 82% had adequate health literacy, compared with 44% of Spanish speakers. Neither language preference nor health literacy was associated with medication recall scores during the first or second phone interviews (P = .709 and 0.644, respectively, for language comparisons; P = .537 and .804, respectively, for health literacy comparisons). However, both language preference and health literacy were associated with the satisfaction scale score (3.75 ± 0.39 in Spanish speakers vs. 4.01 ± 0.48 in English speakers [P < .001]; 3.68 ± 0.47 in the inadequate literacy group vs. 3.98 ± 0.45 in the adequate health literacy group [P < .001]).
Discussion
We conclude that primary care patients like to receive an AVS, but the amount and range of information included in the summary does not affect their recall of health-related visit information or their satisfaction with the amount of information received. Although in the qualitative phase patients expressed a desire to have as much information as possible in the AVS, the recall scores suggested that patients remember only a few categories of content. The most frequently recalled category of information was the medication list, followed by physician instructions. Since recall of these categories was independent of the total number of information categories included on their AVS, it seems that patients filter out all but the most salient information.
Variations in health literacy and native language are a concern for the effective transmission of health information in many settings in the United States. Our study setting provided us with the opportunity to explore the effect of these variables on outcomes associated with the AVS. Our finding that health literacy and language did not result in a disparity of medication recall contrasts with results previously obtained by Persell and colleagues,15,16 who found poorer recall of prescribed medications among English-speaking patients with inadequate health literacy scores on the S-TOFHLA. However, both English- and Spanish-speaking patients likely rely heavily on verbal communication and instructions given during the office visit, and this may particularly be true for patients with low health literacy.17 For example, patients with low health literacy often compensate with greater reliance on the physician for health information.18 As expected, health literacy levels were lower among Spanish-speaking patients in our study, but we did not find any significant difference in the recall of medications. Thus, a standard English-language AVS used for both English and Spanish speakers was not a barrier to patients acquiring information about their medications.
An important conclusion of our study is that the length of the AVS resulting from current CMS MU guidelines does not adversely affect patient recall or satisfaction when compared with versions with less information. On the other hand, much information probably is disregarded or not retained. The implication of these findings is that simply handing patients the AVS at the end of a visit cannot substitute for more concerted efforts to remind patients of important information. Recent research using brief simulated medical encounters indicates that patients have difficulty comprehending and retaining physician instructions and have the best recall of instructions received at the end of the visit.19
One promising approach that could be applied when the patient is provided the AVS is “closing the loop” by asking the patient to repeat back important elements of the information conveyed during the visit.19,20 A systematic review of interventions to enhance patient recall of health care instructions identified relatively few experimental studies of written materials and subsequent recall, and those reviewed did not consistently achieve positive results.21 However, checking the patient's recall of important information before the close of the visit is a practical, evidence-based method for enhancing patient recall after a medical consultation. Positive patient reactions to receiving an AVS, despite limitations of recall of specific content observed in this study and others, supports the AVS as an important component of patient-centered care.22
In addition to its role in providing patients with information to support self-management and adherence, the AVS could be viewed as a tool for the transfer of information among providers. The patients in our study did not indicate that they planned to show the AVS to a different doctor, but 30% said they planned to keep it for their next appointment. These responses could reflect the reliance of this particular study population on their regular primary care providers for most health problems.
Although our EHR system did provide a variety of tailoring features to vary the content and format of the AVS, we were unable to experimentally manipulate several features, including language, reading level, and certain categories of information, such as medications. We recognize that our inability to omit certain categories of information from AVSs generated in the context of an actual clinic visit could have resulted in AVSs that were not sufficiently different across the groups to affect the outcomes. However, these constraints are the same as those faced by the users of commercially available EHRs, and our results would have little generalizability if we had tested counterfactual conditions that would not occur in actual practice, such as an AVS that omits the current medications list or an EHR that automatically translates the AVS into another language.
A strength of our study is the randomized design that controlled for potentially important confounding factors, such as the primary care providers' interactions with patients regarding the AVS and patients' health literacy level. The study was adequately powered to detect a small to moderate effect on the primary endpoints. Thus, the lack of difference in the selected outcomes when the AVS content is varied within parameters that are reasonable in actual practice can be accepted with some confidence. There are certainly questions remaining about the effect of variables we did not vary experimentally, such as the content of the patient instructions section and other sections we could not omit, the mode of delivery, and specific interactions between the provider and the patient about the AVS content. However, the current MU criteria do not specify any features of the AVS other than a list of desired elements, and we felt it was important to focus initially on the question of optimal content.
Conclusion
Our study contributes information needed to guide primary care practices in selecting the content of the their AVS to meet MU guidelines. It is clear from our findings that the number of content areas currently required by the CMS guidelines does not interfere with patients' ability to extract relevant information. Primary care practices can implement the MU guidelines for AVS content without concerns that a lesser amount of information would lead to better outcomes.
Acknowledgments
The authors thank Lillian Carreon for her attention to detail and skill in interacting with patients and staff in diverse settings. Her participation was indispensible to the timely completion of the project. The authors also are grateful for the programming support provided by the IT staff of the Harris County Hospital District and the Baylor Clinic.
Notes
This article was externally peer reviewed.
Funding: This project was funded by the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services.
Conflict of interest: none declared.
Disclaimer: The opinions expressed in this document are those of the authors and do not reflect the official position of the AHRQ or the U.S. Department of Health and Human Services.
- Received for publication May 1, 2013.
- Revision received October 7, 2013.
- Accepted for publication October 14, 2013.
References
In this issue
Jump to section
Related Articles
Cited By...
- Natural Language Processing for Automated Annotation of Medication Mentions in Primary Care Visit Conversations
- Assessing the Quality of the After-Visit Summary (AVS) in a Primary-Care Clinic
- Does pictorial health information improve health behaviours and other outcomes? A systematic review protocol
- Opportunities to improve clinical summaries for patients at hospital discharge
- Investigating Patient-Centered Care