INTRODUCTION

Engaged patients have better health outcomes,1 and clinicians strive to involve patients more actively in their care. Today, many patients access their health information via secure, electronic patient portals.2 Increasingly, this access includes the narrative visit notes clinicians prepare, referred to as open notes.3

Patients report many benefits from reading open notes, including feeling more in control of their care, helping them adhere to their medication regimens, and enabling them to understand their health conditions and care plans better.4,5,6,7,8,9 However, we know little of their perspectives on notes written by different clinicians in different specialties, or of their thoughts on how notes might be changed to meet their needs more effectively.

As patients have become more experienced, we sought to understand their perspectives about reading a note from a single encounter with a clinician from any specialty. We wanted to understand their views on its clarity, accuracy, and completeness, and their suggestions for improving the note. Based on our previous work, we hypothesized that patients with less education would report more difficulty understanding the notes, especially those written by physicians compared with those written by other health professionals.4, 5 We anticipated that notes prepared by primary care clinicians would be understood more readily than those written by specialists. We expected that about a quarter of patients would find the note inaccurate, and that about a tenth would report important information missing.5, 10 And we hypothesized that patients’ perspectives about their notes would be associated with their willingness to recommend their clinicians to others.

METHODS

We surveyed adult outpatients about reading their clinicians’ visit notes through online portals affiliated with 3 US health systems: Beth Israel Deaconess Medical Center (BIDMC) in Boston, Geisinger health system in Pennsylvania, and University of Washington Medicine (UW) in Seattle. We conducted the survey online, June–October 2017, with patients ≥ 18 years of age who had ≥ 1 office visit and had logged into the patient portal at least once in the previous 12 months. To enhance our response rate, we sent patients reminder messages and offered financial incentives through a lottery.11 Of 136,815 invited to participate, 29,656 (21.68%) responded; 22,947 were patients who had opened at least 1 note in the previous 12 months, verified by portal tracking data. Knowing that patients sometimes confuse notes with the portal itself, with other parts of the record, or with an “after visit summary,” we described notes in both the survey invitation and in a question, and we showed a screenshot of the location of notes on their portal. Detailed survey methods have been published.5

We considered asking patients to refer to the most recent note but decided instead to have them choose a note that they recalled: “Please think about one visit note written by a doctor, nurse practitioner, physician assistant, or mental health professional.” We asked them to consider notes by these clinicians because we wanted to know about notes that were likely part of an ongoing relationship, rather than a single visit encounter. We asked patients to identify the type of clinician who wrote the note they would be describing. At BIDMC and Geisinger, patients reported the specialty of the clinical office for the note they selected; UW patients were not asked this question.

We asked 4 closed-ended questions: (1) Who wrote the note (type of clinician, do not know)? (2) How much of the note did you understand (very little, some, almost everything, everything)? (3) The note accurately described the visit (agree, somewhat agree, somewhat disagree, disagree, do not know/not sure), and (4) Was anything important missing from the note (yes, no, do not know/not sure)? If patients indicated they did not understand much of the note, we asked where they looked for help. We asked them to write free text describing suggestions about how the note could have been better. Finally, we asked whether they would recommend the clinician to other family members or friends. The survey instrument is available on request.

Data Analysis

We used descriptive statistics to summarize responses and chi-square tests to evaluate differences between groups. In the clinician type analysis, we excluded notes by mental health professionals because they comprised < 1% of responses. We limited the analysis of differences among primary care, combined specialties, and individual specialty practices to categories with ≥ 150 observations. Finally, we examined patients’ evaluations of their notes by whether they would recommend their clinicians to others.

We previously compared respondents with non-respondents according to sociodemographic characteristics (age, sex, race, and insurance status).5 In this study, we also used diagnosis codes from administrative data to compare them according to the Charlson Comorbidity Index,12 and prevalence of 2 major chronic diseases: coronary heart disease13 (ICD-10 codes I20–I25) and diabetes14 (types 1 and 2, ICD-10 codes E10–E14).

We completed a thematic analysis to evaluate 4562 free-text responses to the question, “Do you have suggestions about how this visit note could have been better?”15 In 4 iterative steps, we defined, revised, and finalized 9 thematic and 2 “other” coding categories, and we decided to exclude responses with fewer than 5 words. Two coders then reviewed and coded the remaining responses, assigning up to 3 codes per response. We calculated the Cohen’s kappa statistic and the proportion of responses in which there was complete agreement. After analysis of the agreement rates, we decided to present responses in which there was complete agreement. Finally, the 2 coders reviewed the comments in each category to select representative examples.

RESULTS

Quantitative Analysis

A total of 21,664 patients (the 94% of patient respondents who read at least 1 note) reported on their experiences in reading individual notes; 63% were women, 75% were aged 45 years or older, and 85% were white. The majority (72%) had completed college, 57% reported being employed, and 85% reported good or excellent health. Compared with non-respondents, respondents had higher scores on the Charlson Index, with 24.6% having scores of 1 or higher versus 18.1% among non-respondents, and higher prevalence of diabetes (7.7% vs 6.2%) and coronary heart disease (3.5% vs 2.2%).

Overall, 96% of patients, and at least 91% of patients in all demographic and health categories, reported they understood all or nearly all of their self-selected notes (Table 1). We observed a 5% difference in proportions who understood their notes among those who completed college, compared to those with high school or less education (97% and 92%, respectively). Most respondents (93%) found that the note accurately described the visit, and few, 6% overall, reported something important to them missing from the note. Patients reporting fair or poor health were least likely to agree that the note was accurate (88.6% vs 94.4% of those in better health) and most likely to report something important was missing (10.5% vs 4.9% of those in better health).

Table 1 Responses from 21,664 Patients About an Individual Note, According to Patient Characteristics
Full size table

We found no material differences in the proportions of patients who understood the note according to type of clinician, with 97% reporting they understood all or most of notes written by physicians or NPs, and 96% reported understanding notes written by PAs (Table 2). Among the 4% of patients who reported understanding some or little of the note, most looked for help: 35% searched the internet, 27% asked a clinician, 7% asked a friend or family member, and 27% reported not looking for help (data not shown).

Table 2 Patient Responses About a Note Authored by Different Types of Clinicians and in Offices of Different Specialties
Full size table

Referring to notes written by physicians, 95% of patients reported that the note accurately described the visit, compared with 92% of notes by NPs, and 90% by PAs. Fewer than 10% of patients overall reported something important to them missing from the note: 5% of physicians’ notes, 6% of NPs’ note, and 9% of PAs’ notes. Nearly all patients (93%) who reported on notes by physicians said they would definitely or probably recommend the clinician to family or friends, compared with 89% of NPs and 85% of PAs.

There were few differences according to clinician specialty (Table 2). Ninety-seven percent of patients reporting on primary care notes, and 94% of those reporting on specialty notes, understood all or most of the notes, and 89% to 96% felt they were accurate. Across specialties, 3–9% reported something important missing from the note. In the different specialties, 88–96% of patients would recommend the clinician to others.

We examined the patients’ likelihood for recommending the authoring clinician to family or friends according to how they responded to the 3 questions about note content (Fig. 1). Among patients who understood their note, 93% would recommend their clinician to others, but this proportion dropped to 77% among those who understood some or very little of their note. While only 4% of patients reported that the note did not accurately describe the visit, they were much less likely to recommend their provider to others, compared with those finding the note accurate (43% and 95%, respectively). Similarly, patients who found something important missing were less likely to recommend their provider, compared with those not finding anything important missing (54% and 96%, respectively).

Figure 1
figure 1

Willingness to recommend provider related to perceptions of visit note.

Full size image

Qualitative Analysis

When asked how the note could have been better, 4562 (21.1%) respondents offered comments. After excluding 750 responses containing fewer than 5 words, we coded the remaining 3812 comments. The 2 coders had fair to good agreement based on all permutations of 11 categories, with kappa = 0.56,16 and they had full agreement on up to 3 coded categories in 2376 responses (62.3%) reported here (Table 3).

Table 3 Examples from 2376 Patient Responses According to 9 Themes: “Do you have suggestions about how this visit note could have been better?”
Full size table

The most common suggestions related to the structure and content of notes. Patients wanted new information prominently featured at the top of the note, including clear instructions about next steps, instructions about referrals, and explanations of test results. They felt that notes were not always sufficiently personalized or detailed, which they often attributed to copying old information and using templates. Comments about use of medical jargon were almost as frequent, with many respondents suggesting technical solutions such as links to glossaries.

Patients also wrote about the accuracy and completeness of notes, particularly mistakes in documenting the visit discussion and outdated patient/family history. Some felt mistakes reflected the rushed nature of visits, the computer’s demand for the clinician’s attention, use of templates, and delays in writing notes. They expressed appreciation for access to notes and being able to identify mistakes. Some suggested new portal functionality that would allow patients to edit or point out needed corrections. They perceived a negative tone if they felt their concerns were unjustifiably dismissed, their perspectives not represented, or read terms such as “patient denies” and “obese.”

They had ideas for improving patient portals. Many cited multiple steps required to find notes and suggested they be prominent on the homepage or visit summary. They suggested redesigning and updating the entire portal, improving mobile functionality, and adding print capability. Some commented on larger issues that might be addressed with new functions in electronic records, such as struggling with decentralization of their information when they see multiple providers, and expressing different, patient-specific needs for privacy (e.g., not wanting all providers to see the entire record, or not needing a secure login to read notes). Some requested avenues for patient input during or after the visit, using portal feedback mechanisms for comments.

Finally, patients expressed difficulties with various aspects of process and communication during visits. Some felt notes were not posted quickly enough following visits. Some described feeling rushed during visits; lack of attentiveness by clinicians busily typing during the encounter; and suboptimal follow-up, referrals, and communication among their clinicians.

An additional 1133 comments coded as “other” consisted almost entirely of praise for open notes (n = 812) and for clinicians or visit experiences (n = 316).

DISCUSSION

In this study, patients overall give high marks both to open notes and to those preparing them. Nearly all reported understanding everything or almost everything in a clinician’s visit note. Contrary to our expectations, we found only a small difference in the proportions of patients who understood notes written by a primary care clinician or a specialist, and almost no differences according to the type of health professional (MD, NP, or PA). The patients’ reports also pointed to how resourceful they are in finding answers to questions arising from notes without calling on their clinicians for explanations. Practitioners concerned that patients will not understand notes and will contact their practices for clarifications14 should find these results reassuring. They may serve also as a partial explanation for a phenomenon experienced almost invariably by practices implementing open notes: While patients report striking benefits, clinicians experience little perturbation in their practice and workflow.

In earlier work on patient-reported benefits of note reading, we found that Blacks and Hispanics, older patients, and those with fewer years of education were more likely to report benefits from note reading than others, and that non-English speakers and those with less education reported greater benefits to medication management.5, 6 In the current study, we found that patients understood their notes regardless of their age, race, or socioeconomic group; there was a slight dip in understanding among those with a high school education or less which was consistent, though modestly so, with our hypothesis. In the information era, people are becoming used to having vast amounts of information at their fingertips and providing full transparency with online health records has come of age. All patients stand to gain from greater access to information about their health and care.

On the other hand, some patients wrote that notes made them feel their clinicians did not care about them or were too busy to provide individualized care. Although affecting only a relatively small proportion of patients, we also found that those reporting they understood only some or very little of the note, disagreed that the note accurately described the visit, or found something important missing may have trust affected substantially, as another study has suggested.17 However, we found fewer patients than hypothesized finding errors or omissions in their notes. While clinicians may not consider some of the reported errors clinically important, for patients they may suggest lack of attentiveness. The written comments suggest that accuracy is important to patients, whether referring to incorrect or to missing information, and that they would like a mechanism on the patient portal to offer corrections. This issue is complex, as evidenced by the many patients who would still recommend their providers despite noting errors. Nevertheless, consistent with our hypothesis, the likelihood of recommending a clinician to others was substantially lower when patients found a note inaccurate or incomplete.

Only about 1 in 5 respondents offered suggestions for improvement, but we should not assume the majority were satisfied. Those who did not write a response may have had limited expectations about their notes, or few ideas on how to improve them. Some respondents hesitated in their comments, reporting that they feared being labeled as a “difficult” or “problem” patient. Such reluctance has particular relevance for those working to reduce medical errors and increase diagnostic accuracy by calling on patients who read their notes.10 As patients become more familiar with reviewing their records on portals, devising user-friendly and secure methods for reporting and correcting errors will become more urgent.

As the practice of opening notes spreads, the commentary exampled in Table 3 provides further insights and points toward new initiatives. Perhaps most importantly, many patients suggested that notes should prominently and clearly indicate next steps in their treatment plans and offer explicit advice about what they should do to maintain health or better manage illness. This may point to a need to reorganize the structure and content of notes to maximize their usefulness for patients and providers. Greater attention to the treatment plan and goals of treatment could improve patient adherence and enhance ongoing evaluation of the care plan. In addition, many patients suggested minimizing abbreviations and what they viewed as medical jargon.

Some of the patients’ suggestions for improving notes point to technical solutions, such as “mouse overs” that define terms and spell out acronyms, or provide links to glossaries. Voice recognition and other advancing technologies may improve accuracy and reduce typing during visits. While decisions about changes to the order of notes would be made by clinicians, making notes easier to find and making portals more patient friendly can best be addressed by those who design and build patient portals, working in collaboration with patients.18 These issues gain urgency at a time when patients’ connections to their health information is accelerating. Under the 21st Century Cures Act, rules taking effect November 2, 2020 require patients be provided access to all the health information in their electronic medical records, with few exceptions.19 Clinical notes are included in the information that must be made available.

This study has important limitations. Patients may have selected those notes they most liked or best understood, not wanting to report on those they did not understand or could only dimly recall. Reporting that they understood the note might indicate they felt they understood what they needed to understand, rather than knowledge of the medical details.20 The survey solicited self-reports. It did not include efforts to confirm either patients’ understanding or their reports of inaccurate or missing information, and future studies should solicit patient responses to notes selected at random from their records. Moreover, respondents were predominantly white and college educated, and our findings may not reflect the experiences of more diverse and less-educated patients; when patients at a safety-net hospital were handed the last note from their clinical record, several reported difficulties understanding the note.21 Further research among more diverse patient groups is needed to better understand the needs and suggestions from more disadvantaged patient populations. Finally, while it addressed a topic relevant to patients and used reminders and incentives, the survey’s response rate was low, although not unexpectedly so for an online survey.4, 22 When we compared respondents to non-respondents, we found modest differences between groups in that those responding to our survey had a heavier burden of major chronic conditions. In a previous report, we noted survey respondents were somewhat older than non-respondents.5 It may be fair to infer that patients who responded were those most interested in having access to their notes and, in turn, given their higher morbidity, may have the most to gain from reading them.

Spurred in part by the OpenNotes movement that began 10 years ago with a study of a few thousand patients,4 more than 50 million Americans today have ready access to their notes.3 The good news from this study is that those written by a broad array of clinicians were comprehensible to an equally broad range of readers, and that most patients found them accurately describing their visits. With no evidence of major disparities among patients according to education or other sociodemographic or health characteristics, these findings complement mounting evidence that open notes can empower patients to become more informed and engaged in their care. Informed patients, working in partnership with their clinicians, may both be safer and make better choices. Patients, clinicians, and health systems working to optimize the structure and quality of notes have a rich opportunity to inform, to generate mutual trust, and to improve the quality of care.