Abstract
Background: Medical scribes have been utilized to reduce electronic health record (EHR) associated documentation burden. Although evidence suggests benefits to scribes, no large-scale studies have quantitatively evaluated scribe impact on physician documentation across clinical settings. This study aimed to evaluate the effect of scribes on physician EHR documentation behaviors and performance.
Methods: This retrospective cohort study used EHR audit log data from a large academic health system to evaluate clinical documentation for all ambulatory encounters between January 2014 and December 2019 to evaluate the effect of scribes on physician documentation behaviors. Scribe services were provided on a first-come, first-served basis on physician request. Based on a physician’s scribe use, encounters were grouped into 3 categories: never using a scribe, prescribe (before scribe use), or using a scribe. Outcomes included chart closure time, the proportion of delinquent charts, and charts closed after-hours.
Results: Three hundred ninety-five physicians (23% scribe users) across 29 medical subspecialties, encompassing 1,132,487 encounters, were included in the analysis. At baseline, scribe users had higher chart closure time, delinquent charts, and after-hours documentation than physicians who never used scribes. Among scribe users, the difference in outcome measures postscribe compared with baseline varied, and using a scribe rarely resulted in outcome measures approaching a range similar to the performance levels of nonusing physicians. In addition, there was variability in outcome measures across medical specialties and within similar subspecialties.
Conclusion: Although scribes may improve documentation efficiency among some physicians, not all will improve EHR-related documentation practices. Different strategies may help to optimize documentation behaviors of physician-scribe dyads and maximize outcomes of scribe implementation.
- Ambulatory Care
- Burnout
- Communication
- Documentation
- Electronic Health Records
- Health Care Systems
- Health Services
- Outcomes Assessment
- Physicians
- Primary Health Care
- Retrospective Studies
- Quality Improvement
- Quantitative Research
Introduction
Over the past decade, a physician burnout epidemic has grown, driven largely in part due to electronic health records (EHRs).1 EHR use has been associated with increased after-hours work and significantly reduced time spent in direct patient care.1⇓⇓–4 Unfortunately, despite a plethora of studies documenting the role poor EHR usability plays in this situation, there have been few practical improvements.5 Notably, primary care and family medicine physicians have been found to spend more time working in the EHR than face-to-face with patients; and they are more likely to work through lunch, remain late after clinic, or take work home to complete EHR duties.6⇓–8 As a result, several workaround solutions have been developed to “untether” the physician from the EHR to improve efficiency and physician satisfaction.
Medical scribes are the traditional, human-based solution to reducing documentation burden and are used by more than 20% (1 of every 9) of physicians in the United States.9 Although multiple professional groups, including medical assistants and nurses, can serve as scribes, the majority are preprofessional school or college graduates who use the experience as a stepping-stone to graduate studies in the medical field. Literature indicates that scribe implementation improves clinical workflows, clinician efficiency, and patient-physician interactions.10,11 This can boost clinician well-being and job satisfaction, mitigating the risk of burnout.12⇓⇓⇓⇓–17 In primary care, scribes have been associated with some improvement to patient experience, increased time spent patient-facing, and the completeness of history of present illness documentation.11,18,19 In addition, some studies suggest that scribes can have the potential to improve the timeliness of documentation and reduce after-hour work, however prior study results remain inconsistent across the literature.11,15,20,21
Prior studies suggest that interventions targeting specific EHR efficiency behaviors may reduce documentation burden among primary care physicians and support from other members of the care team are an important determinant of overall EHR time of physicians.22 Although medical scribes are being increasingly adopted by physicians across various specialties, including family medicine, there remains a need to understand their full impact within the specific context of primary care practice. A recent scoping review of scribe studies conducted in primary care settings underscored the significance of scribe interventions and their potential to address the unique challenges faced by family physicians.21 Although this review highlighted the possible benefits derived from scribe use, it also indicates that many of the prior studies involve small sample sizes and lacked nonscribe using comparison groups.
These limitations are not uncommon among prior investigations of medical scribe use. Most quantitative studies have remained narrow in scope and timescale, confined to only a single or handful of clinical settings over a limited period. These investigations have often been undersized in their sampling of physicians and scribes. There remains a need for generalizable, large-scale studies of scribe impact on documentation in diverse clinical settings. It is imperative to contextualize these findings within the realm of primary care, where the application of scribes has significant implications to improve clinical workflows and physician efficiency, thereby improving burnout. Therefore, this study aimed to evaluate objective measures of the effects of scribe use on the timeliness and timing of completion of clinical documentation by physicians across multiple specialties.
Methods
This study was reviewed and approved by the Oregon Health & Science University (OHSU) Institutional Review Board (STUDY00017599), which waived informed consent due to the study’s retrospective nature. OHSU is an academic medical center that has maintained an internal, “home-grown” scribe program since 2015, deploying scribes to more than 120 physicians across more than 55 clinical environments.23 At physician request, scribe services are provided on a first-come, first-serve basis. All scribes employed by the program participate in didactic training, delivered by the program manager, before working clinically. After initial training, they are supervised by a “lead-scribe” before going on to independent documentation supervised by the physician(s) which the scribes are assigned to.
This study was a secondary data analysis from a retrospective cohort study. We analyzed audit log data from Epic (EpicCare; Epic Systems) ambulatory encounters between 2014 and 2019. This time frame was selected to eliminate the impact of confounders due to the COVID-19 pandemic.24 Encounters were excluded if a trainee was involved in clinical documentation or if the documentation remained unclosed at the time of data extraction.
We identified 3 specific outcome measures to evaluate - (1) “chart closure time,” which was defined as the difference in days between a scheduled visit date-time and the date-time that the physician completed documentation, (2) “delinquent” documentation as defined by organizational policies, as chart closure more than 14 days after an encounter, and (3) “after-hours use,” which was defined as any time spent between 7:00 pm and 7:00 am on weekdays and any time on weekends. These measures are relevant to multiple clinical specialties, including family medicine. In previously published work, our group demonstrated that the use of a medical scribe had no impact on the time to chart completion, the fraction of delinquent charts, or the amount of after-hours EHR use, however, that work did not aggregate at the level of specialty, subspecialty or clinic analyses.25
Physicians included in the present study had either never used scribes (“nonusers”) or had documentation before scribe assignment (“prescribe”) and documentation using scribes (“postscribe”). We limited the analysis to physicians with >100 encounters per category) and only included subspecialties with at least 5 physicians who were nonusers and 5 who were scribe users (encounters both prescribe and postscribe). For the current study, documentation outcome measures were compared between nonusers and prescribe physicians, as well as a paired comparison of outcome measures for physicians from prescribe to postscribe. We evaluated each outcome measure across nonusers, prescribe, and postscribe groups in 4 major medical specialties and then assessed outcome measures by subspecialties. Finally, to study trends in the outcome measures among physicians who used scribes, we plotted scribe-user differences in performance prescribe to postscribe across differences in performance between prescribe and nonuser within the same subspecialty. We tested for between-group differences using Wilcoxon rank sum tests and 2-sample tests for equality of proportions. All analyses were performed using R Version 4.2.2. Descriptive statistics were reported as mean (standard deviation) or median [interquartile range]. All statistical tests were 2-sided, and significance was defined as P < .05.
Results
Description of Sample
This study aimed to compare efficiency measures across physicians who do and do not use medical scribes in various clinical environments. The data set included 1,132,487 encounters across 29 outpatient subspecialties (Table 1). Of these encounters, 790,416 (70%) were with nonusers, 190,115 (17%) prescribe, and 151,956 (13%) postscribe. These encounters were linked to 29 subspecialties and 395 physicians, of which 91 (23%) had used 133 scribes (Table 1).
Encounters were grouped by subspecialties under 1 of the 4 major specialties, including internal medicine (574,416; 51%), surgery (107,961; 9.5%), obstetrics and gynecology (119,218; 11%), and pediatrics (330,892; 29%). Subspecialties grouping labels and titles are presented in Table 2.
Outcome Measures by Specialty
To evaluate the differences between the nonusers and prescribe groups, outcome measures were analyzed across 4 major specialties at the encounter level by group (Figure 1). Apart from obstetrics and gynecology, the mean and median chart closure times across each specialty were higher prescribe than nonusers (Figure 1A). The prescribe group also had higher proportions of delinquent charts compared with nonusers, and comparable results were observed for the proportion of charts closed after-hours (Figure 1B & Figure 1C), again with obstetrics and gynecology being an exception.
We next assessed the outcome measures of scribe users, evaluating the difference in outcome measures prescribe to postscribe. Mean chart closure time was higher postscribe than prescribe in pediatrics and obstetrics and gynecology, but lower in internal medicine and surgery (Figure 1, Figure A). In the same comparison, median chart closure time was higher in internal medicine but lower in pediatrics, obstetrics and gynecology, and surgery. Similar findings were observed for the proportion of delinquent charts, except with obstetrics and gynecology which had higher proportions postscribe to prescribe. In comparison, lower proportions were found postscribe to prescribe in internal medicine, pediatrics, and surgery (Figure 1B). The proportion of charts closed after-hours was lower postscribe in all 4 specialties compared with prescribe (Figure 1C). When comparing postscribe with nonusers, each outcome measure was higher for the postscribe group across all 4 specialties.
Outcome Measures by Subspecialty
Based on these findings, outcome measures were then analyzed, by group, for all subspecialties within each of the 4 major specialties. Overall, the nature of each outcome measure was highly variable between subspecialties belonging to the same major specialty and within a given subspecialty from prescribe to postscribe. For example, for the 12 subspecialties of internal medicine (Figure 2), the mean and median chart closure time was higher prescribe compared with nonusers in the subspecialties (10 vs 2 or 83% and 9 vs 3 or 75%, respectively). Comparing the prescribe to postscribe conditions, more internal medicine subspecialties had a higher mean and median chart closure time postscribe (6 vs 6 or 50% and 7 vs 5 or 58%, respectively). Similar results were observed for the proportion of delinquent charts and charts closed after-hours, subspecialties had higher values in outcome measures postscribe compared with prescribe (5 vs 7 or 42% and 4 vs 8 or 32%, respectively).
Again, among internal medicine subspecialties, the mean and median chart closure times were higher postscribe compared with nonusers (10 vs 2 or 83% and 10 vs 2, 83%, respectively). The postscribe group also had higher proportions of delinquent charts and proportions of after-hour chart closure (8 vs 4 or 67% and 8 vs 4 or 67%, respectively). Similar variability in the nature of the outcome measures by group was observed across subspecialties within pediatrics, obstetrics and gynecology, and surgery (Figure 3, Figure 4, and Figure 5).
Outcome Measures by Physician
Given the variability of the impact of scribe adoption at the specialty and subspecialty levels, we next ascertained the variability of impact at the physician level and, precisely, whether this was related to a physicians’ degree of deviation in their behavior prescribe compared with nonuser peers. For each outcome, we plotted each physician’s prescribe deviation (difference) from the mean of nonuser peers within their specialty versus the magnitude of change (difference) prescribe to postscribe. (Figure 6). More than half of scribe users had higher mean chart closure time prescribe compared with nonusers (57 vs 32 or 64%; Figure 6A, Q-I & Q-II) and, of those 57 physicians, less than half had higher chart closure time postscribe than prescribe (27 vs 30 or 47%; Figure 6A, Q-II). Of physicians who had a lower mean chart closure time prescribe than to nonusers (32 vs 57 or 36%; Figure 6B, Q-III & Q-IV), very few had higher chart closure time postscribe than prescribe (9 vs 23 or 28%; Figure 6A, Q-IV). In total, almost half of all physicians had an increase in mean chart closure time after scribe adoption (36 vs 21 or 41%; Figure 6A, Q-II & Q-III). Similar trends were observed for the other outcome measures (Figure 6B-D). Finally, there was little consistency in whether physicians remained in the same quadrant across all outcome measures, with only 23 (25%) of physicians being in the same quadrant across all outcome measures.
Discussion
The key findings from this study, analyzing a large sample of EHR audit-log data, were that physicians who were using medical scribes had poorer chart closure time, delinquent charts, and after-hours documentation prescribe compared with physicians who never used scribes. The results also suggest that adopting scribes may have little to no impact on outcome measures for the cohort as a whole, with many scribe-using physicians rarely approaching the performance levels of nonuser colleagues, and mild to moderate improvements in outcome measures for others. In addition, considerable variability was observed in outcome measures across medical specialties, and this heterogeneity was sustained across analyses of similar subspecialties. However, at the individual physician level, the difference in outcome measures at baseline compared with performance with a scribe fluctuated – some physicians had considerable differences in outcomes once assigned a scribe, whereas others had little to no change in some or all outcome measures.
To our knowledge, this is one of the first studies to analyze a wide sampling of physicians, stratified by clinical specialty and subspecialty, to examine the differences in clinical documentation behaviors when scribes are used. For each outcome measure, mean and median chart closure time, delinquent chart rates, and after-hour chart closure rate, encounters with physicians prescribe had higher values than encounters with physicians who never used scribes. The difference between prescribe and postscribe outcome measures also varied by clinical specialty. Differences in documentation styles between specialties may lead to longer documentation in some settings compared with others, and prior evidence supports that EHR time of clinicians varies greatly between and within specialties.26,27 However, the proportion of time spent on clinically relevant EHR activities remains similar across specialties.26 Comparing findings with prior studies of scribes is difficult, as previous investigators have categorized and reported scribe-related outcome measures disparately. Two notable quantitative evaluations of scribe use have reported on proportions of charts closed, at specific intervals, with varying results; charts completed within 48 hours (increased) and completed at 72 hours (not significantly different).16,28
Our findings regarding chart closure time with scribes are notably different from others.29⇓⇓⇓–33 The discordance between our results and those of prior work is likely attributable to the volume and variety of physicians included in our sample. In addition, a large proportion of the literature on scribe use is based in the emergency department setting, which has a fundamentally different workflow regarding chart closure and chart hygiene compared with the ambulatory clinics in this study.34 Interestingly, little work has been done to explore how these outcome measures are exhibited across multiple levels of a nested sample of specialties, subspecialties, and physicians. These types of multi-level analyses are important because ambulatory clinical workflows can vary across medical specialties and practice settings.26 In addition, other factors not evaluated in this analysis may impact chart closure time. Physician time in practice, worktime spent with or without scribes, length of physician-scribe relationship, and the use of templates may be important characteristics that influence workflow and warrant further investigation. The variability in the difference for outcome measures observed in subspecialties of this study suggest that some clinical environments may be better suited to benefit from scribe use to support clinical work and documentation.
When we looked to see whether the impact of scribe adoption for an individual physician was related to their prescribe deviance from nonscribe using peers, we again observed significant heterogeneity, with even some of the worst performing physicians prescribe still showing little improvement postscribe and scribe adoption actually worsening postscribe performance of physicians who had high prescribe performance. Further, trends in these patterns were variable across mean and median chart closure time, delinquent chart rates, and after-hour chart closure rate. These findings suggest, but do not prove, that the variability observed in the study is more likely to be accounted for within individual physicians rather than by clinical specialty or subspecialty.
One explanation for why there was not broad improvements in the reported outcomes with scribe adoption is that some workflow efficiencies gained by using scribes during the clinical encounter are negatively affected after the fact, influencing the rate of delinquencies and after-hour completion of charts. For example, one of the main responsibilities for scribe using physicians, is to ensure the accuracy of the clinical documentation, but the degree to which this is done may be highly variable and provider dependent.35⇓–37 Importantly, this individual-level variability was observed across all ambulatory specialties and encompassed the broad patient characteristics of those often managed by family medicine physicians (pediatrics, obstetrics and gynecology, general surgery, etc.), which suggests this is likely a pervasive phenomenon. This individual level variability is important, because although scribes are broadly associated with higher job satisfaction, our data reflect the fact that this may not be reflective of general improvements in multiple metrics of workflow, many of which are independently associated with higher levels of burnout (eg, after-hours work).1⇓⇓–4 Combined, this may explain the observation of some large scale studies suggesting that burnout, in particular, is not reduced by medical scribes. Thus, if the primary goal is to improve physician burnout, it is likely that scribes will not only fail to uniformly address the issue, but rather, they should be part of a suite of other reported beneficial solutions including improved training and workflow analysis with sprints.38⇓–40
The findings of this study suggest that when physicians are given a medical scribe to improve chart closure time, delinquent charts, or after-hour EHR usage, some will benefit whereas others may not. The timely completion of clinical documentation is an important benchmark for physicians, health care organization leadership, and payers. Complete and timely documentation serves several purposes: as a record for patient care and communication between physicians, a necessary record for claim reimbursement, and a legal document should a matter end up in litigation. Further research, potentially with qualitative components, would be valuable to explore how and why implementation strategies influence observed outcomes of scribe utilization. When physicians elect to implement scribes, physician-level measures will allow individuals to compare their documentation behaviors with those reported for their subspecialty, illuminating their performance in the context of other physicians and how the use of a scribe has changed such behaviors.
Health care organization leadership can use these types of data to gain a realistic insight into the changes in physician documentation behaviors after scribe implementation, justify investment in medical scribes, or highlight where alternative interventions may be more beneficial to improve clinical documentation timeliness and, subsequently, revenue cycle operations.41⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓–53 Through data visualization tools, organizations using EHRs have access to a wealth of user-level physician efficiency data, including in-depth data on how physicians complete clinical notes.13,18 Clinician executives have an opportunity to leverage this data which can inform their informatics strategies and identify how users compare with peers when using documentation assistance tools such as medical scribes–knowledge that would enable them to target and implement user-specific solutions to reduce burnout related to documentation burdens.
The limitations of this study warrant consideration. The retrospective nature of the study limits our ability to infer causal relationships between scribe use and documentation efficiency, due to potential selection bias and uncontrolled confounding. Second, due to the descriptive nature of the analysis, we were unable to adjust for important confounders that may be relevant to clinical documentation behaviors or have influenced physician-scribe pair assignments. For example, workflow factors such as daily patient volume, number of clinic days per week, or full-time equivalent. In addition, the approach used in the roll-out of a scribe program may also introduce additional confounders which were not accounted for. The philosophy behind the implementation of a scribe program, including whether it is prioritized for highly productive clinicians or used as a remediation strategy for physicians with frequent delinquent charts, could influence the observed outcomes. It is important to acknowledge that many of the prior studies of scribes are conducted in groups of physicians who willingly requested scribe services (eg, scribe services are not randomly assigned to physicians), which may introduce a potential bias as these physicians may differ in characteristics or practice patterns from those who do not seek scribe services. The current body of literature on medical scribes includes very few randomized control trials, and this lack of randomization could influence the generalizability of prior findings. This is another opportunity for future work.
The data used in this study originated from a single academic medical system’s EHR. As a result, some findings may not be generalizable to other clinical environments due to alternative EHR systems, organizational policies and practices, and differences in institutional culture. Finally, the outcomes included in this analysis represent narrow parameters of clinical documentation behaviors and do not necessarily reflect the absolute gains that could be attributed to the implementation of medical scribes regarding changes in clinical workflows, physician experience, or charge capture and billing. Although, many of these financial factors are likely less significant with the recent changes to CMS billing requirements, greatly reducing the role for documentation of specific elements to improve charge capture and reimbursement.54
Conclusions
This study proposes that EHR audit logs contain rich information regarding physician documentation behaviors in ambulatory care and across a wide range of workflows relevant to family medicine. It proposes a model to review both the inter- and intraclinician impact of scribe adoption on a number of metrics related to documentation and EHR use. These data, combined with other measurements of physician satisfaction can be used by organizations to not only better target initial scribe allotment, but also allow for real-time monitoring of scribe impact. These data may have utility to inform organizational practices regarding scribe utilization.
Responsible Research Conduct
All the methods and procedures conducted in this study followed relevant guidelines and regulations defined in the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects. This study was reviewed and approved by the OHSU Institutional Review Board (STUDY00017599), which waived informed consent due to the study’s retrospective nature.
Notes
This article was externally peer reviewed.
Funding: This project is supported under contract grant #R01HS025141 from the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services. The results from this paper are from the authors and do not represent the views of AHRQ or the U.S.
Conflict of interest: The authors of this manuscript declared that they have no conflicts of interest, financial or otherwise.
To see this article online, please go to: http://jabfm.org/content/37/2/228.full.
- Received for publication May 31, 2023.
- Revision received August 10, 2023.
- Revision received September 12, 2023.
- Accepted for publication September 18, 2023.