Abstract
Background: Creating useful clinical quality measure (CQM) reports in a busy primary care practice is known to depend on the capability of the electronic health record (EHR). Two other domains may also contribute: supportive leadership to prioritize the work and commit the necessary resources, and individuals with the necessary health information technology (IT) skills to do so. Here we describe the results of an assessment of the above 3 domains and their associations with successful CQM reporting during an initiative to improve smaller primary care practices’ cardiovascular disease CQMs.
Methods: The study took place within an AHRQ EvidenceNOW initiative of external support for smaller practices across Washington, Oregon and Idaho. Practice facilitators who provided this support completed an assessment of the 3 domains previously described for each of their assigned practices. Practices submitted 3 CQMs to the study team: appropriate aspirin prescribing, use of statins when indicated, blood pressure control, and tobacco screening/cessation.
Results: Practices with advanced EHR reporting capability were more likely to report 2 or more CQMs. Only one-third of practices were “advanced” in this domain, and this domain had the highest proportion of practices (39.1%) assessed as “basic.” The presence of advanced leadership or advanced skills did not appreciably increase the proportion of practices that reported 2 or more CQMs.
Conclusions: Our findings support previous reports of limited EHR reporting capabilities within smaller practices but extend these findings by demonstrating that practices with advanced capabilities in this domain are more likely to produce CQM reports.
- Electronic Medical Records
- Idaho
- Medical Informatics
- Oregon
- Primary Health Care
- Quality Improvement
- Quality of Care
- Washington
Introduction
Clinical quality measurement is a core component of any primary care improvement initiative and there is great promise in using electronic health records (EHRs) to make this possible. However, practice burdens such as data entry, interference with patient interactions, and lack of interoperability,1,2 contribute to a high burden of creating and reporting these measures in primary care settings.3 Unfortunately, the basic EHR reporting capability that meets requirements for Meaningful Use certification is often of limited utility in these efforts.4,5
Beyond the capabilities of the EHR to capture data and generate useful reports, other domains may contribute to the ability to both create and report clinical quality measures (CQMs). It is widely acknowledged that practice change requires supportive leadership to both prioritize the work within a busy clinic setting and commit the necessary resources.6–8 The development of CQM reporting capability should be similar. In addition, creating useful CQM reports requires individuals in the practice with the necessary skill set to do so. This skill set has been reported as mostly absent in smaller primary care practices, and often requires external support and training to acquire.4,9
Assessing and strengthening these domains may be a potentially promising strategy to increase a practice’s capacity to improve the quality of care it delivers.10 The purpose of this study is to describe the results of an assessment of the above 3 domains and their associations with successful CQM reporting during an initiative to improve smaller primary care practices’ cardiovascular disease (CVD) risk reduction efforts.
Methods
The Healthy Hearts Northwest (H2N) study took place from 2016 to 2020 within 209 smaller primary care practices across Washington, Oregon and Idaho.11 To be eligible, practices were required to have fewer than 10 full-time clinicians in a single location and to have participated in stage 1 Meaningful Use federal certification for their Electronic Health Record (EHR). The overall purpose was to lower the risk of cardiovascular disease (CVD) as measured by their performance on 3 CVD CQMs–appropriate aspirin prescribing,12 blood pressure control,13 and tobacco screening/cessation,14 by building QI capacity within the practice. To achieve this, each of the enrolled practices was supported by a practice facilitator in making changes to improve their performance on these CQMs. The facilitators were supported by 1 of 2 organizations, Qualis Health (now Comagine Health) for practices in Washington and Idaho, and the Oregon Rural Practice Research Network for practices in Oregon. Facilitators met monthly with a team within each practice over 15 months to assist them in extracting their CQMs and developing an improvement plan. The primary outcomes of this study and a detailed description of the facilitator intervention have been previously published.10,15
To guide the facilitator in supporting the practice in extracting and reporting CQM measures from the EHR, 2 of the authors (JH, RH) developed a rapid assessment of the 3 domains previously described: leadership support, EHR reporting capabilities, and staff skills for reporting. A relevant description of each domain was written, with input from the facilitators, along with a 3-level response: basic, intermediate, or advanced. Two of the authors (JH with facilitators in Washington and Idaho, and RH with facilitators in Oregon) administered the assessment by phone with each facilitator by asking them to score each of their practices on each domain. These calls were held after at least 6 months of interaction with each practice to give facilitators the opportunity to understand the practice’s quality reporting environment and use this understanding to guide their support for the practice.
We calculated the number and proportion of practices within each level of each domain, and the association between being advanced in 1 domain with being advanced in another domain. We then examined the association between being advanced in each domain with the likelihood that the practice was able to report 2 or more of the 3 CQMs, and the association between the number of advanced domains present in each practice and ability to report 2 or more CQMs. Chi-squared tests with odds ratios were used to assess these associations. This study was determined to be exempt (category 2) by the Kaiser Permanente Washington Health Research Institute’s Institutional Review Board, waiving the requirement for informed consent but not ethics review.
Results
A total of 209 practices enrolled in the study. Of those, 205 were still actively engaged in study activities at the time of the assessments with the facilitators. Of those, 199 had complete data for all 3 domains and reporting of CQMs. Organizational characteristics of the practices and their baseline performance on each CQM are shown in Table 1. The distribution of practices across the levels (basic, intermediate, advanced) for each domain is found in Table 2. Approximately 39% of practices were assessed as “basic” within the software reporting capability domain compared with approximately 21% within the leadership and skills domains. Only 14.1% were assessed as advanced in all 3 domains whereas 39.1% were assessed as advanced in none of the domains. All 3 of the domains were significantly associated with each other: advanced leadership with advanced skills (OR = 2.76, 95% CI = 1.48, 5.14), advanced leadership with advanced EHR reporting capability (OR = 2.31, 95% CI = 1.26, 4.22), and advanced EHR reporting capability with advanced skills (OR = 12.87, 95% CI = 6.28, 26.37).
Practice Characteristics (n = 199)
Assessment Results
The relationship between being advanced in each domain or combinations of domains and ability to report 2 or more CQMs is found in Table 3. When examined individually, the presence of advanced EHR reporting capability was the only domain associated with reporting 2 or more CQMs, with 75% of practices in this category capable of doing so. Practices with advanced EHR reporting capability were significantly more likely to report 2 or more CQMs (72.7%) compared with those with basic or intermediate capabilities (42.9%) (O.R. 4.02, 95% CI 2.13, 7.60). When combining domains, the combinations that included advanced EHR reporting capability had the highest proportion of practices able to report 2 or more CQMs (Table 3). In addition, as the total number of advanced domains present in each practice increased so did the proportion reporting 2 or more CQMs: 39.8% for those with none, 46.2% for those with 1, 67.9% for those with 2, and 75% for those with all 3. (P < .01).
Association of “Advanced” in Each Domain with Reporting 2 or More CQMs (n = 199)
Discussion
Across 199 smaller primary care practices enrolled in a study of support to improve care quality, practices with advanced EHR reporting capability were more likely to successfully report at least 2 of the 3 CQMs. However, only a third of practices were assessed as advanced in this domain, and this domain had the highest proportion of practices assessed as “basic.” The presence of advanced leadership or advanced skills, when examined individually, was not associated with the proportion of practices that were able to report 2 or more CQMs. Our findings support previous reports of limited EHR reporting capabilities within smaller practices,4,16,17 but extend these findings by demonstrating that practices with advanced capabilities in this domain are more likely to produce CQM performance reports.
The significant associations between the 3 domains, along with the finding that the total number of advanced domains present in each practice, raises the question of how each might contribute to strengthening the presence of the other when practices work to improve their CQM reporting capabilities. For example, it is plausible that advanced leadership contributes to acquiring software with advanced reporting capability and to hiring or protecting staff time to acquire the skills needed to produce reports from the software available. In similar fashion, it may be easier for staff to acquire an advanced skill set if they have the advanced EHR reporting system needed to do so. In addition, it is possible that advanced leadership and staff skills may drive quality of clinical care in ways that current CQMs do not capture.
The reporting functionality required of EHRs for Meaningful Use certification was designed for yearly reporting of performance to determine value-based payment as part of payor-based quality initiatives. It was not designed to produce the types of internally facing reports such as lists of patients with care gaps that practices themselves can use to improve quality. Practices with only basic reporting capability as defined here may be able to purchase analytic services from external vendors at additional cost and effort to produce reports of performance that they can use internally, but in small primary care settings this may be cost prohibitive.
Limitations of this study include the cross-sectional nature of the data and the reliance on an external observer to assess these 3 domains. The cross-sectional data do not provide us with an opportunity to test hypotheses about how 1 domain may depend on another for development. It is also possible that facilitators may have had limited opportunity to engage with some of their practices, limiting their ability to accurately assess these practices’ status within each domain.
More than a decade after the 2009 Health Information Technology for Economic and Clinical Health Act created regional extensions centers to support the adoption of EHRs in primary care practices, a large proportion of the practices in this study did not have the EHR reporting capability needed to report CQMs, much less use those reports to improve the quality of the care they deliver. Improving the capability of EHRs to report CQMs in primary care practices is an important first step but smaller practices may require external support, such as that provided by facilitators, to use those reports to realize sustained gains in quality of care.5,18–20
Ethics Approval and Consent to Participate
This study was determined to be exempt (category 2) by the Kaiser Permanente Washington Health Research Institute’s Institutional Review Board, waiving the requirement for informed consent but not ethics review.
Acknowledgments
The authors are grateful to the primary care practices that participated in the study and the practice facilitators from Qualis/Comagine Health and the Oregon Rural Practice Research Network (ORPRN). We also appreciate feedback on drafts of this manuscript provided by Ellen O’Meara and Leah Tuzzio at the Kaiser Permanente Washington Health Research Institute, and those by David Dorr, L.J. Fagnan and Caitlin Dickinson at ORPRN.
Notes
This article was externally peer reviewed.
Michael L. Parchman MD, MPH (mparchman@chwcoalition.org), Laura-Mae Baldwin MD, MPH (lmb@uw.edu), Ross Howell MPH (rosshowell@gmail.com), Jeffrey Hummel MD, MPH (philip.jeffrey.hummel@gmail.com)
Funding: This project was supported by grant number R18HS023908 from the Agency for Healthcare Research and Quality. (AHRQ) The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ. Additional support was provided by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number UL1 TR002319. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Conflict of interest: None.
To see this article online, please go to: http://jabfm.org/content/37/2/316.full.
- Received for publication March 22, 2023.
- Revision received November 16, 2023.
- Accepted for publication November 27, 2023.
