The Ability of Primary Care Practices to Measure and Report on Care Quality
===========================================================================

* Michael L. Parchman
* Laura-Mae Baldwin
* Ross Howell
* Jeffrey Hummel

## Abstract

*Background:* Creating useful clinical quality measure (CQM) reports in a busy primary care practice is known to depend on the capability of the electronic health record (EHR). Two other domains may also contribute: supportive leadership to prioritize the work and commit the necessary resources, and individuals with the necessary health information technology (IT) skills to do so. Here we describe the results of an assessment of the above 3 domains and their associations with successful CQM reporting during an initiative to improve smaller primary care practices’ cardiovascular disease CQMs.

*Methods:* The study took place within an AHRQ EvidenceNOW initiative of external support for smaller practices across Washington, Oregon and Idaho. Practice facilitators who provided this support completed an assessment of the 3 domains previously described for each of their assigned practices. Practices submitted 3 CQMs to the study team: appropriate aspirin prescribing, use of statins when indicated, blood pressure control, and tobacco screening/cessation.

*Results:* Practices with advanced EHR reporting capability were more likely to report 2 or more CQMs. Only one-third of practices were “advanced” in this domain, and this domain had the highest proportion of practices (39.1%) assessed as “basic.” The presence of advanced leadership or advanced skills did not appreciably increase the proportion of practices that reported 2 or more CQMs.

*Conclusions:* Our findings support previous reports of limited EHR reporting capabilities within smaller practices but extend these findings by demonstrating that practices with advanced capabilities in this domain are more likely to produce CQM reports.

*   Electronic Medical Records
*   Idaho
*   Medical Informatics
*   Oregon
*   Primary Health Care
*   Quality Improvement
*   Quality of Care
*   Washington

## Introduction

Clinical quality measurement is a core component of any primary care improvement initiative and there is great promise in using electronic health records (EHRs) to make this possible. However, practice burdens such as data entry, interference with patient interactions, and lack of interoperability,1,2 contribute to a high burden of creating and reporting these measures in primary care settings.3 Unfortunately, the basic EHR reporting capability that meets requirements for Meaningful Use certification is often of limited utility in these efforts.4,5

Beyond the capabilities of the EHR to capture data and generate useful reports, other domains may contribute to the ability to both create and report clinical quality measures (CQMs). It is widely acknowledged that practice change requires supportive leadership to both prioritize the work within a busy clinic setting and commit the necessary resources.6–8 The development of CQM reporting capability should be similar. In addition, creating useful CQM reports requires individuals in the practice with the necessary skill set to do so. This skill set has been reported as mostly absent in smaller primary care practices, and often requires external support and training to acquire.4,9

Assessing and strengthening these domains may be a potentially promising strategy to increase a practice’s capacity to improve the quality of care it delivers.10 The purpose of this study is to describe the results of an assessment of the above 3 domains and their associations with successful CQM reporting during an initiative to improve smaller primary care practices’ cardiovascular disease (CVD) risk reduction efforts.

## Methods

The Healthy Hearts Northwest (H2N) study took place from 2016 to 2020 within 209 smaller primary care practices across Washington, Oregon and Idaho.11 To be eligible, practices were required to have fewer than 10 full-time clinicians in a single location and to have participated in stage 1 Meaningful Use federal certification for their Electronic Health Record (EHR). The overall purpose was to lower the risk of cardiovascular disease (CVD) as measured by their performance on 3 CVD CQMs–appropriate aspirin prescribing,12 blood pressure control,13 and tobacco screening/cessation,14 by building QI capacity within the practice. To achieve this, each of the enrolled practices was supported by a practice facilitator in making changes to improve their performance on these CQMs. The facilitators were supported by 1 of 2 organizations, Qualis Health (now Comagine Health) for practices in Washington and Idaho, and the Oregon Rural Practice Research Network for practices in Oregon. Facilitators met monthly with a team within each practice over 15 months to assist them in extracting their CQMs and developing an improvement plan. The primary outcomes of this study and a detailed description of the facilitator intervention have been previously published.10,15

To guide the facilitator in supporting the practice in extracting and reporting CQM measures from the EHR, 2 of the authors (JH, RH) developed a rapid assessment of the 3 domains previously described: leadership support, EHR reporting capabilities, and staff skills for reporting. A relevant description of each domain was written, with input from the facilitators, along with a 3-level response: basic, intermediate, or advanced. Two of the authors (JH with facilitators in Washington and Idaho, and RH with facilitators in Oregon) administered the assessment by phone with each facilitator by asking them to score each of their practices on each domain. These calls were held after at least 6 months of interaction with each practice to give facilitators the opportunity to understand the practice’s quality reporting environment and use this understanding to guide their support for the practice.

We calculated the number and proportion of practices within each level of each domain, and the association between being advanced in 1 domain with being advanced in another domain. We then examined the association between being advanced in each domain with the likelihood that the practice was able to report 2 or more of the 3 CQMs, and the association between the number of advanced domains present in each practice and ability to report 2 or more CQMs. Chi-squared tests with odds ratios were used to assess these associations. This study was determined to be exempt (category 2) by the Kaiser Permanente Washington Health Research Institute’s Institutional Review Board, waiving the requirement for informed consent but not ethics review.

## Results

A total of 209 practices enrolled in the study. Of those, 205 were still actively engaged in study activities at the time of the assessments with the facilitators. Of those, 199 had complete data for all 3 domains and reporting of CQMs. Organizational characteristics of the practices and their baseline performance on each CQM are shown in Table 1. The distribution of practices across the levels (basic, intermediate, advanced) for each domain is found in Table 2. Approximately 39% of practices were assessed as “basic” within the software reporting capability domain compared with approximately 21% within the leadership and skills domains. Only 14.1% were assessed as advanced in all 3 domains whereas 39.1% were assessed as advanced in none of the domains. All 3 of the domains were significantly associated with each other: advanced leadership with advanced skills (OR = 2.76, 95% CI = 1.48, 5.14), advanced leadership with advanced EHR reporting capability (OR = 2.31, 95% CI = 1.26, 4.22), and advanced EHR reporting capability with advanced skills (OR = 12.87, 95% CI = 6.28, 26.37).

View this table:
[Table 1.](http://www.jabfm.org/content/37/2/316/T1)

Table 1. 
Practice Characteristics (n = 199)

View this table:
[Table 2.](http://www.jabfm.org/content/37/2/316/T2)

Table 2. 
Assessment Results

The relationship between being advanced in each domain or combinations of domains and ability to report 2 or more CQMs is found in Table 3. When examined individually, the presence of advanced EHR reporting capability was the only domain associated with reporting 2 or more CQMs, with 75% of practices in this category capable of doing so. Practices with advanced EHR reporting capability were significantly more likely to report 2 or more CQMs (72.7%) compared with those with basic or intermediate capabilities (42.9%) (O.R. 4.02, 95% CI 2.13, 7.60). When combining domains, the combinations that included advanced EHR reporting capability had the highest proportion of practices able to report 2 or more CQMs (Table 3). In addition, as the total number of advanced domains present in each practice increased so did the proportion reporting 2 or more CQMs: 39.8% for those with none, 46.2% for those with 1, 67.9% for those with 2, and 75% for those with all 3. (*P < .01*).

View this table:
[Table 3.](http://www.jabfm.org/content/37/2/316/T3)

Table 3. 
Association of “Advanced” in Each Domain with Reporting 2 or More CQMs (n = 199)

## Discussion

Across 199 smaller primary care practices enrolled in a study of support to improve care quality, practices with advanced EHR reporting capability were more likely to successfully report at least 2 of the 3 CQMs. However, only a third of practices were assessed as advanced in this domain, and this domain had the highest proportion of practices assessed as “basic.” The presence of advanced leadership or advanced skills, when examined individually, was not associated with the proportion of practices that were able to report 2 or more CQMs. Our findings support previous reports of limited EHR reporting capabilities within smaller practices,4,16,17 but extend these findings by demonstrating that practices with advanced capabilities in this domain are more likely to produce CQM performance reports.

The significant associations between the 3 domains, along with the finding that the total number of advanced domains present in each practice, raises the question of how each might contribute to strengthening the presence of the other when practices work to improve their CQM reporting capabilities. For example, it is plausible that advanced leadership contributes to acquiring software with advanced reporting capability and to hiring or protecting staff time to acquire the skills needed to produce reports from the software available. In similar fashion, it may be easier for staff to acquire an advanced skill set if they have the advanced EHR reporting system needed to do so. In addition, it is possible that advanced leadership and staff skills may drive quality of clinical care in ways that current CQMs do not capture.

The reporting functionality required of EHRs for Meaningful Use certification was designed for yearly reporting of performance to determine value-based payment as part of payor-based quality initiatives. It was not designed to produce the types of internally facing reports such as lists of patients with care gaps that practices themselves can use to improve quality. Practices with only basic reporting capability as defined here may be able to purchase analytic services from external vendors at additional cost and effort to produce reports of performance that they can use internally, but in small primary care settings this may be cost prohibitive.

Limitations of this study include the cross-sectional nature of the data and the reliance on an external observer to assess these 3 domains. The cross-sectional data do not provide us with an opportunity to test hypotheses about how 1 domain may depend on another for development. It is also possible that facilitators may have had limited opportunity to engage with some of their practices, limiting their ability to accurately assess these practices’ status within each domain.

More than a decade after the 2009 Health Information Technology for Economic and Clinical Health Act created regional extensions centers to support the adoption of EHRs in primary care practices, a large proportion of the practices in this study did not have the EHR reporting capability needed to report CQMs, much less use those reports to improve the quality of the care they deliver. Improving the capability of EHRs to report CQMs in primary care practices is an important first step but smaller practices may require external support, such as that provided by facilitators, to use those reports to realize sustained gains in quality of care.5,18–20

### Ethics Approval and Consent to Participate

This study was determined to be exempt (category 2) by the Kaiser Permanente Washington Health Research Institute’s Institutional Review Board, waiving the requirement for informed consent but not ethics review.

## Acknowledgments

The authors are grateful to the primary care practices that participated in the study and the practice facilitators from Qualis/Comagine Health and the Oregon Rural Practice Research Network (ORPRN). We also appreciate feedback on drafts of this manuscript provided by Ellen O’Meara and Leah Tuzzio at the Kaiser Permanente Washington Health Research Institute, and those by David Dorr, L.J. Fagnan and Caitlin Dickinson at ORPRN.

## Notes

*   This article was externally peer reviewed.

*   Michael L. Parchman MD, MPH (mparchman@chwcoalition.org), Laura-Mae Baldwin MD, MPH (lmb@uw.edu), Ross Howell MPH (rosshowell@gmail.com), Jeffrey Hummel MD, MPH (philip.jeffrey.hummel@gmail.com)

*   *Funding:* This project was supported by grant number R18HS023908 from the Agency for Healthcare Research and Quality. (AHRQ) The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ. Additional support was provided by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number UL1 TR002319. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

*   *Conflict of interest:* None.

*   To see this article online, please go to: [http://jabfm.org/content/37/2/316.full](http://jabfm.org/content/37/2/316.full).

*   Received for publication March 22, 2023.
*   Revision received November 16, 2023.
*   Accepted for publication November 27, 2023.

## References

1.  1.Meigs SL, Solomon M. Electronic health record use a bitter pill for many physicians. Perspect Health Inf Manag 2016;13:1d.
    
    

2.  2.Arndt BG, Beasley JW, Watkinson MD, et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med 2017;15:419–26.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTUvNS80MTkiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzM3LzIvMzE2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

3.  3.Penson DF. Re: US physician practices spend more than $15.4 billion annually to report quality measures. J Urol 2016;196:1233–4.
    
    

4.  4.Cohen DJ, Dorr DA, Knierim K, et al. Primary care practices' abilities and challenges in using electronic health record data for quality improvement. Health Aff (Millwood) 2018;37:635–43.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1377/hlthaff.2017.1254&link_type=DOI) 

5.  5.Hemler JR, Hall JD, Cholan RA, et al. Practice facilitator strategies for addressing electronic health record data challenges for quality improvement: EvidenceNOW. J Am Board Fam Med 2018;31:398–409.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMzEvMy8zOTgiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzM3LzIvMzE2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

6.  6.Nutting PA, Crabtree BF, Miller WL, Stange KC, Stewart E, Jaén C. Transforming physician practices to patient-centered medical homes: lessons from the national demonstration project. Health Aff (Millwood) 2011;30:439–45.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjMwLzMvNDM5IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8zNy8yLzMxNi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

7.  7.Wolfson D, Bernabeo E, Leas B, Sofaer S, Pawlson G, Pillittere D. Quality improvement in small office settings: an examination of successful practices. BMC Fam Pract 2009;10:14.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1186/1471-2296-10-14&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=19203386&link_type=MED&atom=%2Fjabfp%2F37%2F2%2F316.atom) 

8.  8.Goldberg DG, Mick SS, Kuzel AJ, Feng LB, Love LE. Why do some primary care practices engage in practice improvement efforts whereas others do not? Health Serv Res 2013;48:398–416.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1111/1475-6773.12000&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=23034072&link_type=MED&atom=%2Fjabfp%2F37%2F2%2F316.atom) 

9.  9.Heisey-Grove D, Danehy LN, Consolazio M, Lynch K, Mostashari F. A national study of challenges to electronic health record adoption and meaningful use. Med Care 2014;52:144–8.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1097/MLR.0000000000000038&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=24309669&link_type=MED&atom=%2Fjabfp%2F37%2F2%2F316.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000331087100010&link_type=ISI) 

10. 10.Coleman KF, Krakauer C, Anderson M, et al. Improving quality improvement capacity and clinical performance in small primary care practices. Ann Fam Med 2021;19:499–506.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTkvNi80OTkiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzM3LzIvMzE2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

11. 11.Parchman ML, Fagnan LJ, Dorr DA, et al. Study protocol for “Healthy Hearts Northwest”: a 2 × 2 randomized factorial trial to build quality improvement capacity in primary care. Implement Sci 2016;11:138.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1186/s13012-016-0502-7&link_type=DOI) 

12. 12.The Office of the National Coordinator for Health Information Technology, Centers for Medicare & Medicaid Services, Department of Health and Human Services. *Electronic Clinical Quality Improvement (eCQM) Resource Center*. Ischemic Vascular Disease (IVD): use of aspirin or another antithrombotic. [https://ecqi.healthit.gov/sites/default/files/ecqm/measures/CMS164v7.html](https://ecqi.healthit.gov/sites/default/files/ecqm/measures/CMS164v7.html). Accessed July 19, 2023.
    
    

13. 13.The Office of the National Coordinator for Health Information Technology, Centers for Medicare & Medicaid Services, Department of Health and Human Services. *Electronic Clinical Quality Improvement (eCQM)Resource Center*. Controlling high blood pressure. Available at: [https://ecqi.healthit.gov/ecqm/ec/2021/cms165v9](https://ecqi.healthit.gov/ecqm/ec/2021/cms165v9). Accessed July 19, 2023.
    
    

14. 14.The Office of the National Coordinator for Health Information Technology, Centers for Medicare & Medicaid Services, Department of Health and Human Services. *Electronic Clinical Quality Improvement (eCQM) Resource Center*. Preventative care and screening: tobacco use: screening and cessation intervention. Available at: [https://ecqi.healthit.gov/ecqm/ec/2021/cms138v9](https://ecqi.healthit.gov/ecqm/ec/2021/cms138v9). Accessed July 19, 2023.
    
    

15. 15.Parchman ML, Anderson ML, Dorr DA, et al. A randomized trial of external practice support to improve cardiovascular risk factors in primary care. Ann Fam Med 2019;17:S40–S49.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTQ6IjE3L1N1cHBsXzEvUzQwIjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8zNy8yLzMxNi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

16. 16.Heisey-Grove D, Patel V. National findings regarding health IT use and participation in health care delivery reform programs among office-based physicians. J Am Med Inform Assoc 2017;24:130–9.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1093/jamia/ocw065&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=27185812&link_type=MED&atom=%2Fjabfp%2F37%2F2%2F316.atom) 

17. 17.Rittenhouse DR, Wiley JA, Peterson LE, Casalino LP, Phillips Jr RL. Meaningful use and medical home functionality in primary care practice. Health Affairs 2020;39:1977–83.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1377/hlthaff.2020.00782&link_type=DOI) 

18. 18.Ryan AM, Bishop TF, Shih S, Casalino LP. Small physician practices in New York needed sustained help to realize gains in quality from use of electronic health records. Health Affairs 2013;32:53–62.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjc6IjMyLzEvNTMiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzM3LzIvMzE2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

19. 19.Grumbach K, Bainbridge E, Bodenheimer T. Facilitating improvement in primary care: the promise of practice coaching. Issue brief (Commonwealth Fund) 2012;15:1–14.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=22712103&link_type=MED&atom=%2Fjabfp%2F37%2F2%2F316.atom) 

20. 20.Cykert S, Lefebvre A. Regional extension coordinators: use of practice support and electronic health records to improve quality and efficiency. North Carolina Medical Journal 2011;72:237–9.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoibmNtIjtzOjU6InJlc2lkIjtzOjg6IjcyLzMvMjM3IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8zNy8yLzMxNi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)