Abstract
Background: Engaging primary care practices in quality improvement (QI) efforts has been challenging. Literature provides little guidance on the engagement of small to medium-sized practices in QI. This study examined the association between practice readiness and practice characteristics and engagement during a targeted QI effort.
Methods: The study analyzed cross-sectional data collected by the Heart of Virginia Health care, a cardiovascular disease QI intervention study with 195 practices. Data sources include 1) coach-assessed practice engagement in 7 domains (outcome), 2) surveys of readiness completed by 2529 clinicians and staff, a response rate of 86%, and 3) surveys of practice characteristics completed by a physician leader or practice manager. We used descriptive statistics and ordered logit regression for the analysis.
Results: Associations between readiness and engagement were statistically significant for clinician engagement (odds ratio [OR] = 5,74; 95% CI, 1.79-18.42; P = .003) and leadership engagement (OR = 3.19; 95% CI, 1.10-9.24; P = .032). Adjusting for covariates, being a hospital-owned practice was associated with a lower level of clinician engagement (OR = 0.35; 95% CI, 0.16-0.76; P = .009) relative to independent practices.
Discussion: Our study highlights the importance of clinician and leadership engagement as drivers of practice readiness to change in a QI effort. Lack of clinician engagement in hospital-owned practices could be driven by other factors such as burnout that need to be explored in future studies.
Conclusions: Clinicians and leadership involvement in QI efforts is critical. The findings suggest that QI plans should involve clinicians and leaders early in the process to foster commitment, establish practice readiness, and sustain improvement efforts.
- Cross-Sectional Studies
- Leadership
- Primary Health Care
- Quality Improvement
- Surveys and Questionnaires
- Virginia
Introduction
As health care organizations go through major changes to improve quality, engaging primary care practices in improvement activities has been vital and challenging.1,2 Limited research has assessed the engagement of small to medium-sized primary care practices during quality improvement (QI) efforts. One recent study reveals that time and resource constraints make it challenging to keep small practices engaged during QI efforts.3 Other reasons for a lack of practice engagement include a perception of the cost of change exceeding the anticipated benefit4 and a conflict in leadership priorities or values set by the physicians.5 Practice disengagement may also result from a lack of leadership support and a nonsupportive practice culture or organizational structure.1,4,5
Leaders play a major role in establishing a high-performing practice culture before starting a change process.6 To establish a culture open to a change, leaders should ensure that there is effective communication in the practice, and staff is engaged in QI processes.7 Since physician leaders in clinical care are major influencers of transformation efforts, physician readiness and commitment to QI efforts are critical. Previous studies have lacked examining the influence of readiness on small to medium-sized primary care practice engagement during a QI effort.
Weiner’s work on organizational change resulted in a theory on organizational readiness.8 According to readiness theory, QI is most likely to occur when practice members feel committed to a change and perceive they have the ability to implement that change.8,9 When organizational readiness is high, the organization members are more likely to accept a change and engage in implementation activities.10 To test this theory in the context of primary care practices, we studied the relationship between practice readiness and engagement among small to medium-sized primary care practices that participated to the Heart of Virginia Healthcare (HVH) project.
Methods
Setting
The HVH was 1 of 7 national collaboratives funded as part of the EvidenceNOW initiative of the Agency for Healthcare Research and Quality.11 The objective of the HVH was to help small to medium-sized primary care practices in Virginia to improve population cardiovascular care.11 Practices participating in the intervention received in-person coaching support for the first 3 months, followed by a 9-month continuing support phase.11 The goal of the coach support was to help practices better identify patients at risk for cardio-vascular disease and to mitigate those risks.12 Practice recruitment and sampling strategy for the HVH are explained elsewhere.13 The study included 195 small to medium-sized primary care practices, defined as having up to 10 to 15 clinicians.14 We obtained approval from the Institutional Review Board of George Mason University in 2016.
Data Sources
This study used primary data collected by the HVH project in 2016 and 2017. Data were derived from the 3 sources: 1) Coaches assessed12 practice engagement across 7 domains at baseline using a standardized instrument developed for this purpose.11 2) Clinicians and staff assessed practice readiness on a 5-point Likert scale based on a member survey.9 The survey was administered at baseline, near the start of the HVH intervention, and filled out by all practice members, including physicians (MD, DO), nurse practitioners (NP), nurses, practice managers, and nonclinician staff. We calculated the individual-level survey response rate by using the total practice size. 3) Finally, we collected a self-reported practice survey which measured practice characteristics (Table 1). The survey was filled out by the physician leader or practice manager at baseline. Both surveys were administered online.
Baseline Sample Characteristics, Heart of Virginia Healthcare, 2016-2017 (n = 195)
Outcome Measure
The outcome measure was practice engagement across 7 domains (Table 2). We asked coaches to determine the level of practice engagement in 7 areas: 1) responsiveness to the external support, 2) overall practice response at coach visit, 3) overall team assessment for the HVH activity, 4) assessment of site leadership, 5) team engagement, 6) clinician engagement, and 7) staff engagement.11 Engagement was assessed using 3 response categories: 3 = “high,” 2 = “moderate,” and 1 = “low” (Figure 1). Some of the engagement questions that included a response category of “N/A” (no one present or could not assess) or “unknown” were removed from the data during the analysis.
Descriptive results: engagement across 7 domains. Notes: An ordered scale is from 1 to 3 (3 was the highest, 1 was the lowest response). Sample varies due to the removal of “N/A” or “Unknown” response categories. Responsiveness to external support, overall team assessment, leadership assessment, and team engagement (n = 195). Overall practice response at coach visit (n = 184), staff engagement (n = 162), and clinician engagement (n = 152). Clinician includes MD, DO, NP, PA. Staff includes RN, LPN, MA, front desk, back office, billing, practice manager. Clinician vs staff engagement: Spearman’s correlation coefficient rank test: P = .0001.
Engagement Questionnaire Items*
Independent Variables
Our key independent variable was readiness. An 11-item readiness scale specifically developed for this purpose,11 measured readiness (see Figure 2). The survey respondents were asked to choose 1 of the 5 response categories (5 = “strongly agree” to 1 = “strongly disagree”). The Cronbach α for the eleven-item scale was 0.80. For the readiness composite score, we first calculated the mean score for each of the readiness item at the individual level. Then, we aggregated the individual-level responses to the practice level, and finally took the overall mean practice-level response. We treated the readiness composite score as a continuous measure and included the measure in the regression models.
Readiness to change (mean score) by practice, Heart of Virginia Healthcare, (n = 195). Notes: Readiness was measured by the “readiness” questionnaire11 with a 5-point Likert scale instrument (5 = Strongly agree to 1 = Strongly disagree). The survey was conducted at baseline.
Our covariates included several practice level characteristics: practice ownership (independent, hospital owned, or federally qualified health centers),13⇓–15 practice size (1 to 5 clinicians, 6 to 10 clinicians, 11 and above clinicians),13⇓–15 and practice specialty-mix (single specialty or multispecialty practice).15 We included whether the practice was recognized as a patient-centered medical home (PCMH) or the practice was part of an accountable care organization (ACO) and whether the practice was located in a medically underserved area.13,15 We also controlled for the percentage of patients in the practice who were 60 years and above, and whether or not the practice reported participating in the Million Hearts project.16 We chose these control variables in our study based on previous research.13⇓–15
Statistical Approach
We computed descriptive statistics and performed multivariate analysis of the outcomes using ordered logit, proportional odds models. We assessed the relationship between practice engagement and readiness, controlling for practice characteristics. We ran separate regression models for each of the 7 engagement domains. We also conducted several robustness checks. First, we examined the influence of potentially competing initiatives, including the Million Hearts project or whether the practice was recognized as a PCMH or part of an ACO. Finally, we tested the proportional odds assumption. The chi-squared test was 5.06 with P = .89, which showed that the assumption of equal proportional odds was satisfied. All analyses were conducted using STATA software.17
Results
Study Population
The study population included a total of 195 small to medium-sized primary care practices that responded to the survey: 61 independent, physician-owned practices, (32%), 110 hospital-owned practices (56%), and 24 federally qualified health centers (12%). Our response rate for the member survey was 86%. A total of 2529 individuals from 195 practices responded to the member survey. The response rate for the practice survey (n = 195) was 96%. Table 1 provides sample characteristics of the small to medium-sized primary care practices in Virginia.
Descriptive Statistics
Figure 1 displays the results across 7 engagement areas. Practice engagement in responsiveness to external support was determined to be frequently high (42%) or moderate (38%). Overall practice engagement in response to coach visits was also high (78%). On the other hand, almost two thirds of participating practices (61%) at baseline demonstrated low engagement with HVH related activities. More than two thirds of the practices (66%) were observed to have moderate leadership engagement during the HVH project, only 24% had high, and 10% had low engagement. Team engagement was determined to be high (41%) or moderate (37%) for most of the participating practices, only 22% had low engagement. Staff engagement was higher (51%) compared with clinicians (38%). When we included the high and moderate scores, the staff engagement was still higher than the clinicians (86% vs 82%). The Spearman’s correlation coefficient rank test indicated a statistically significant association between clinician engagement and staff engagement, P = .0001.
The mean readiness composite score was 3.83 using a 5-point Likert scale. Figure 2 provides the responses to individual readiness items to highlight which components contributed to greater readiness. Among the readiness components, the mean score was highest for 1) “We can support providers as they adjust to these changes” (mean score = 3.97), 2) “We want to implement these changes” (mean score = 3.91), and 3) “We are committed to making these changes” (mean score = 3.87). The lowest 3 mean scores included: “We can manage the politics of these changes” (mean score = 3.72), “We can keep track of progress in implementing these changes” (mean score = 3.78), and “We will do whatever it takes to implement these changes” (mean score = 3.79). Most responses fell into the “strongly agree” to “agree” categories.
Regression Analysis
Results from multivariate ordered logit models are presented in Table 3. Controlling for other factors, readiness was positively associated with higher levels of engagement for “response to external support,” “response at coach visit,” “leadership engagement,” “clinician engagement,” and “staff engagement.” Of these, the results were statistically significant for “clinician engagement” (P = .003) and “leadership engagement” (P = .032). For the other engagement outcomes, “team assessment” and “team engagement,” the association of readiness and level of engagement was negative, and results were not statistically significant. Thus, readiness was a significant, positive predictor of “leadership engagement” and “clinician engagement.”
Adjusted Multivariate Ordered Logistic Regression (Odds Ratios and 95% CIs): Association of Practice Engagement and Practice Readiness and Practice Characteristics
Table 3 also displays the estimated incremental differences of hospital ownership relative to independent practices at each engagement level. Adjusting for other covariates, being a hospital-owned practice was associated with a lower level of clinician engagement (odds ratio = 0.35; 95% CI, 0.16-0.76) relative to independent practices, and the association was statistically significant (P = .009). However, there were no statistically significant differences between ownership type, independent, hospital or federally qualified health centers, and across other engagement areas.
Practice size was statistically significant for “response to external support” (P = .02), “leadership engagement” (P = .001), and “team engagement” (P = .016). In addition, the association between practice size (11 and more clinicians) and practice readiness was also statistically significant (P = .001). Further, practice designation as a PCMH was a positive and statistically significant predictor of “leadership engagement” (P = .04). Finally, being part of an ACO was negatively associated with all outcomes, except team assessment which was close to an odds ratio of 1.
Discussion
We found leadership engagement and clinician engagement were associated with practice readiness. Consistent with previous research, our finding highlights the importance and influence of leadership4,5 and clinician involvement18,19 with practice QI efforts. The finding suggests that QI plans should involve clinicians and leaders in improvement efforts to establish practice readiness and maintain ongoing quality.
We also found practice ownership plays an important role in practice engagement. Interestingly, hospital-owned practices had a lower level of clinician engagement relative to independent practices. Previous studies examined the influence of practice ownership on various practice outcomes and revealed conflicting results.13⇓–15 For example, recent studies found hospital-owned practices had a lower level of change capacity score14 and lower level of QI strategy implementation15 compared with independent practices. Another study found more favorable ratings of the work environment, burnout, and psychological safety in hospital-owned practices compared with independent practices.13 Although effective strategies are needed for engaging clinicians in QI activities, our finding of a lower level of clinician engagement in hospital-owned practices may indicate that corporate could be centralizing their resources devoted to QI activities. Further, a lack of clinician engagement could be driven by other influences such as time scarcity, overwhelming bureaucratic tasks, or burnout20⇓–22 that should be explored in future research.
Our study demonstrated that practice size, specifically larger practices with 11 and more clinicians, had a positive association with readiness and engagement. The finding reflects that smaller practices may need additional support for QI activities, including more upfront preparation and longer face-to-face support from external facilitators. This finding also suggests that providers should focus on removing barriers, especially for small practices, before start implementing a change.23
We found a practice designation of a PCMH was a positive and significant predictor of leadership engagement. The finding was consistent with the structure of a PCMH, in which a strong leadership should be in place who can establish a high performing practice culture, set a common organizational goal, communicate clearly, and encourage team efforts for continuous improvement.24
Our study has several limitations. First, it might be possible that practices already doing well may have been more likely to participate in the study thus creating sampling bias. Second, this study analyzes self-reported data that might affect the study results due to “response bias.” Third, coaches’ knowledge and skillsets varied depending on their background, work experience, and their objectivity;25 this might have affected the assessment of the coaches on practice engagement. Fourth, it is very likely that practice engagement is influenced by many other factors (i.e., practice participation in multiple QI initiatives)14 that this study did not have the opportunity to analyze. Finally, the study was based on almost 200 primary care practices in Virginia willing to participate in HVH, but results may not generalize to other geographic areas or practices.
This study contributes to health services research by providing empirical evidence on small to medium-sized primary care practice engagement during a QI effort initiative. The findings provide guidance for QI authorities and institutions for successful intervention design and preparation. The study findings suggest that QI plans should involve clinicians and leaders early in the improvement process so that clinicians’ priorities and concerns are addressed, and commitment is obtained before the initiative takes place.
Notes
Conflicts of interest: AJK was the key investigator of the overall project funded by the Agency for Healthcare Research and Quality. TGS, AEC, and DGG report no conflicts of interest.
Funding: This research was funded by the Agency for Healthcare Research and Quality (R18HS023913; PI, AJK).
To see this article online, please go to: http://jabfm.org/content/34/1/40.full.
- Received for publication April 20, 2020.
- Revision received August 20, 2020.
- Accepted for publication August 24, 2020.
