Abstract
Purpose: To understand how focused versus general practice facilitation can impact goal setting, action planning, and team performance in primary care transformation.
Background: Practice transformation in primary care is a crucial part of health reform, but can fatigue teams, leading to variable results. Practice facilitation may reduce primary care fatigue to help teams reach challenging transformation goals, but may require a more focused approach than previous studies suggest.
Methods: We performed a 12-month cluster randomized trial, during which 8 primary care clinics received practice facilitation. Four practices in the intervention arm received targeted facilitation to focus quality improvement (QI) goals on high-value elements (HVEs) intended to reduce cost and utilization, whereas 4 control practices received generalized QI facilitation. We investigated the impact of the targeted versus generalized approach on goal selection, action item selection and achievement, HVE attainment, and collaborative practice, using quantitative and qualitative methods.
Results: Intervention clinics selected an average of 7 goals and 29 action items, compared with 8 goals and 40 action items among controls. Eighty-three percent of intervention goals were related to HVEs, compared with 27% of goals among controls. Intervention clinics selected 101 HVE goals and met 68%, while controls selected 41 and met 61%. Analysis of pre-post practice surveys indicated greater improvement among intervention across 4 of 8 domains of collaborative practice.
Conclusion: Targeted facilitation may be more effective than a generalized approach to support practices in reaching high-value change goals, as well as fostering improvement of team focus on goals, roles and responsibilities.
Primary care practice transformation is a cornerstone in health care reform due to its broad population reach and longitudinal focus on patient needs; however, selecting QI goals and developing QI infrastructure to best achieve improved population health, better patient experience of care, and improved efficiency can be challenging and fatiguing for primary care practices.1⇓⇓–4 As more practices work to transform into patient-centered medical homes and prepare for alternative payment models (APMs), it is crucial to understand how practice facilitation occurs and what aspects of facilitation could be effective in achieving focused practice changes to improve patient outcomes. This study aims to assess the impact of general versus targeted practice facilitation on a group of Oregon primary care practices' QI by reporting intermediate measures of goal selection, team participation in goal and action item selection, action item achievement, and time to achieve goals.
Previous studies have demonstrated that practice facilitation guidance has been used to successfully assist with transformation efforts and system-level change.5,6 Practice facilitation provides strategies and frameworks to increase awareness of needed change, identify leadership and improve team function, emphasize local context, and monitor transformation progress.7,8 In brief, the practice facilitator (PF) interacts directly with practices to find a way forward toward transformation goals and to support tailored implementation of evidence-based interventions or guideline recommendations. A randomized controlled trial of 23 primary care practices demonstrated that practices with a PF compared with those without were more likely to define improvement objectives, successfully complete projects,9 and avoid change fatigue.10 Recent efforts to understand how PFs are effective draw from organizational learning theory and build on the learning-performance link for adult learners.11
PFs commonly support practices with general QI strategies to improve prevention services, practice efficiency, or disease-specific outcomes.12 Within these generalized QI interventions, the PF provides a structure for QI processes, and practices work collaboratively with the PF to identify and select activities appropriate for improvements. Results of generalized QI with a PF are mixed: some studies have shown increases in prevention services such as mammography13; implementation of evidence-based guidelines for treatment of asthma,14 and outreach to promote health screening services,15,16 while others have shown no effect.14,15,17 Limited work has explored whether practice facilitation improves broader health reform goals, such as reduction in health care utilization and cost for patients with complex needs; Nutting10 showed that while practices had more change energy with practice facilitation, facilitation did not affect cost and utilization outcomes. Some studies have tested methods of practice facilitation to understand what aspects are most effective.18⇓–20 Targeted or focused practice facilitation where the PF introduces a set of key improvements through a change package and promotes predefined QI activities likely to achieve key driver goals might be particularly effective in assisting practices to achieve cost and utilization outcomes. Parchman et al21 demonstrated that targeted practice facilitation resulted in significant and sustained improvements while implementing goals derived from the Chronic Care Model. With increasing movement toward APMs, testing practice facilitation under incentive models is a question for study as it is unknown whether incentives reduce practice willingness to engage across improvement not directly tied to payments.
We hypothesized that use of a predefined menu of goals in conjunction with monthly practice facilitation would result in greater progress toward achieving QI goals. We further hypothesized that clinics receiving targeted facilitation would be more likely than the clinics receiving the general approach to improve team function without losing their “change energy” or adaptive reserve.22 Our study setting is the Transforming Outcomes for Patients through Medical home Evaluation and reDesign (TOPMED) trial23 and this is a focused presentation on the PF's impact on goals and action planning.
Methods
TOPMED was funded by the Gordon and Betty Moore Foundation from 2011 to 2015 to understand how practice facilitation, performance incentives, and information technology (IT) support could be better utilized as practice transformation tools to reduce avoidable health care costs and service utilization in primary care. The complete TOPMED protocol23 and preliminary results on patient satisfaction24 are published elsewhere. In brief, 8 primary practices in Oregon participated in this year-long, 2-arm, pragmatic cluster randomized trial and received general or targeted practice facilitation, IT-based milestone reporting, and financial incentives based on achievement of QI goals.
Participating practices were recruited from member and affiliated practices of the Oregon Rural Practice–based Research Network (ORPRN).25 The study team recruited physician champions and practice leadership via phone and e-mail and eligible practices were engaged in health reform, had achieved Oregon's Patient-Centered Primary Care Home (PCPCH) standards at level 3 (out of 3), were willing to receive practice facilitation, and were willing to use a population registry management health information technology (HIT) tool, the Integrated Care Coordination System (ICCIS).26 A single, masters-level PF trained in QI methodology and practice transformation supported all 8 participating practices, and practices had variable experience both with QI and in working with an external PF. The study Principle Investigator (PI; DD), ORPRN Director (LJF), and PF traveled to practices to finalize participation details and to complete the study memorandum of understanding. Enrolled practices were matched based on size, location, ownership, and proportion of high-risk patients, followed by matched randomization using computer algorithms completed simultaneously.
Study Protocol and Data Collection
The study protocol is described below, including data collection and monitoring steps. The Oregon Health & Science University (OHSU) Institutional Review Board reviewed and monitored the study. Data sources for this study include field notes and goal tracking documentation from practice facilitation, and practice surveys assessing team development and collaborative practice.
Practice Facilitation and Goal Tracking
Targeted practice facilitation was provided to the 4 intervention practices that focused on the achievement of selected benchmarks associated with improved cost and utilization in within the medical home model. These benchmarks, or “high-value elements” (HVEs), were a subset of PCPCH criteria determined from a literature review and structured expert input to have greatest likelihood to reduce cost and utilization.27 Each intervention practice received encouragement to specifically work toward the HVE change package. The 4 control practices received general QI guidance from the PF. All participating practices received an equal payment across the study for the trial period; however, a portion of the intervention practice's payment was reserved for meeting HVEs to mimic payments for meeting quality and utilization metrics in some APMs.
Before study initiation, the PF met with each practice QI implementation team to establish the TOPMED QI structure and to document practice features relevant to a transformation environment. In April 2013, practice QI implementation teams from all 8 practices attended an in-person training session to receive study details, results of baseline staff surveys, and an orientation to the IT tools. Starting in May 2013, the PF held a minimum of 12 1-hour team meetings with each clinic over the course of the project, and conducted a meeting at project conclusion to evaluate the final set of goals and plan for sustainability. During monthly PF visits, practice QI implementation teams reviewed a customized dashboard displaying practice data coupled with PF-led consensus building to select goals and create small tests of change to accomplish goals. Monthly meetings were audio recorded to assist in documentation of activities. The purpose of the PF was to encourage and record goal setting and actions related to goals in addition to documenting achievement of HVEs and communicating with practices about study progress.
The PF created semistructured field notes capturing progress toward goals and meeting details, including team participation and goal selection, description of goals and action items for the upcoming month, and communications with implementation team members during and between monthly meetings. The project team defined “goals” as the long-term, overarching objectives that practices worked on for at least 1 month, whereas “action items” are monthly task assignments to individuals or team members generally expected to be completed by the next PF meeting. The PF compiled a database summarizing details of monthly visits that included QI goal drivers, contextual information, plans for the current month, progress toward goals, and action item outcomes from the previous month. During the 12 months of facilitation, an OHSU evaluation team with expertise in primary care, practice facilitation, and data-driven QI reviewed field notes and worksheets, and met with the PF monthly to evaluate progress and make joint recommendations for activities in subsequent months.
Achievement of HVEs
There are a total of 12 HVEs with 1 to 3 achievement levels each, yielding 26 total measures (see Table 1). For example, the first level of the care plan utilization HVE requires practices to report data on high-risk patients, whereas levels 2 and 3 require practices to provide care plans for more than 25% of high-risk patients and 50% of high-risk patients, respectively. For our analysis, HVE achievement is measured by progress toward all 26 total measures and is summed for up to 26 HVEs achieved cumulative for allocation arm, or 104 points per arm.
Goals and Action Items
All practices were measured monthly on their achievements in HVEs using a PCPCH Attestation Tool module in ICCIS with both the standard PCPCH measures and HVEs. Measurement consisted of 2 phases; first, electronic health record (EHR) data were drawn into the tool and measured for each quantitative element. Second, the PF met with practice staff for 1 to 2 hours to explain additional elements and assist staff with assessing and calculating measures. Practices also reviewed the quantitative measurements and were allowed to provide more specific evidence to adjust these calculations. Both arms used the HIT tool ICCIS, which was designed for care management tracking and measured clinical data to report progress toward HVEs.
Incentives and Feedback
Incentives were calculated based on goal selection and attainment. Specifically, control clinics received a percentage score based on the amount of effort they put into goal selection and attainment, and this score was averaged and translated into the quarterly payment of up to $10,000. The intervention clinics were given a score based on goal selection; this score was augmented by the proportion of HVEs attained. This score was also translated into a monthly payment which had to average $10,000 over 4 quarters, but could provide up to a 10% bonus for high achievement. Incentives at this level are only large enough to cover staff time and data collection costs.
Clinician and Staff Surveys: Team Development and Adaptive Reserve
We used questionnaires to understand the impact of practice facilitation on team function, adaptive reserve (or ability to continue transformation efforts), and collaboration; practice facilitation has been shown to enhance these aspects in transformation, but focused PF may worsen effects by being too directive. At baseline and post intervention, clinicians, and staff members completed 3 surveys: the Team Development Measure (TDM), the Clinician and Staff Questionnaire (CSQ), and the Collaborative Practice Assessment Tool (CPAT). The TDM measures a clinic's ability to function as a team, including cohesiveness, communication, and role and goal clarity.28–29 The questionnaire contains 31 items with 4-point ordinal ratings (Strongly Disagree to Strongly Agree) and demonstrates good reliability. The scores are summed and converted to a 0 to 100 score, with higher scores representing more team development components firmly in place.29 The CSQ is a validated assessment that measures attributes important for understanding practices in the process of change.12,17 It contains 38 questions with 5-point ordinal ratings (Strongly Disagree to Strongly Agree) across 5 domains: 1) adaptive reserve (ability to make and sustain change), 2) community knowledge, 3) health IT integration, 4) cultural sensitivity, and 5) patient safety culture. The CPAT is a validated tool used in health-care settings to measure the degree to which team members perceive that they collaborate effectively.30 It includes 56 items with a 7-point ordinal rating (Strongly Disagree to Strongly Agree) across 8 domains: 1) mission, meaningful purpose, goals; 2) general relationships; 3) team leadership; 4) general role responsibilities, autonomy; 5) communication and information exchange; 6) community linkages and coordination of care; 7) decision making and conflict management; and 8) patient involvement.
Analysis
Qualitative
After the intervention, 2 qualitative research personnel reviewed the PF's field notes. Using the recorded text of the item, each action item was assigned to a general QI goal or HVE. Specifically, if the description matched the HVE explicitly, it was assigned the HVE designation; all others were designated as a general goal. From the field notes, the clinic staff member who selected the goal was noted, and action items were tracked month to month for completion. Coded action items were tabulated for each HVE or QI goal and month in which they occurred. These results were charted to understand each practice's progress toward achieving goals and to compare intervention and control practices.
Quantitative
Descriptive statistics were calculated on numbers of goals set and goals achieved, numbers of action steps set, HVE achievement, and on each individual clinician and staff survey scale. HVE achievement was calculated as number of 26 achieved at baseline, at each quarter, and end of study. We performed difference-in-difference analyses to compare pre- and post-trial goals, action plans, HVE achievement, and survey results between intervention and control clinics.
Multivariable models adjusted standard errors for clustering at the clinic level using the SAS Surveyreg procedure, which can handle complex survey sample designs, including designs with stratification, clustering, and unequal weighting. The procedure fits linear models for survey data and computes regression coefficients by generalized least-squares estimations and their variance-covariance matrix. Surveyreg computes the regression coefficient estimators by generalized least-squares estimation and computes variances of the regression parameters using Taylor series linearization. For each of the survey composite scores, we fit a probability model with the computed survey score as the unit of analysis. We assessed differences in change from pre- to post-trial between intervention and control clinics using the following regression model: survey outcome = clinic + intervention group + time + (intervention × time). A statistically significant interaction term (P < .05) indicates difference in difference between intervention and control clinics.
Results
A total of 8 practices participated in the study, were randomized, and received 12 months of practice facilitation from May 2013 to April 2014. Table 2 displays the descriptive statistics by clinic in their paired randomization group. The clinics ranges in size from 7,257 to 14,119 empaneled patients, with between 4 and 13 provider full-text equivalents (FTE) per clinic. The clinics were predominantly fee-for-service, with <20% of payments from other forms, such as capitation or quality/care management fees. Insurance status of the patients was mixed, with Medicare ranging from 16% to 34%, Medicaid from 5% to 42%, and the rest commercially insured. For the patient panels, range of average ages was 39.7 to 52.5 years, with between 1.3% to 5.6% of patients above 85 years. Gender of patients in the clinics ranges from 56.8% to 64.4% female. Hierarchical Condition Category (HCC) scores in the clinic ranged from 0.87 to 1.09. To accommodate differences between clinics, a stratified matching approach was used for intervention assignment and for patient level analyses.
High Value Element (HVE) Descriptions and Clinic Selection of HVEs
Goal Selection and Action Items
Goals were selected monthly. Examination of both general QI goals and HVE goals showed nearly an equal number set across both arms (7.3 goals for intervention and 7.5 for control), as shown in Table 3. Each practice in the intervention and control arms selected goals related to HVEs, and intervention practices selected a higher percentage of HVE goals (82.8%) than control (26.7%). Intervention practices worked on HVE goals for an average of 2.7 months and on other goals an average of 2.2 months. Control practices worked on HVE goals an average of 2.4 months and other QI goals an average of 2.1 months. Overall intervention practices selected a total of 101 HVE goals and met 68% while control practices selected 41 HVE goals and met 61%. Control practices set more individual action items (mean = 40) to achieve goals than intervention practices (mean = 28.75). Clinician implementation team participants within intervention practices led selection of 30% of action items compared with 20% in the control arm.
Practice Characteristics of Intervention and Control Clinics
Achievement of HVE Goals
Table 4 shows selection and progress toward achieving 12 HVE topics. Intervention practices selected goals related to 9 HVE topics. None of the intervention practices selected goals related to after hours access, clinical information exchange or care coordination outreach. Although HVEs were not specifically introduced to control practices, they independently selected goals related to 7 HVEs: tracking third next available appointment, utilization follow-up, utilization prevention, performance data utilization, care coordination outreach, care plan utilization, and advance directive utilization.
Clinician and Staff Survey Results Before and After TOPMED Trial
Intervention practices showed a marked increase in achieving HVE elements at project end (Figure 1). At baseline, intervention practices had a cumulative score of 38 HVE elements achieved compared with control practices at 27 HVE elements. At quarter 4, intervention practices doubled to 76 HVE elements achieved (average 19 per practice, or 73% of maximum) while baseline practices increased to 44 HVE elements (11 per practice or 42% of maximum).
Summed HVE selection and achievement by arm.
Survey Results
Clinician and staff survey results were mixed. Response rate for the TDM, measuring team function and cohesiveness, was 80.4% pretrial (N = 137) and 43.5% post-trial (N = 83). Scores increased for the intervention relative to the control clinics (+2.38/100 difference-in-difference), but this was not statistically significant (P = .312). CSQ response rates were 92.3% pretrial (N = 210) and 60.4% post-trial (N = 104). Intervention and control clinics performed similarly on 4 out of 5 domains, while control clinics demonstrated statistically significant (P = .030) improvement in community knowledge, compared with the intervention clinics (+.38/5.0 difference-in-difference). Response rates for the CPAT were 71.3% pretrial (N = 119) and 44.0% post-trial (N = 101). The intervention clinics performed significantly better relative to controls (P < .05) on 4 of the 8 domains, including Mission, Meaningful Purpose, Goals (+.61/7.0 difference-in-difference); Team Leadership (+.85/7.0 difference-in-difference); General Role, Responsibility, Autonomy (+.63/7.0 difference-in-difference); and Communication and Information Exchange, and Coordination of Care (+.58/7.0 difference-in-difference).
Discussion
TOPMED compared the ability of targeted practice facilitation coupled with a change package to that of generalized QI in driving clinics toward more effective components of practice transformation. Practices in the intervention arm selected HVE goals consistently over the 12 months of the project, continued to progress across 9 of the 12 HVE topics, and sought to reach increasingly challenging clinical benchmark tiers within each HVE. From baseline to quarter 4, intervention practices doubled their achievement of measurable HVE elements from 38 to 76 (of 104). Control practices progressed from 27 to 44 over the same time period. Staff survey results on team function and ability were mixed and did not demonstrate a lot of change; however, the intervention arm performed better on 2 of 3 surveys.
HVE goals challenge practices to make changes that might reduce cost and utilization. It seemed likely that practices selecting HVEs would require more time to achieve the goals and would accomplish fewer goals overall because of their difficulty. Notably, intervention and control practices achieved nearly the same number of goals over the course of the project. Our review of action items showed clinician members of implementation teams were more involved in selecting improvement goals when chosen from the HVE menu set. Practices were able to achieve HVEs with fewer action items. This could be because the PF was able to efficiently guide implementation teams to focus on the most effective actions to undertake for HVEs, or it could reflect the complex or nonlinear nature of selected non-HVE goals. Field notes show the PF actively spread success ideas from 1 practice to another in HVE work. This combination of rapid and efficient goal progression, clinician involvement and achievement of increasingly challenging benchmarks supports the effectiveness of a high-value menu set in conjunction with targeted practice facilitation in reaching goals most likely to impact practice improvement.
As the field of practice facilitation grows, there is a need to identify and document those strategies that best enable practices to achieve and maintain transformation. To understand correlations between improvement activities and external factors like practice facilitation requires in-depth analysis. Other studies demonstrate the effectiveness of continuous QI compared with a reflective adaptive process18 and the use of practice-tailored facilitation. In TOPMED, we found that use of a targeted HVE change package across 12 noncondition-specific practice improvement elements enabled practices to set and achieve valued improvement goals. Coding goals and action items allowed the research team to measure progress across the diverse practice improvement elements that were selected, and may be useful for similar projects.
The study has several limitations. The pragmatic design of the trial makes it more difficult to determine trial effectiveness. Many of the practices were contemporaneously involved with other QI initiatives and programs. The small sample size and recruitment approach made it difficult to match practice pairs, and the intervention arm had already achieved more HVEs at baseline compared with control. The practices themselves also underwent changes that may have diverted them from TOPMED efforts, including transitions in ownership and changes to EHRs. The duration of the practice facilitation was limited to 12 months and studies of practice change show that transformation is uneven, unstable, and takes time.31 It is challenging to understand the impact financial incentives had on practice engagement. However, as the goals for QI changes are the outcome of interest, we were able to demonstrate whether these more challenging goals could be met. Finally, the post-trial survey had a low response rate from a different distribution of respondents, making it difficult to compare with the baseline surveys.
Conclusion
As predicted, intervention clinics worked on and completed more HVEs than control clinics regardless of the difficult nature of HVEs. Targeted practice facilitation was effective in assisting practices to achieve HVEs, and intervention and control practices achieved the same number of goals. Intervention practices were more efficient in action item selection. Targeted facilitation improved care team function and collaboration. Overall, the TOPMED trial supports the effectiveness of combining a menu set with targeted practice facilitation in reaching HVE goals. Because HVEs have demonstrated capacity to improve the Triple Aim, working on them may improve patient care and reduce cost and utilization.
Acknowledgments
The authors would like to extend special acknowledgment to Beth Sommers, Melinda Davis and Sonya Howk for their contributions to this study.
Notes
This article was externally peer reviewed.
Funding: Gordon and Betty Moore Foundation Grant ID: GBMF2908; AHRQ R18 HS17832 (Enhancing Complex Care Coordination Information Systems).
Conflict of interest: none declared.
To see this article online, please go to: http://jabfm.org/content/30/5/572.full.
- Received for publication February 3, 2017.
- Revision received February 3, 2017.
- Accepted for publication February 3, 2017.
