A Randomized Trial of High-Value Change Using Practice Facilitation
===================================================================

* LeAnn Michaels
* Tracy Anastas
* Elizabeth Needham Waddell
* Lyle Fagnan
* David A. Dorr

## Abstract

*Purpose:* To understand how focused versus general practice facilitation can impact goal setting, action planning, and team performance in primary care transformation.

*Background:* Practice transformation in primary care is a crucial part of health reform, but can fatigue teams, leading to variable results. Practice facilitation may reduce primary care fatigue to help teams reach challenging transformation goals, but may require a more focused approach than previous studies suggest.

*Methods:* We performed a 12-month cluster randomized trial, during which 8 primary care clinics received practice facilitation. Four practices in the intervention arm received targeted facilitation to focus quality improvement (QI) goals on high-value elements (HVEs) intended to reduce cost and utilization, whereas 4 control practices received generalized QI facilitation. We investigated the impact of the targeted versus generalized approach on goal selection, action item selection and achievement, HVE attainment, and collaborative practice, using quantitative and qualitative methods.

*Results:* Intervention clinics selected an average of 7 goals and 29 action items, compared with 8 goals and 40 action items among controls. Eighty-three percent of intervention goals were related to HVEs, compared with 27% of goals among controls. Intervention clinics selected 101 HVE goals and met 68%, while controls selected 41 and met 61%. Analysis of pre-post practice surveys indicated greater improvement among intervention across 4 of 8 domains of collaborative practice.

*Conclusion:* Targeted facilitation may be more effective than a generalized approach to support practices in reaching high-value change goals, as well as fostering improvement of team focus on goals, roles and responsibilities.

*   Medical Home
*   Patient-Centered Care
*   Pay for Performance
*   2017 PBRN Issue
*   Practice Faciliation

Primary care practice transformation is a cornerstone in health care reform due to its broad population reach and longitudinal focus on patient needs; however, selecting QI goals and developing QI infrastructure to best achieve improved population health, better patient experience of care, and improved efficiency can be challenging and fatiguing for primary care practices.1⇓⇓–4 As more practices work to transform into patient-centered medical homes and prepare for alternative payment models (APMs), it is crucial to understand how practice facilitation occurs and what aspects of facilitation could be effective in achieving focused practice changes to improve patient outcomes. This study aims to assess the impact of general versus targeted practice facilitation on a group of Oregon primary care practices' QI by reporting intermediate measures of goal selection, team participation in goal and action item selection, action item achievement, and time to achieve goals.

Previous studies have demonstrated that practice facilitation guidance has been used to successfully assist with transformation efforts and system-level change.5,6 Practice facilitation provides strategies and frameworks to increase awareness of needed change, identify leadership and improve team function, emphasize local context, and monitor transformation progress.7,8 In brief, the practice facilitator (PF) interacts directly with practices to find a way forward toward transformation goals and to support tailored implementation of evidence-based interventions or guideline recommendations. A randomized controlled trial of 23 primary care practices demonstrated that practices with a PF compared with those without were more likely to define improvement objectives, successfully complete projects,9 and avoid change fatigue.10 Recent efforts to understand how PFs are effective draw from organizational learning theory and build on the learning-performance link for adult learners.11

PFs commonly support practices with general QI strategies to improve prevention services, practice efficiency, or disease-specific outcomes.12 Within these generalized QI interventions, the PF provides a structure for QI processes, and practices work collaboratively with the PF to identify and select activities appropriate for improvements. Results of generalized QI with a PF are mixed: some studies have shown increases in prevention services such as mammography13; implementation of evidence-based guidelines for treatment of asthma,14 and outreach to promote health screening services,15,16 while others have shown no effect.14,15,17 Limited work has explored whether practice facilitation improves broader health reform goals, such as reduction in health care utilization and cost for patients with complex needs; Nutting10 showed that while practices had more change energy with practice facilitation, facilitation did not affect cost and utilization outcomes. Some studies have tested methods of practice facilitation to understand what aspects are most effective.18⇓–20 Targeted or focused practice facilitation where the PF introduces a set of key improvements through a change package and promotes predefined QI activities likely to achieve key driver goals might be particularly effective in assisting practices to achieve cost and utilization outcomes. Parchman et al21 demonstrated that targeted practice facilitation resulted in significant and sustained improvements while implementing goals derived from the Chronic Care Model. With increasing movement toward APMs, testing practice facilitation under incentive models is a question for study as it is unknown whether incentives reduce practice willingness to engage across improvement not directly tied to payments.

We hypothesized that use of a predefined menu of goals in conjunction with monthly practice facilitation would result in greater progress toward achieving QI goals. We further hypothesized that clinics receiving targeted facilitation would be more likely than the clinics receiving the general approach to improve team function without losing their “change energy” or adaptive reserve.22 Our study setting is the Transforming Outcomes for Patients through Medical home Evaluation and reDesign (TOPMED) trial23 and this is a focused presentation on the PF's impact on goals and action planning.

## Methods

TOPMED was funded by the Gordon and Betty Moore Foundation from 2011 to 2015 to understand how practice facilitation, performance incentives, and information technology (IT) support could be better utilized as practice transformation tools to reduce avoidable health care costs and service utilization in primary care. The complete TOPMED protocol23 and preliminary results on patient satisfaction24 are published elsewhere. In brief, 8 primary practices in Oregon participated in this year-long, 2-arm, pragmatic cluster randomized trial and received general or targeted practice facilitation, IT-based milestone reporting, and financial incentives based on achievement of QI goals.

Participating practices were recruited from member and affiliated practices of the Oregon Rural Practice–based Research Network (ORPRN).25 The study team recruited physician champions and practice leadership via phone and e-mail and eligible practices were engaged in health reform, had achieved Oregon's Patient-Centered Primary Care Home (PCPCH) standards at level 3 (out of 3), were willing to receive practice facilitation, and were willing to use a population registry management health information technology (HIT) tool, the Integrated Care Coordination System (ICCIS).26 A single, masters-level PF trained in QI methodology and practice transformation supported all 8 participating practices, and practices had variable experience both with QI and in working with an external PF. The study Principle Investigator (PI; DD), ORPRN Director (LJF), and PF traveled to practices to finalize participation details and to complete the study memorandum of understanding. Enrolled practices were matched based on size, location, ownership, and proportion of high-risk patients, followed by matched randomization using computer algorithms completed simultaneously.

### Study Protocol and Data Collection

The study protocol is described below, including data collection and monitoring steps. The Oregon Health & Science University (OHSU) Institutional Review Board reviewed and monitored the study. Data sources for this study include field notes and goal tracking documentation from practice facilitation, and practice surveys assessing team development and collaborative practice.

#### Practice Facilitation and Goal Tracking

Targeted practice facilitation was provided to the 4 intervention practices that focused on the achievement of selected benchmarks associated with improved cost and utilization in within the medical home model. These benchmarks, or “high-value elements” (HVEs), were a subset of PCPCH criteria determined from a literature review and structured expert input to have greatest likelihood to reduce cost and utilization.27 Each intervention practice received encouragement to specifically work toward the HVE change package. The 4 control practices received general QI guidance from the PF. All participating practices received an equal payment across the study for the trial period; however, a portion of the intervention practice's payment was reserved for meeting HVEs to mimic payments for meeting quality and utilization metrics in some APMs.

Before study initiation, the PF met with each practice QI implementation team to establish the TOPMED QI structure and to document practice features relevant to a transformation environment. In April 2013, practice QI implementation teams from all 8 practices attended an in-person training session to receive study details, results of baseline staff surveys, and an orientation to the IT tools. Starting in May 2013, the PF held a minimum of 12 1-hour team meetings with each clinic over the course of the project, and conducted a meeting at project conclusion to evaluate the final set of goals and plan for sustainability. During monthly PF visits, practice QI implementation teams reviewed a customized dashboard displaying practice data coupled with PF-led consensus building to select goals and create small tests of change to accomplish goals. Monthly meetings were audio recorded to assist in documentation of activities. The purpose of the PF was to encourage and record goal setting and actions related to goals in addition to documenting achievement of HVEs and communicating with practices about study progress.

The PF created semistructured field notes capturing progress toward goals and meeting details, including team participation and goal selection, description of goals and action items for the upcoming month, and communications with implementation team members during and between monthly meetings. The project team defined “goals” as the long-term, overarching objectives that practices worked on for at least 1 month, whereas “action items” are monthly task assignments to individuals or team members generally expected to be completed by the next PF meeting. The PF compiled a database summarizing details of monthly visits that included QI goal drivers, contextual information, plans for the current month, progress toward goals, and action item outcomes from the previous month. During the 12 months of facilitation, an OHSU evaluation team with expertise in primary care, practice facilitation, and data-driven QI reviewed field notes and worksheets, and met with the PF monthly to evaluate progress and make joint recommendations for activities in subsequent months.

#### Achievement of HVEs

There are a total of 12 HVEs with 1 to 3 achievement levels each, yielding 26 total measures (see Table 1). For example, the first level of the care plan utilization HVE requires practices to report data on high-risk patients, whereas levels 2 and 3 require practices to provide care plans for more than 25% of high-risk patients and 50% of high-risk patients, respectively. For our analysis, HVE achievement is measured by progress toward all 26 total measures and is summed for up to 26 HVEs achieved cumulative for allocation arm, or 104 points per arm.

View this table:
[Table 1.](http://www.jabfm.org/content/30/5/572/T1)

Table 1. 
Goals and Action Items

All practices were measured monthly on their achievements in HVEs using a PCPCH Attestation Tool module in ICCIS with both the standard PCPCH measures and HVEs. Measurement consisted of 2 phases; first, electronic health record (EHR) data were drawn into the tool and measured for each quantitative element. Second, the PF met with practice staff for 1 to 2 hours to explain additional elements and assist staff with assessing and calculating measures. Practices also reviewed the quantitative measurements and were allowed to provide more specific evidence to adjust these calculations. Both arms used the HIT tool ICCIS, which was designed for care management tracking and measured clinical data to report progress toward HVEs.

#### Incentives and Feedback

Incentives were calculated based on goal selection and attainment. Specifically, control clinics received a percentage score based on the amount of effort they put into goal selection and attainment, and this score was averaged and translated into the quarterly payment of up to $10,000. The intervention clinics were given a score based on goal selection; this score was augmented by the proportion of HVEs attained. This score was also translated into a monthly payment which had to average $10,000 over 4 quarters, but could provide up to a 10% bonus for high achievement. Incentives at this level are only large enough to cover staff time and data collection costs.

#### Clinician and Staff Surveys: Team Development and Adaptive Reserve

We used questionnaires to understand the impact of practice facilitation on team function, adaptive reserve (or ability to continue transformation efforts), and collaboration; practice facilitation has been shown to enhance these aspects in transformation, but focused PF may worsen effects by being too directive. At baseline and post intervention, clinicians, and staff members completed 3 surveys: the Team Development Measure (TDM), the Clinician and Staff Questionnaire (CSQ), and the Collaborative Practice Assessment Tool (CPAT). The TDM measures a clinic's ability to function as a team, including cohesiveness, communication, and role and goal clarity.28–29 The questionnaire contains 31 items with 4-point ordinal ratings (Strongly Disagree to Strongly Agree) and demonstrates good reliability. The scores are summed and converted to a 0 to 100 score, with higher scores representing more team development components firmly in place.29 The CSQ is a validated assessment that measures attributes important for understanding practices in the process of change.12,17 It contains 38 questions with 5-point ordinal ratings (Strongly Disagree to Strongly Agree) across 5 domains: 1) adaptive reserve (ability to make and sustain change), 2) community knowledge, 3) health IT integration, 4) cultural sensitivity, and 5) patient safety culture. The CPAT is a validated tool used in health-care settings to measure the degree to which team members perceive that they collaborate effectively.30 It includes 56 items with a 7-point ordinal rating (Strongly Disagree to Strongly Agree) across 8 domains: 1) mission, meaningful purpose, goals; 2) general relationships; 3) team leadership; 4) general role responsibilities, autonomy; 5) communication and information exchange; 6) community linkages and coordination of care; 7) decision making and conflict management; and 8) patient involvement.

### Analysis

#### Qualitative

After the intervention, 2 qualitative research personnel reviewed the PF's field notes. Using the recorded text of the item, each action item was assigned to a general QI goal or HVE. Specifically, if the description matched the HVE explicitly, it was assigned the HVE designation; all others were designated as a general goal. From the field notes, the clinic staff member who selected the goal was noted, and action items were tracked month to month for completion. Coded action items were tabulated for each HVE or QI goal and month in which they occurred. These results were charted to understand each practice's progress toward achieving goals and to compare intervention and control practices.

#### Quantitative

Descriptive statistics were calculated on numbers of goals set and goals achieved, numbers of action steps set, HVE achievement, and on each individual clinician and staff survey scale. HVE achievement was calculated as number of 26 achieved at baseline, at each quarter, and end of study. We performed difference-in-difference analyses to compare pre- and post-trial goals, action plans, HVE achievement, and survey results between intervention and control clinics.

Multivariable models adjusted standard errors for clustering at the clinic level using the SAS Surveyreg procedure, which can handle complex survey sample designs, including designs with stratification, clustering, and unequal weighting. The procedure fits linear models for survey data and computes regression coefficients by generalized least-squares estimations and their variance-covariance matrix. Surveyreg computes the regression coefficient estimators by generalized least-squares estimation and computes variances of the regression parameters using Taylor series linearization. For each of the survey composite scores, we fit a probability model with the computed survey score as the unit of analysis. We assessed differences in change from pre- to post-trial between intervention and control clinics using the following regression model: survey outcome = clinic + intervention group + time + (intervention × time). A statistically significant interaction term (*P* < .05) indicates difference in difference between intervention and control clinics.

## Results

A total of 8 practices participated in the study, were randomized, and received 12 months of practice facilitation from May 2013 to April 2014. Table 2 displays the descriptive statistics by clinic in their paired randomization group. The clinics ranges in size from 7,257 to 14,119 empaneled patients, with between 4 and 13 provider full-text equivalents (FTE) per clinic. The clinics were predominantly fee-for-service, with <20% of payments from other forms, such as capitation or quality/care management fees. Insurance status of the patients was mixed, with Medicare ranging from 16% to 34%, Medicaid from 5% to 42%, and the rest commercially insured. For the patient panels, range of average ages was 39.7 to 52.5 years, with between 1.3% to 5.6% of patients above 85 years. Gender of patients in the clinics ranges from 56.8% to 64.4% female. Hierarchical Condition Category (HCC) scores in the clinic ranged from 0.87 to 1.09. To accommodate differences between clinics, a stratified matching approach was used for intervention assignment and for patient level analyses.

View this table:
[Table 2.](http://www.jabfm.org/content/30/5/572/T2)

Table 2. 
High Value Element (HVE) Descriptions and Clinic Selection of HVEs

### Goal Selection and Action Items

Goals were selected monthly. Examination of both general QI goals and HVE goals showed nearly an equal number set across both arms (7.3 goals for intervention and 7.5 for control), as shown in Table 3. Each practice in the intervention and control arms selected goals related to HVEs, and intervention practices selected a higher percentage of HVE goals (82.8%) than control (26.7%). Intervention practices worked on HVE goals for an average of 2.7 months and on other goals an average of 2.2 months. Control practices worked on HVE goals an average of 2.4 months and other QI goals an average of 2.1 months. Overall intervention practices selected a total of 101 HVE goals and met 68% while control practices selected 41 HVE goals and met 61%. Control practices set more individual action items (mean = 40) to achieve goals than intervention practices (mean = 28.75). Clinician implementation team participants within intervention practices led selection of 30% of action items compared with 20% in the control arm.

View this table:
[Table 3.](http://www.jabfm.org/content/30/5/572/T3)

Table 3. 
Practice Characteristics of Intervention and Control Clinics

### Achievement of HVE Goals

Table 4 shows selection and progress toward achieving 12 HVE topics. Intervention practices selected goals related to 9 HVE topics. None of the intervention practices selected goals related to after hours access, clinical information exchange or care coordination outreach. Although HVEs were not specifically introduced to control practices, they independently selected goals related to 7 HVEs: tracking third next available appointment, utilization follow-up, utilization prevention, performance data utilization, care coordination outreach, care plan utilization, and advance directive utilization.

View this table:
[Table 4.](http://www.jabfm.org/content/30/5/572/T4)

Table 4. 
Clinician and Staff Survey Results Before and After TOPMED Trial

Intervention practices showed a marked increase in achieving HVE elements at project end (Figure 1). At baseline, intervention practices had a cumulative score of 38 HVE elements achieved compared with control practices at 27 HVE elements. At quarter 4, intervention practices doubled to 76 HVE elements achieved (average 19 per practice, or 73% of maximum) while baseline practices increased to 44 HVE elements (11 per practice or 42% of maximum).

![Figure 1.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/30/5/572/F1.medium.gif)

[Figure 1.](http://www.jabfm.org/content/30/5/572/F1)

Figure 1. 
Summed HVE selection and achievement by arm.

### Survey Results

Clinician and staff survey results were mixed. Response rate for the TDM, measuring team function and cohesiveness, was 80.4% pretrial (N = 137) and 43.5% post-trial (N = 83). Scores increased for the intervention relative to the control clinics (+2.38/100 difference-in-difference), but this was not statistically significant (*P* = .312). CSQ response rates were 92.3% pretrial (N = 210) and 60.4% post-trial (N = 104). Intervention and control clinics performed similarly on 4 out of 5 domains, while control clinics demonstrated statistically significant (*P* = .030) improvement in *community knowledge*, compared with the intervention clinics (+.38/5.0 difference-in-difference). Response rates for the CPAT were 71.3% pretrial (N = 119) and 44.0% post-trial (N = 101). The intervention clinics performed significantly better relative to controls (*P* < .05) on 4 of the 8 domains, including *Mission, Meaningful Purpose, Goals* (+.61/7.0 difference-in-difference); *Team Leadership* (+.85/7.0 difference-in-difference); *General Role, Responsibility, Autonomy* (+.63/7.0 difference-in-difference); and *Communication and Information Exchange*, and *Coordination of Care* (+.58/7.0 difference-in-difference).

## Discussion

TOPMED compared the ability of targeted practice facilitation coupled with a change package to that of generalized QI in driving clinics toward more effective components of practice transformation. Practices in the intervention arm selected HVE goals consistently over the 12 months of the project, continued to progress across 9 of the 12 HVE topics, and sought to reach increasingly challenging clinical benchmark tiers within each HVE. From baseline to quarter 4, intervention practices doubled their achievement of measurable HVE elements from 38 to 76 (of 104). Control practices progressed from 27 to 44 over the same time period. Staff survey results on team function and ability were mixed and did not demonstrate a lot of change; however, the intervention arm performed better on 2 of 3 surveys.

HVE goals challenge practices to make changes that might reduce cost and utilization. It seemed likely that practices selecting HVEs would require more time to achieve the goals and would accomplish fewer goals overall because of their difficulty. Notably, intervention and control practices achieved nearly the same number of goals over the course of the project. Our review of action items showed clinician members of implementation teams were more involved in selecting improvement goals when chosen from the HVE menu set. Practices were able to achieve HVEs with fewer action items. This could be because the PF was able to efficiently guide implementation teams to focus on the most effective actions to undertake for HVEs, or it could reflect the complex or nonlinear nature of selected non-HVE goals. Field notes show the PF actively spread success ideas from 1 practice to another in HVE work. This combination of rapid and efficient goal progression, clinician involvement and achievement of increasingly challenging benchmarks supports the effectiveness of a high-value menu set in conjunction with targeted practice facilitation in reaching goals most likely to impact practice improvement.

As the field of practice facilitation grows, there is a need to identify and document those strategies that best enable practices to achieve and maintain transformation. To understand correlations between improvement activities and external factors like practice facilitation requires in-depth analysis. Other studies demonstrate the effectiveness of continuous QI compared with a reflective adaptive process18 and the use of practice-tailored facilitation. In TOPMED, we found that use of a targeted HVE change package across 12 noncondition-specific practice improvement elements enabled practices to set and achieve valued improvement goals. Coding goals and action items allowed the research team to measure progress across the diverse practice improvement elements that were selected, and may be useful for similar projects.

The study has several limitations. The pragmatic design of the trial makes it more difficult to determine trial effectiveness. Many of the practices were contemporaneously involved with other QI initiatives and programs. The small sample size and recruitment approach made it difficult to match practice pairs, and the intervention arm had already achieved more HVEs at baseline compared with control. The practices themselves also underwent changes that may have diverted them from TOPMED efforts, including transitions in ownership and changes to EHRs. The duration of the practice facilitation was limited to 12 months and studies of practice change show that transformation is uneven, unstable, and takes time.31 It is challenging to understand the impact financial incentives had on practice engagement. However, as the goals for QI changes are the outcome of interest, we were able to demonstrate whether these more challenging goals could be met. Finally, the post-trial survey had a low response rate from a different distribution of respondents, making it difficult to compare with the baseline surveys.

## Conclusion

As predicted, intervention clinics worked on and completed more HVEs than control clinics regardless of the difficult nature of HVEs. Targeted practice facilitation was effective in assisting practices to achieve HVEs, and intervention and control practices achieved the same number of goals. Intervention practices were more efficient in action item selection. Targeted facilitation improved care team function and collaboration. Overall, the TOPMED trial supports the effectiveness of combining a menu set with targeted practice facilitation in reaching HVE goals. Because HVEs have demonstrated capacity to improve the Triple Aim, working on them may improve patient care and reduce cost and utilization.

## Acknowledgments

The authors would like to extend special acknowledgment to Beth Sommers, Melinda Davis and Sonya Howk for their contributions to this study.

## Notes

*   This article was externally peer reviewed.

*   *Funding:* Gordon and Betty Moore Foundation Grant ID: GBMF2908; AHRQ R18 HS17832 (Enhancing Complex Care Coordination Information Systems).

*   *Conflict of interest:* none declared.

*   To see this article online, please go to: [http://jabfm.org/content/30/5/572.full](http://jabfm.org/content/30/5/572.full).

*   Received for publication February 3, 2017.
*   Revision received February 3, 2017.
*   Accepted for publication February 3, 2017.

## References

1.  1.Nutting PA, Crabtree BF, Miller WL, et al. Transforming physician practices to patient-centered medical homes: Lessons from the national demonstration project. Health Aff (Millwood) 2011;30:439–45.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjMwLzMvNDM5IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8zMC81LzU3Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

2.  2.McNellis RJ, Genevro JL, Meyers DS. Lessons learned from the study of primary care transformation. Ann Fam Med 2013;11(Suppl 1):S1–5.
    
    [FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTM6IjExL1N1cHBsXzEvUzEiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzMwLzUvNTcyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

3.  3.Geonnotti K, Taylor EF, Peikes D, et al. Engaging primary care practices in quality improvement: Strategies for practice facilitators (White Paper). Princeton, NJ: Mathematica Policy Research; 2015.
    
    

4.  4.Willard-Grace R, Hessler D, Rogers E, et al. Team structure and culture are associated with lower burnout in primary care. J Am Board Fam Med 2014;27:229–38.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjcvMi8yMjkiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzMwLzUvNTcyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

5.  5.Nagykaldi Z, Mold JW, Aspy CB. Practice facilitators: A review of the literature. Fam Med 2005;37(8):581.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=16145629&link_type=MED&atom=%2Fjabfp%2F30%2F5%2F572.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000231859300014&link_type=ISI) 

6.  6.Sugarman JR, Phillips KE, Wagner EH, et al. The safety net medical home initiative: Transforming care for vulnerable populations. Med Care 2014;52(11 Suppl 4):S1–10.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=25310631&link_type=MED&atom=%2Fjabfp%2F30%2F5%2F572.atom) 

7.  7.Dogherty EJ, Harrison MB, Graham ID. Facilitation as a role and process in achieving evidence-based practice in nursing: A focused review of concept and meaning. Worldviews Evid Based Nurs 2010;7:76–89.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=20180826&link_type=MED&atom=%2Fjabfp%2F30%2F5%2F572.atom) 

8.  8.Nagykaldi Z, Mold JW, Robinson A, et al. Practice facilitators and practice-based research networks. J Am Board Fam Med 2006;19:506–10.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMTkvNS81MDYiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzMwLzUvNTcyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

9.  9.Engels Y, van den Hombergh P, Mokkink H, et al. The effects of a team-based continuous quality improvement intervention on the management of primary care: A randomised controlled trial. Br J Gen Pract 2006;56:781–7.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMDoiNTYvNTMxLzc4MSI7czo0OiJhdG9tIjtzOjIwOiIvamFiZnAvMzAvNS81NzIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

10. 10.Nutting PA, Crabtree BF, Miller WL, et al. Journey to the patient-centered medical home: A qualitative analysis of the experiences of practices in the National Demonstration Project. Ann Fam Med 2010;8(Suppl 1):S45–S56.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTM6IjgvU3VwcGxfMS9TNDUiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzMwLzUvNTcyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

11. 11.Whitney B, Cranley L, Dearing JW, et al. Why (we think) facilitation works: Insights from organizational learning theory. Implement Sci 2015;10:141.
    
    

12. 12.Baskerville NB, Liddy C, Hogg W. Systematic review and meta-analysis of practice facilitation within primary care settings. Ann Fam Med 2012;10:63–74.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6NzoiMTAvMS82MyI7czo0OiJhdG9tIjtzOjIwOiIvamFiZnAvMzAvNS81NzIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

13. 13.Aspy CB, Enright M, Halstead L, et al. Improving mammography screening using best practices and practice enhancement assistants: An Oklahoma Physicians Resource/Research Network (OKPRN) study. J Am Board Fam Med 2008;21:326–33.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjEvNC8zMjYiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzMwLzUvNTcyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

14. 14.Mold JW, Fox C, Wisniewski A, et al. Implementing asthma guidelines using practice facilitation and local learning collaboratives: A randomized controlled trial. Ann Fam Med 2014;12:233–40.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTIvMy8yMzMiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzMwLzUvNTcyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

15. 15.Hogg W, Lemelin J, Graham I, et al. Improving prevention in primary care: Evaluating the effectiveness of outreach facilitation. Fam Pract 2008;25:40–8.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1093/fampra/cmm070&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=18209107&link_type=MED&atom=%2Fjabfp%2F30%2F5%2F572.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000254294000008&link_type=ISI) 

16. 16.Margolis PA, Lannon CM, Stuart JM, et al. Practice based education to improve delivery systems for prevention in primary care: Randomised trial. BMJ 2004;328(7436):388.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjgvNzQzNi8zODgiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzMwLzUvNTcyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

17. 17.Jaén CR, Ferrer RL, Miller WL, et al. Patient outcomes at 26 months in the patient-centered medical home National Demonstration Project. Ann Fam Med 2010;8(Suppl 1):S57–67.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTM6IjgvU3VwcGxfMS9TNTciO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzMwLzUvNTcyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

18. 18.Dickinson WP, Dickinson LM, Nutting PA, et al. Practice facilitation to improve diabetes care in primary care: A report from the EPIC randomized clinical trial. Ann Fam Med 2014;12:8–16.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6NjoiMTIvMS84IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8zMC81LzU3Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

19. 19.Meropol SB, Schiltz NK, Sattar A, et al. Practice-tailored facilitation to improve pediatric preventive care delivery: A randomized trial. Pediatrics 2014;133(6):e1664–75.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6MTE6IjEzMy82L2UxNjY0IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8zMC81LzU3Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

20. 20.Lessard S, Bareil C, Lalonde L, et al. External facilitators and interprofessional facilitation teams: A qualitative study of their roles in supporting practice change. Implement Sci 2016;11(1):1.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1186/s13012-016-0525-0&link_type=DOI) 

21. 21.Parchman ML, Noel PH, Culler SD, et al. A randomized trial of practice facilitation to improve the delivery of chronic illness care in primary care: Initial and sustained effects. Implement Sci 2013;8(1):93.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1186/1748-5908-8-93&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=23965255&link_type=MED&atom=%2Fjabfp%2F30%2F5%2F572.atom) 

22. 22.Nutting PA, Crabtree BF, Miller WL, et al. Journey to the patient-centered medical home: A qualitative analysis of the experiences of practices in the National Demonstration Project. Ann Fam Med 2010;8(Suppl 1):S45–56; S92.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTM6IjgvU3VwcGxfMS9TNDUiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzMwLzUvNTcyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

23. 23.Dorr DA, McConnell KJ, Williams MPJ, et al. Study protocol: Transforming outcomes for patients through medical home evaluation and redesign: A cluster randomized controlled trial to test high value elements for patient-centered medical homes versus quality improvement. Implement Sci 2015;10(1):13.
    
    

24. 24.Dorr DA, Anastas T, Ramsey K, et al. Effect of a pragmatic, cluster-randomized controlled trial on patient experience with care: The Transforming Outcomes for Patients Through Medical Home Evaluation and reDesign (TOPMED) Study. Med Care 2016;54(8):745–51.
    
    

25. 25.Fagnan LJ, Morris C, Shipman SA, et al. Characterizing a practice-based research network: Oregon Rural Practice-based Research Network (ORPRN) survey tools. J Am Board Fam Med 2007;20:204–19.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjAvMi8yMDQiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzMwLzUvNTcyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

26. 26.Dale JA, Behkami NA, Olsen GS, et al. A multi-perspective analysis of lessons learned from building an Integrated Care Coordination Information System (ICCIS). AMIA Annu Symp Proc 2012;2012:129–35.
    
    

27. 27.Oregon Health Authority. Patient Centered Primary Care Program - Home Page. Available from: [http://www.oregon.gov/oha/pcpch/Pages/index.aspx](http://www.oregon.gov/oha/pcpch/Pages/index.aspx). Accessed on April 25, 2016.
    
    

28. 28.Salem-Schatz S, Ordin D, Mittman B. Guide to the team development measure. 2010. Available from: [http://www.queri.research.va.gov/ciprs/projects/team\_development\_measure.pdf](http://www.queri.research.va.gov/ciprs/projects/team_development_measure.pdf).
    
    

29. 29.Stock R, Mahoney E, Carney PA. Measuring team development in clinical care settings. Fam Med 2013;45:691–700.
    
    

30. 30.Schroder C, Medves J, Paterson M, et al. Development and pilot testing of the collaborative practice assessment tool. J Interprof Care 2011;25:189–95.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.3109/13561820.2010.532620&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=21182434&link_type=MED&atom=%2Fjabfp%2F30%2F5%2F572.atom) 

31. 31.Donahue KE, Newton WP, Lefebvre A, et al. Natural history of practice transformation: Development and initial testing of an outcomes-based model. Ann Fam Med 2013;11:212–9.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTEvMy8yMTIiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzMwLzUvNTcyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==)