Abstract
Background: Four practice-based research networks (PBRNs) participated in a project to increase the diffusion of evidence-based treatment guidelines for chronic kidney disease (CKD). A multicomponent organizational intervention engaged regionally proximal primary care practices in a series of facilitated meetings, referred to as local learning collaboratives (LLCs).
Methods: The 2-wave strategy began with 8 practices in each PBRN receiving practice facilitation and subsequently joining an LLC. A sequential mixed-methods design addressed the conduct, content, and fidelity of the intervention; clinicians in 2 PBRNs participated in interviews, and PBRN coordinators reflected on implementation challenges.
Results: LLCs were formed in 3 PBRNs, with 121 monthly meetings held across 20 LLCs. Slightly more than half of the participants were clinicians. Qualitative data suggest that clinicians increased the priority for CKD care, improved knowledge and skills, were satisfied with the project, and attempted to improve care. Implementation challenges were encountered and concerns about sustainability expressed.
Conclusion: While PBRNs can successfully leverage resources to diffuse treatment guidelines, and LLCs are well-accepted by clinical staff, the formation of LLCs was not feasible for 1 PBRN, and others struggled to meet regularly and have performance data available despite logistic support.
Between 1988 to 1994 and the 2003 to 2006 National Health and Nutrition Examination Surveys, the prevalence of chronic kidney disease (CKD) among individuals ≥60 years old increased from 18.8% to 24.5%.1 CKD is associated with a 5-year all-cause mortality rate of 24% and a 20% 5-year requirement for transplant or dialysis.2 The burden of CKD and accompanying morbidity and mortality is especially heavy for the poor, minorities, those with poor literacy, and safety-net patients.3
Guidelines have been available since 2002,4 and there is evidence that guideline-based care can delay CKD progression and reduce mortality.5⇓⇓–8 The implementation of recommendations most applicable to primary care4 involves several care processes,9,10 including documenting diagnosis, ordering appropriate tests, discontinuing inappropriate medications, prescribing new medications, managing diabetes and cardiovascular disease risk factors, educating patients, administering appropriate immunizations, and referring patients with advanced disease. While there is some controversy regarding possible overdiagnoisis of CKD using estimated glomerular filtration rate (eGFR) as diagnostic criteria for stage 3A CKD without proteinuria testing (especially in the elderly, because of the age-related decline in eGFR),11,12 this target behavior seemed appropriate as the project was designed to improve implementation of current CKD guidelines, with which almost 50% of primary care physicians report being unfamiliar.
Dissemination, implementation, and diffusion of guideline recommendations remain suboptimal,13⇓–15 leading to an increase in studies of approaches to improve guideline uptake in primary care.16 These implementation strategies are “methods or techniques used to enhance the adoption, implementation, and sustainability of a clinical program or practice.”17 For example, although vast sums are spent on continuing medical education,18 evidence of effectiveness in translating research into practice has yet to emerge.19 Peer-to-peer learning has been found to motivate practice change, with formal learning collaboratives designed to create competition and urgency.20,21 Learning collaboratives typically involve large numbers of practices that receive education/information, create quality improvement (QI) teams, perform periodic medical record reviews, develop registries, and strive to implement evidence-based strategies over time.22
This article describes a multicomponent organizational intervention that engaged small groups of regionally proximal practices in facilitated meetings focused on care of patients with or at risk of CKD. These “local” learning collaboratives (LLCs), defined as a “data-driven group/network of change teams from organizations with commonality of care that work collectively to enhance performance through process improvement,”23(p391) are a scaled down version of more common breakthrough learning collaboratives championed by the Institute for Healthcare Improvement and others. Our experience suggests the LLCs are well-accepted by clinicians and staff,24 and can achieve the same motivation, competition, and collaborative learning generated in larger collaboratives through more frequent, briefer meetings held in more convenient locations. Optimally, LLCs meet monthly (in person), with fewer than 10 individuals, and have the capacity to generate practice performance data for feedback and comparison, sufficient meeting support, and an agenda with clear objectives. Participants are expected to improve guideline implementation, share positive anecdotes and lessons learned, and create practical solutions for common implementation problems. Forming small groups of clinicians interested in practice improvement and sharing best practices can affect factors driving intention to change (ie, beliefs about capabilities and consequences) as well as improve knowledge and skills pertaining to the care of patients with CKD.
In 2010 a consortium of practice-based research networks (PBRNs) was funded by the Agency for Healthcare Research and Quality to improve implementation in primary care of the National Kidney Foundation Chronic Kidney Disease guidelines. The project was designed to evaluate how PBRN resources can be leveraged to demonstrate the effectiveness of a strategy to disseminate and diffuse guidelines of importance in primary care. We aimed to improve the implementation of key guidelines as evidence of the effectiveness of our strategy, which involved practice facilitation (wave I) followed by the formation of LLCs (wave II). Given evidence of effectiveness of this 2-wave strategy,25 a secondary study aim was to conduct a mixed-methods process evaluation to examine lessons learned, including (1) whether the LLC approach was implemented as planned, (2) the CKD-related QI goals engaged in, and (3) the experiences of participating clinicians. Qualitative data explored the hypothesis that certain factors are critical to LLC implementation, feasibility, and satisfaction—namely, practice similarity (eg, same practice group or electronic health record [EHR]), geographic proximity, and existing relationship between clinicians. This article reports results for this secondary aim.
Methods
QI Strategy
The strategy used a combination of practice facilitation and LLCs. In the implementation phase (wave I), practices received assistance from an external facilitator, followed by a diffusion phase (wave II) in which members of wave I practices met with 2 other practices to focus on CKD guideline implementation. This approach involved translation and diffusion of strategies learned by early adopters (wave I clinician champions) to a second group of clinicians (wave II) during 6 monthly, 1-hour LLC meetings. Wave I clinicians were expected to be actively involved and to build on lessons learned in wave I. The LLCs received support from the practice facilitator (PF), who coordinated meetings and helped practices provide monthly performance data.
Population and Setting
Four regional PBRNs in Oklahoma (OKPRN), California (LANet), Wisconsin (WREN), and Minnesota (MAFPRN), and a research corporation (Westat), collaborated on the project; each PBRN had experience leading QI projects with PFs. Our goal was to support the formation of 8 LLCs within each PBRN, each with 3 practices (1 from wave I, 2 from wave II), with the potential reach of 96 practices.
Process and Outcome Measurement
The mixed-methods design structure was QUAN → QUAL, with sequential collection and analysis of quantitative and qualitative data. The function of the analysis was primarily expansion, whereby qualitative data addressed questions regarding project implementation raised by the quantitative data.26 LLC meeting and practice summaries were collected from the PFs in each PBRN, and data on guidelines worked on and fidelity were abstracted, including the number of meetings with (1) a representative from each practice, (2) a wave I clinician present, and (3) discussion of performance data. Because implementation was a function of the resources and constraints of each PBRN,27 coordinators were asked to write short descriptions of the feasibility of their implementation strategies, considering the context of the PBRN.
The quantitative fidelity data were used to identify 2 high- and 2 low-performing LLCs to collect data through follow-up interviews with participating clinicians in these practices. The interview guide addressed reactions to the LLCs and perceptions of changes in skills, knowledge, and clinical care. Participants were informed that responses would be kept private and reported in the aggregate, and consent to participate and audio-record the discussion was obtained in advance. Cleaned, formatted interview notes were checked against recordings for accuracy and imported into NVivo 9 (QSR International, Burlington, MA). Two coders identified potential codes of interest, based on the content areas and literature identifying drivers for successful LLCs,28,29 and developed a codebook with primary and secondary code names, definitions, and examples. The analysis process was initiated through regular analysis team meetings; disagreements in classification were discussed and resolved through consensus. The NVivo Matrix Coding query function was used to identify patterns and compare responses by PBRN and high/low fidelity score. Queries also assessed whether themes were expressed across respondents to ensure that topics representative of multiple respondents' experiences were identified. Throughout the analysis process led by the first author (PDL), emergent themes were shared with the PBRN coordinators for feedback.
Results
With the assistance of the PBRNs, each wave I practice was grouped with 2 additional practices. Wave I practices were PBRN members interested in improving CKD care and willing to assist 2 other practices during the subsequent LLC intervention. Wave II practices were less likely to be a PBRN member and to have participated in a QI project, but they were otherwise similar to wave I practices and included private practices, hospital outpatient clinics, academic practices, and federally designated health centers or rural clinics. A total of 89 practices participated across both waves of the project (93% of the target goal of 96 practices).
LLCs were implemented in 3 of the 4 PBRNs. Two collected complete meeting minutes, from which fidelity outcomes were extracted. Because the qualitative study was designed to stratify based on the assessment of high versus low fidelity, a subset of clinicians from these 2 PBRNs participated in the interviews. Table 1 summarizes LLC formation and the availability of data.
Local Learning Collaborative (LLC) Formation and Availability of Data by Practice-based Research Network (PBRN)
Implementation Conduct, Fidelity, and Feasibility
LLC Conduct and Fidelity
Twenty LLCs were formed and met at least once. Although 8 LLCs were initially formed in LANet, none was able to meet as designed. OKPRN formed 8, WREN 7, and MAFPRN 5 LLCs. There were 121 monthly meetings held across 20 LLCs, with most holding the 6 required meetings (Table 2). A total of 210 unique primary care staff participated in at least 1 LLC meeting; WREN had the highest number of unique attendees. Slightly more than half (54%) were clinicians, with the largest proportion of clinician participants (68%) in OKPRN.
Implementation Outcomes Representing Intervention Conduct and Fidelity for Three LLCs. (N = 20 LLCs, 3 PBRNs)
Feasibility of the Implementation Strategy within Each PBRN
Each PBRN coordinator provided a summary of the feasibility issues related to the implementation strategy for each PBRN.27 Statements are presented in their entirety (Table 3).
Practice-based Research Network (PBRN) Coordinator Perspectives on Local Learning Collaborative (LLC) Feasibility
LLC Fidelity Outcomes
Data on LLC fidelity were extracted from minutes of 90 meetings held across 15 LLCs in 2 PBRNs (Table 4). At the majority of meetings, all practices were in attendance (79%), the wave I clinician was present (82%), and performance data were discussed (68%).
Fidelity Outcomes by Practice-based Research Network (PBRN) (N = 15 LLCs, 2 PBRNs)
CKD Guidelines Targeted for Change
During the intervention, some practices worked on as few as 3 guidelines, whereas others worked on as many as 6 (Table 5). Patient education (smoking cessation, counseling) was the guideline most frequently addressed, followed by adding CKD diagnosis to the problem list. Least frequently addressed was eGFR in patient charts. More OKPRN practices worked on adding appropriate medications and removing inappropriate medications (20 practices each), and fewer worked on eGFR in patient charts (8 practices). WREN practices tended to work on patient education most frequently (21 practices) and ordering of appropriate laboratory tests least frequently (7 practices).
CKD Guidelines Addressed by Practices in Two LLCs
Clinician Experience (Qualitative Interview Data)
Interviews were conducted with 21 of 24 clinicians in 8 LLCs (4 each in OKPRN and WREN). Using a modified grounded theory approach, the analysis was based on the first 2 levels of the 4-level evaluation framework developed by Kirkpatrick30: (1) participants' reactions and (2) perceived change in knowledge, skills, and attitudes (Table 6). There were no findings related to LLC fidelity (comparing clinicians in LLCs with high- vs low-fidelity outcomes) or PBRN.
Themes and Examples from Clinician Interviews
Level 1: Participants' Reactions
Three prominent themes include perceived impact of practice similarities and prior relationships, perceptions of the feasibility of LLCs, and overall project satisfaction. Most observations reflected positive aspects of shared practice characteristics, such as the same EHR or practice group, as well as the impact of a prior relationship among clinicians. Practice dissimilarity was a barrier to LLC effectiveness: as 1 clinician explained, “in meetings we'd talk about what they could do, but the clinic settings are different.” Another participant commented that differences in patient volume may explain reluctance to implement suggestions for improvements in ordering lab work. One interviewee believed that implementing new strategies and altering processes was easier in smaller offices compared with larger ones, whereas others commented that larger practices had the advantage because of more resources and staff.
Feasibility of LLCs
The majority of comments referred to a lack of protected time to devote to the project, including meeting preparation. One clinician stated, “I was happy to make the commitment. It was difficult in practice to pull it off.” In general, clinicians agreed that LLCs can be productive if groups are small, the topic is highly relevant, and the time commitment reasonable. Despite competing demands, participants were interested in attending, especially when meetings were held at their own sites at lunch time. Meeting frequency was considered adequate for both setting goals and assessing changes in performance data. The role played by the PFs in logistics and agenda-setting (and providing assistance between meetings, if needed), relevance of the CKD topic, a comfortable setting, and support from the entire practice team were identified as critical elements. Setting goals and deadlines at the first meeting was also considered helpful in achieving results.
The impact of sharing feedback data on motivation for improvement was a key finding. Even those less experienced with this strategy found the data helpful for comparing practices and monitoring their progress. Having clinical data available in advance was advised so clinicians had time to review the information and come prepared with questions. As 1 clinician stated, “The meetings were helpful to answer questions that did come up. In addition [they] conveyed level of accountability that helps keep you motivated.”
Satisfaction with the Project
Most participants were highly satisfied, though typical demands of primary care practice had an impact: “… I would have gotten more out of it if I did not have so many other stressors.” Another participant stated that reimbursement for time would also contribute to LLC success: “Anytime I am not seeing patients and am doing paperwork, [I am] going in the hole. If I have to do that and was not reimbursed, it is not practical.” Two clinicians noted that an incentive, such as lunch at the time of meeting, “just brings members right on in” and would “improve the morale.”
Level 2: Changes in Knowledge, Skills, and Attitudes
Increasing knowledge of CKD was cited frequently as the most significant outcome, particularly CKD screening and the effect of nonsteroidal anti-inflammatory drugs. Meetings had a positive impact on perceived ability to care for patients with CKD. Several clinicians reported improvements in patient education skills and protocols to identify and contact at-risk patients. For those with protocols or systems already in place, participating in meetings provided additional reinforcement and feedback. The LLC meetings were found to be less effective for clinicians who were already knowledgeable about CKD or perceived that they were already meeting expectations for effective CKD care.
Priority for Improving CKD Care
Participation also increased awareness of CKD care by providing a better understanding of the condition and increased confidence to identify and treat it. Some interviewees appreciated reinforcement on the importance of CKD, and others stated that this new information supplemented medical school training.
Discussion
This mixed-methods study provides different perspectives on a peer-to-peer intervention approach, facilitated by PBRNs, intended to affect clinician performance. Quantitative results show that the LLCs were not implemented as planned in every PBRN and that practice participation was challenging. Qualitative data suggest that clinicians increased priority for CKD care, improved knowledge and skills, were satisfied with the project, and attempted to improve care. Implementation challenges were encountered and concerns about sustainability expressed through a variety of sources, including LLC meeting summaries (from the PF perspective), coordinator descriptions, and discussions with participating clinicians. The wealth of data generated through this multi-PBRN project provides concrete reflections on the challenges associated with achieving CKD-related QI goals, as well as practical advice for the design and implementation of similar collaborative QI interventions. In particular, clinicians confirmed that sharing performance data, a key element of the LLC design, was critical to the effectiveness of the intervention and confirmed the importance of social relations and trust in establishing a productive learning environment.31,32 Not surprisingly, travel was a barrier to LLC attendance, even when meetings were scheduled at lunch time, and meetings held either by video or telephone were perceived as less effective unless groups were known to each other.
QI goals targeted by practices were often PBRN-specific, given regional differences in practice oversight and particular practice standards and influences, as described from the perspective of the PBRN coordinators. The data show that patient education was most frequently addressed, perhaps because clinicians in these particular PBRNs (OKPRN and WREN) did not have educational materials and appreciated access to these resources, and that the goal was relatively easy to achieve. Having a clear diagnosis in the problem list also emerged as a frequent goal, as specific care protocols were triggered by a CKD diagnosis and associated with having an eGFR value in the chart. For some, this was seamless when lab values were posted in the EHR by the cooperating laboratory; others found the required levels of bureaucracy made this goal unachievable under existing constraints. Based on our prior experience, achievements in the targeted QI goals were ones that were likely to be sustained.
Conclusions
PBRNs are unique environments in which to foster opportunities to improve the quality of primary care and to facilitate the sharing of evidence-based and practical solutions to common problems. While PBRNs can successfully leverage resources to disseminate treatment guidelines, and LLCs are well accepted by clinical staff, the formation of LLCs may not be feasible or effective in all contexts, even with logistic support. Data collected in this multi-PBRN project indicate that sharing performance data created motivation to improve patient care, attendees learned valuable lessons from each other, and clinicians were actively involved in attempts to improve the implementation of CKD guidelines.
In this article we have outlined a mixed-methods approach to assessing clearly defined intervention fidelity criteria. A key finding is confirmation from the perspectives of coordinators and clinicians that designing LLCs with certain factors in mind motivates a learning collaborative and contributes to feasibility and participant satisfaction. LLCs were perceived as more successful when there were prior relationships and when there was either more geographic proximity between practices in a single LLC and/or a health system infrastructure common to all. The coordinators and PFs observed that the likelihood of disseminating a best practice from 1 setting to another was enhanced under these conditions.
Acknowledgments
The authors appreciate the contributions of PBRN project coordinators Melody Bockenfeld (WREN), Carol Lange (MAFPRN), and Lyndee Knox (LANet), who provided their perspective on LLC implementation. The authors also greatly appreciate the practice facilitators, who provided the meeting minutes on which these analyses are based, and the practice staff who participated in the interviews. At Westat, Dr. Jennifer Huang led the qualitative task and conducted interviews with Vibha Vij, and Jeffrey Elmer served as coordinator.
Notes
This article was externally peer reviewed.
Funding: This work was supported by a grant from the Agency for Healthcare Research and Quality (1R18HS019945).
Conflict of interest: none declared.
- Received for publication January 29, 2016.
- Revision received April 12, 2016.
- Accepted for publication April 19, 2016.
