Abstract
Introduction: Complex behavioral interventions such as diabetes shared medical appointments (SMAs) should be tested in pragmatic trials. Partnerships between dissemination and implementation scientists and practice-based research networks can support adaptation and implementation to ensure such interventions fit the context. This article describes adaptations to and implementation of the Targeted Training in Illness Management (TTIM) intervention to fit the primary care diabetes context.
Methods: The Invested in Diabetes pragmatic trial engaged 22 practice-based research network practices to compare 2 models of diabetes SMAs, based on TTIM. We used surveys, interviews, and observation to assess practice contextual factors, such as practice size, location, payer mix, change and work culture, motivation to participate, and clinical and administrative capacity. The enhanced Replicating Effective Programs framework was used to guide adaptations to TTIM and implementation in participating practices.
Results: Practices varied in size and patient demographics. All practices had integrated behavioral health, but limited health educators or prescribing providers. Adaptations to SMA delivery accommodated the need for flexibility in personnel and reduced scheduling burden. Adaptations to TTIM content were designed to fit general primary care diabetes and Spanish-speaking patients.
Conclusion: Enhanced Replicating Effective Programs is a useful process framework for adaptation, implementation, and testing of diabetes SMAs in primary care. Adapting intervention content, delivery, and training to fit context can help ensure pragmatic trials have both internal and external validity. Attention to intervention fit to context can support continued practice engagement in research and sustainability of evidence-based interventions.
- Comparative Effectiveness Research
- Diabetes Mellitus
- Evidence-Based Medicine
- Implementation Science
- Mental Health
- Motivation
- Patient Care Team
- Practice-Based Research
- Primary Health Care
- Shared Medical Appointments
- Stakeholder Participation
- Surveys and Questionnaires
Introduction
Calls for conducting pragmatic research emphasize generating evidence that can be readily adopted and sustained in real-world care settings.1,2 In contrast to traditional randomized controlled trials, pragmatic trials are “undertaken in the ‘real world’ and with usual care and [are] intended to help support a decision on whether to deliver an intervention.”3 Pragmatic research methods have grown in use and acceptability.4⇓–6 Pragmatic trials should be designed to closely approximate existing resources, systems, and processes in diverse usual care settings and patient populations.3 Pragmatic trials for complex behavioral interventions in primary care are still emerging, and methods are needed to minimize practice burden while maintaining fidelity to research protocols.7,8
One challenge is ensuring intervention fit for the primary care context in terms of feasibility and sustainability (using existing staff, resources, and payment mechanisms), acceptability, and responsiveness to the needs of patients, practice members, and organizational leaders. Here, we describe our approach to ensuring intervention fit to context for the Invested in Diabetes pragmatic trial, including support for intervention adoption and implementation. Invested in Diabetes is testing the comparative effectiveness of 2 models of diabetes shared medical appointments (SMAs) in primary care.9 The study compares “patient-driven” to “standardized” diabetes SMAs. Both behavioral intervention models include diabetes self-management education and support (DSME/S) using the evidence-based Targeted Training in Illness Management (TTIM) curriculum.10⇓⇓–13 The setting is 2 practice-based research networks (PBRNs): the State Networks of Colorado Ambulatory Practices and Partners and the American Academy of Family Physicians National Research Network. PBRNs are an important resource for conducting pragmatic trials,14 connecting researchers with practices that are both open to innovation and research and tolerant of balancing fidelity needs with local customization.15,16
PBRNs provide infrastructure for identification of relevant settings, practice recruitment and engagement, and methods for randomization that account for practice context and variation.17
Heintzman et al14 noted that PBRNs benefit from using dissemination and implementation (D&I) frameworks and collaborating with D&I scientists and called for studies that highlight these mutually beneficial partnerships. D&I frameworks inform the processes by which interventions are adapted and implemented in the primary care context and subsequent reporting and evaluation.18 The enhanced Replicating Effective Programs (REP) framework supports intervention uptake through adaptation to local practice settings, while retaining sufficient fidelity to core elements. Enhanced REP consists of 5 phases: preconditions, preimplementation, implementation, maintenance and evolution, and facilitation.19⇓–21 Facilitation within enhanced REP is a collaborative process in which practice coaches and local stakeholders work together to implement and sustain new interventions by encouraging organizational buy-in and support.22 Here, we describe application of the enhanced REP framework for adapting diabetes SMAs to fit PBRN practice context in the Invested in Diabetes pragmatic trial. Specifically, we describe (1) use of PBRNs to recruit practices for pragmatic research, (2) use of the enhanced REP framework to adapt the TTIM intervention to the primary care context before implementation, and (3) use of practice facilitation and mixed methods to refine intervention adaptations and uptake.
Methods
Study Design
Invested in Diabetes is a cluster randomized pragmatic trial testing the comparative effectiveness of patient-driven versus standardized models of diabetes SMAs.9 We used quantitative and qualitative methods to assess practice contextual factors relevant to intervention adaptation and implementation. We used the enhanced REP framework to adapt interventions for fit within this context, while retaining core elements needed to address research questions. Core elements included (1) use of TTIM curriculum materials, (2) a closed group setting (2+ patients, ideally 8 to 10 per cohort), (3) a set number of sessions with equal duration (12 total hours of instruction), (4) a standardized condition (content delivered by a health educator with topics in a set order) versus the addition of a behavioral health provider (BHP) who delivers TTIM mental and behavioral health content (in 1 to 2+ sessions) and a peer mentor (in all sessions) with topic order selected by each group (patient-driven condition), and (5) patients meeting individually with prescribing providers for medical management.
Setting and Participants
Twenty-two practices were randomized to deliver patient-driven or standardized diabetes SMAs (11 per condition) using existing clinical staff and payment mechanisms. Eligible practices were required to have existing health education staff and BHPs, defined as licensed clinical social workers and/or licensed clinical psychologists who worked in primary care (note that we did not restrict participation to or formally assess level or type of integrated care model for this project; practices endorsed a range of colocated to fully integrated models). As part of usual processes of care, patients could be referred to BHPs in both study conditions for behavioral health concerns; for the patient-driven condition, BHPs were intentionally included in the delivery of the TTIM content, thus actively integrating the BHPs into diabetes education and self-management support. Covariate constrained randomization ensured balance in practice characteristics across conditions.17
Quantitative Measures and Data Collection
A practice representative (eg, practice manager) first completed a baseline practice characteristics survey, including 20 items assessing practice size and type, types and numbers of providers in the practice (including clinicians with prescribing privileges and those eligible to serve as health educators), patient demographics, payer mix, and prior experience with SMAs and quality improvement (QI) initiatives. This survey also included several subscales from the Patient-Centered Medical Home Practice Monitor, consisting of 33 items addressing QI processes, data capacity, team-based care, and population management subscales.23,24 All practice staff and clinicians were invited to complete the Practice Culture Assessment, a 22‐item survey of organizational change culture, work culture, and practice chaos.25
Qualitative Data Collection
Individual interviews were conducted with specific care team members from each practice, including 27 medical providers, 19 BHPs, 23 health educators, 27 SMA coordinators, and 14 practice managers. A semistructured interview guide focused on experience and understanding of SMAs, perceived importance of SMAs, sustainability factors, and expected patient response to the SMAs. A demographic form assessed interviewee practice role, background, and credentials.
Quantitative Analysis
Baseline practice characteristics and culture surveys were analyzed with descriptive statistics and compared across conditions. Categorical variables were analyzed via the Fisher's exact test due to the small sample size. Medians were compared via the Wilcoxon rank-sum test. Practice Culture Assessment scores were compared using linear regression with a fixed term for study arm and random facility effect to account for correlation among responses from the same practice.
Qualitative Analysis
Interview transcripts were analyzed using thematic analysis (data triangulation and a constant comparative technique). Using immersion crystallization with repeated phases of reading, reviewing, and refining themes and codes,26 a team of 3 qualitative researchers developed a master code list using a priori codes based on the underlying project theory and de novo codes developed through an iterative, collaborative process.27 The team then independently coded the same documents and compared them until a high degree of conceptual interrater reliability was established, at which point documents were independently coded by 1 team member. We used a matrix approach to summarize qualitative data across themes for each practice.28 Corroborating/legitimizing (ie, determining validity of qualitative themes) was conducted to ensure accuracy of themes and as an approach to member checking.26 This includes a process of reviewing interviews after analysis, reviewing the literature to confirm/disconfirm themes, and seeking input from the research team. The qualitative interview findings were intended to complement and triangulate with quantitative data and stakeholder discussions; insights were consistent across sources. We intentionally interviewed specific care team members from each practice to ensure representation of perspectives from each practice and, therefore, continued interviews even after thematic saturation was achieved.
Enhanced REP Implementation Process
Figure 1 shows the application of enhanced REP to the Invested in Diabetes study.
The enhanced Replicating Effective Programs framework for Invested in Diabetes Study planning and adaptations. Abbreviation: SMAs, shared medical appointments.
Preconditions Phase
During this phase, an evidence-based intervention for a targeted patient population that fits with local clinical settings is selected. Intervention materials are adapted based on local stakeholder input. Intervention training, implementation, and outcome measures are refined and “packaged” in a user-friendly way.19,22,29 Before this study, we engaged patients with diabetes, family members, and health care providers in the Boot Camp Translation (BCT) community engagement process.30,31 The BCT objective was to identify preferences about DSME/S in primary care.32 Based on BCT, the study team (including patient stakeholders) identified TTIM as an evidenced-based DSME/S curriculum that fit stakeholder preferences for addressing concurrent medical and mental health needs. The TTIM version selected for Invested in Diabetes was developed for diabetes and co-occurring serious mental illness.10⇓⇓–13 Diabetes distress was selected as the primary patient-centered outcome and glycohemoglobin (HbA1c) as the primary clinical outcome. Cost and sustainability were prioritized as practice-level outcomes.
Preimplementation Phase
The next step is to identify and engage practices in preparing for implementation. After orientation to core elements of the intervention, participating practices identify staff to deliver the intervention. Communication protocols are established between practices and researchers to create dialog regarding further adaptations needed to fit local context. In this study, PBRNs recruited practices with interest and capacity to implement diabetes SMAs. We elicited local stakeholder input to refine the protocol and clarify potential payment mechanisms for diabetes SMAs. We assessed practice context and resources and negotiated adaptations to intervention materials and delivery protocols for fit while retaining fidelity to core elements. Practice staff roles were also determined for intervention delivery as well as operational support.
Implementation Phase
The next phase involves dissemination of the packaged intervention, training in intervention delivery, ongoing technical assistance, resolving implementation barriers, and evaluation and fidelity monitoring. For this study, the research team disseminated the packaged intervention to practice members, including training and recruitment materials (with additional guidance for recruiting peer mentors in the patient-driven condition) and electronic health record templates for documentation. Practice coaches met with practice staff to facilitate implementation tailoring. Practice facilitators assisted in process mapping and identifying potential implementation barriers and adaptations. Communication strategies such as quarterly practice stakeholder calls and scheduled practice coach check-ins were established. Practice coaches scheduled at least 4 facilitation sessions per practice, plus more outreach as needed.
Iterative Process of Adaptations
Adaptations are deliberate modifications made to intervention design or delivery to fit the context or address implementation barriers.33 Informed by preimplementation data and insights from practice coaching sessions, and in collaboration with stakeholder groups, the study team discussed and approved adaptations needed to fit practice capacity, culture, workflows, resources, and systems, without compromising fidelity. We used the Framework for Reporting Adaptations and Modifications-Expanded (FRAME) by Stirman et al.33,34 to codify the type of intervention adaptation, organized by intervention content (curriculum and materials), context (format, setting, population, and personnel), training, and evaluation.
Results
Practice Contextual Characteristics
Practice contextual characteristics are shown in Table 1 (practice-level measures) and Table 2 (provider and staff-level measures). Practices varied along several dimensions related to practice type, size, location, culture, staffing, health information systems, patient demographics and language, and payer mix. Twelve federally qualified health centers and 10 non-federally qualified health centers enrolled. Most practices self-identified as urban/suburban and 3 were rural. Most practices had experience with SMAs (77%), whereas half had experience with diabetes-specific SMAs (50%). All practices had an existing QI team and used an electronic health record. Half of the practices served a Latino population and 1 served a Native American population. There were no differences between study conditions.
Baseline Practice Contextual Characteristics
Baseline Practice Culture Assessment Scores among All Clinicians and Staff in Participating Practices
Baseline Practice Interviews
Thematic analysis revealed perceived opportunities and barriers to address in preparing for SMA implementation. Practices saw value in opportunities to improve diabetes care using SMAs, as most interviewees reported QI efforts had not adequately improved diabetes metrics. Practices recognized that a team-based approach including behavioral health and other practice resources was especially important:
“I think somehow figuring out how to intertwine behavioral health more in with diabetics because…it can be a very stressful condition for people. As I mentioned earlier, stress contributing to higher cortisol levels, higher blood sugars, generally being unwell” [physician]
“…improve diabetes management and health literacy of patients, empowerment of patients and yeah, the ability for patients to feel able to do these things, like confidence and compassion.” [BHP]
Practices with previous experience delivering SMAs described several past challenges but also interest in trying again, with Invested in Diabetes project implementation support. Patient recruitment and retention was one challenge to address: “The variable attendance, trying to get that buy-in [from patients] and that follow-through…people would be really motivated at first, and then quickly taper off after [initial sessions].” [prescribing provider]
Involving prescribing providers in SMAs—needed to ensure reimbursement but also logistically difficult—was also an anticipated challenge. “We've learned a lot about how to make the SMA structure easier on providers, which I think is challenging, especially when you are dealing with issues of productivity, which is always important, and then, of course, sustainability. Our program—we've pushed the limits of productivity, and we're still not yet sustainable.” [SMA coordinator].
Adaptations to Fit Context
Table 3 shows key differences in personnel, setting, population, and format between the original versus adapted delivery of TTIM to fit context while retaining core elements. TTIM was not originally designed for delivery by a multidisciplinary care team, including a prescribing provider, or for patients to select topic order. Training, facilitation, and intervention materials provided guidance for these features. Table 4 summarizes adaptations made during enhanced REP phases to intervention content, delivery (personnel, format, and setting/population), and trainings. One major adaptation was condensing TTIM from 12 1-hour modules to 6 2-hour modules, given challenges of coordinating visits and patient burden. As many practices prioritized education on stress management (vs serious mental illness), we added an alternative general stress and coping module based on a related intervention.13 Table 5 shows how practice contextual characteristics informed adaptation decision making. The study team also at times negotiated with individual practices based on their specific needs and circumstances to ensure the project's success. Figure 2 provides case vignettes describing how the intervention was adapted in consideration of diverse practice contexts.
Case vignettes of adaptations to fit practice characteristics. Abbreviations: SMA, shared medical appointments; FQHCs, federally qualified health centers.
High-Level Intervention Adaptations to Targeted Training in Illness Management Intervention Content and Context for Delivery in Primary Care
Intervention Content and Delivery Adaptations across the Replicating Effective Programs Phases
Intervention Content, Delivery, and Training: Adaptations Fit to Context
Significant adaptations involved personnel delivering SMAs. Several practices had concerns about involving prescribing providers at every session given scheduling challenges and patient copay burden. Not all practices determined they needed reimbursement from prescribing provider encounters. The resulting adaptation was to support practices in determining how many and in which sessions to include prescribing providers, while recommending their involvement at minimum during the first and last sessions. Similarly, the health educator role was expanded based on staffing needs to include nutritionists, medical assistants, and community health workers (with adequate supervision), whereas our initial protocol called for a registered nurse, health educator, or certified diabetes educator. BHPs were ineligible for the health educator role, despite some requests for this adaptation, as BHP involvement is a core distinguishing element of the patient-driven condition. BHP eligibility for the patient-driven condition also expanded. Although people filling the BHP role were predominantly masters-level licensed clinical social workers and doctoral-level licensed clinical psychologists, some had a Bachelor's degree in social work and psychology, were licensed family therapists, or were going through training to be a BHP, either through education or on-the-job training. As the patient-driven condition also includes a peer mentor, the study's patient stakeholders developed peer mentor selection criteria and recruitment materials for use by practices.
We adapted practice training for diabetes SMAs. For health educators, training first includes a comprehensive review of the TTIM instructor's manual, as with prior TTIM training, but was modified to further cover group facilitation skills, how to involve the peer mentor and BHP (for the patient-driven condition), and role playing sessions. To reduce time and travel burden based on feedback from initial training sessions, we shortened training from 8 to 6 hours and offered choices of in-person locations or video conferencing and the option to complete training on a single day or split over 2 days. For prescribing providers, a 1-hour training provides a brief review of the TTIM curriculum and guidance on documentation to support reimbursement. Peer mentors receive a 5-hour training that focuses on the peer support role, with materials adapted from Peers for Progress35 and TTIM skill-building exercises for goal setting, action planning, glucose monitoring, problem solving, and communication skills to prepare peer mentors for demonstrations to SMA patients.
Discussion
Prior research has often overlooked how to best adapt interventions for real-world health care settings, resulting in a limited uptake of evidence-based practices.36 Furthermore, external validity limitations have often resulted in evidence-based interventions lacking alignment with practice needs and priorities, such as yielding a return on investment and minimizing burden.36 Pragmatic trials can address these limitations, balancing equally important external and internal validity concerns.37 In turn, pragmatic trials should report contextual factors that may influence replication and relevance of research findings36—for which D&I frameworks may be useful toward “designing for dissemination and sustainability.” In a pragmatic trial among PBRN-affiliated primary care practices, we used the enhanced REP framework22 to guide the process of engaging practices in identifying and negotiating a variety of adaptations needed to fit practice contextual factors expected to influence adoption, implementation, and sustainability of diabetes SMAs.
Our findings showed considerable practice context variability across several dimensions, and corresponding adaptations were made to intervention content (eg, general stress and coping), delivery (eg, fewer sessions), training (eg, group facilitation techniques), and facilitation (eg, tailored technical assistance depending on prior SMA experience). Similar adaptations to reduce patient barriers and encourage practice-level buy in were reported by Kowalski et al.8 Adaptations were made through an iterative process in which patient and practice stakeholders informed intervention content, format, and delivery before and during initial implementation. Practice coaches also identified opportunities for adaptations at individual practices, jointly maintaining fidelity to protocols and supporting implementation and sustainability for each practice—an important function of practice facilitation.38,39 Concurrently, planned adaptations and explicit discussions and decisions relative to core elements of the study protocol helped to ensure fidelity and rigor (ie, adaptations were intended to be “fidelity consistent”).40,41 FRAME was a useful framework for codifying and categorizing the type of adaptations.33,34 Practices are likely to make additional unplanned, fidelity-inconsistent adaptations and will be evaluated separately.33,42
Limitations and Conclusions
The specific adaptations made to SMAs for this project may not generalize to other contexts, as the practices that participated in this project are PBRN members with existing behavioral health and health education resources. However, the primary contribution of this article concerns the process for adaptation to context—whatever that context may be. In summary, this project illustrates the use of mixed methods and D&I frameworks, namely, the REP framework to guide the process of adaptation of an intervention to fit context and FRAME to describe and categorize adaptations—expected to simultaneously advance science and support potential for future uptake—as is the goal of pragmatic trials.43
Acknowledgments
We acknowledge the important effort and commitment of the patient, practice, and community stakeholders who participated in the engagement work that led to the conceptualization of this study and those who continue to participate as patient representatives for the Invested in Diabetes study. Study team members who also contributed to the work described in this manuscript include Dennis Gurfinkel, Robyn Wearner, Miriam Dickinson, Jodi Summers Holtrop, and Patrick Hosokawa. We also extend our gratitude to our amazing patient stakeholders, Ramona Koren, Tom Carrigan, Barbara Clay, Jim and Jo Smith, David Downey, and Sharon Trujillo. Your contributions to the success of this project are invaluable. Thank you to the participating practices for investing in diabetes care and research; your contributions to society are substantial.
Notes
This article was externally peer reviewed.
Conflict of Interest: Dr. Sajatovic has had research grant funding from Nuromate, Otsuka, Alkermes, Janssen, International Society for Bipolar Disorders, Reuter Foundation, Woodruff Foundation, and Reinberger Foundation. She has been a consultant to Bracket, Otsuka, Janssen, Neurocrine, Health Analytics, Frontline Medical Communications and has received book/publication royalties from Springer Press, Johns Hopkins University Press, Oxford Press and UpToDate. The other authors have no conflicts of interest to declare.
Funding: Research reported in this manuscript was funded through a Patient-Centered Outcomes Research Institute (PCORI) award (IHS-1609-36322). The views, statements, and opinions presented in this work are solely the responsibility of the author(s) and do not necessarily represent the views of the PCORI, its Board of Governors or Methodology Committee.
To see this article online, please go to: http://jabfm.org/content/33/5/716.full.
- Received for publication January 30, 2020.
- Revision received June 12, 2020.
- Accepted for publication June 17, 2020.
