Major Themes and Lessons Learned, Organized by Project Phase
| Project Phase/Theme | Definition | Supporting Quotes |
|---|---|---|
| Project development | ||
| Compressed timeline | A limited timeline for project development and practice recruitment. | “The negative is I don't think it was enough time.” |
| “Boy, I wish that had been longer. I would have liked it to be longer.” | ||
| Recruitment challenges | Difficulties with practice recruitment included a lack of time, lack of willing practices, and low turnout in recruitment events. | “The first [challenge] being that we absolutely needed more time to recruit. We had to go back to AHRQ and say…' We cannot make this happen.” |
| Lessons learned | Adequate time for project development and recruitment of practices could improve success of the overall project. Project development for large-scale quality improvement initiatives could take 9 to 12 months. | |
| Collaborative research team | Project team worked well together, held regular team meetings, exhibited a diversity of skills, and had mutual respect of other team members' abilities. | “One of the reasons it was a success was the collaborative effort and the partnership and the team assembled.” |
| “We had many, many meetings leading up to the kick-off.” | ||
| Lessons learned | A strong research team representing multiple organizations and areas of expertise is critical for developing resources and accessing essential knowledge. Regular meetings help to enable teamwork, alignment of goals, methods, and problem solving. | |
| Intervention flexibility | A flexible research design was necessary for practice implementation of the intervention; however, there are strengths and weaknesses of this approach. | “We did keep a focus on the key HVH targets but adjusted the project to meet the participants where they were, not placing a single intervention on all of them regardless of their culture or preparation.” |
| “It's hard research-wise because the menu they offered was huge…one practice might have gotten X and the other one might have gotten B, right? So, they're totally, totally different.” | ||
| Lessons learned | Flexible design allows for intervention strategies to be customized to accommodate local context and meet the needs of practices. | |
| Well-regarded, but underused practice improvement toolkit | The tools and resources designed to support practices were highly regarded but were also thought to be underused. | “What was positive is that we were able to write a ton of educational material to practices such as smoking cessation.” |
| “Well, every week we're working so hard to post some new content and really try to drive engagement and people just are not logging in and using it.” | ||
| Lessons learned | A widespread collection of easily accessible educational materials and pragmatic tools is critical for practice transformation and evidence dissemination. A preintervention survey of participating practices on communication preferences could improve usefulness of tools. | |
| Intervention | ||
| Successful kickoff event | The kickoff event was a highly successful and collective launch of the initiative. | “The kickoff was powerful enough and really cast the vision.” |
| “The ones that didn't come, it was almost like…we had to kind of reeducate them on the kickoff.” | ||
| Lessons learned | A face-to-face session with presentations and active learning techniques for participating practices can be beneficial for buy-in and overall engagement with the initiative. | |
| Complicated coaching position | Coaches were extremely valuable to the initiative and were well regarded for their skills and expertise. | “The coaching was the most important thing we did.” |
| The coach's process of problem identification and resolution at the practice level was not well understood by the team. | “I kind of wish that we had received more. Not like, “We're at this practice right now', but I think that having a little bit more monitoring of what constituted visits or contacts with the practices would have been helpful.” | |
| Lessons learned | Detailed guidelines for external quality improvement support, including processes for practice facilitation and communication, could improve project outcomes and team cohesiveness. | |
| Underused experts | Expert physician consultants were a highly valuable resource that were underused. | “I think that 100% of the providers who would have benefited from that but… talked to expert faculty, no.” |
| Lessons learned | Guidelines for interacting with expert consultants could improve practice adoption and sustainability of clinical intervention and transformation strategies. | |
| More intense, less remote facilitation | The 3-month intense intervention phase distinguished by direct employment of coaches and expert consultants to support practices was considered too short. This was followed by a 9-month maintenance phase that was viewed as less useful than the intense phase. | “It's way too short of time to actually transform a practice. It was only three months of active intervention.” |
| “I would have liked to have seen a longer active phase where we were allowed to be onsite with them and a shorter virtual phase.” | ||
| Lessons learned | A longer intervention phase where practices have face-to-face interaction with coaches and expert physician consultants could improve practice adoption and sustainability of clinical intervention and transformation strategies. | |
| External burden on practices | The burden placed on practices by external sources, including insurance requirements and similar government initiatives, has a negative influence on practice engagement with the project. | “I think [what] AHRQ under anticipated was the fatigue among small practices.” |
| “In the context of practices being very busy, right, and really challenged to carve out time for this.” | ||
| Lessons learned | Recruitment should involve assessing practices' time availability and organizational capacity for quality improvement. | |
| Evaluation and Monitoring | ||
| Difficult data extraction | The evaluation team experienced numerous challenges in extracting data, which included variability in EHR type and use, difficulty achieving approvals, and a lack of practice knowledge of EHR capabilities. | “‘Why aren't they looking at their data?’ I think through this process I've come to learn, ‘Because it's really hard.’ These [EHR] systems are really not very good, most of them. They were designed for billing.” |
| “My smaller independent practices had just never sat down and learned how to use their EHR, but my larger health systems didn't have access to it.” | ||
| Missed opportunity for practice feedback | The difficulties extracting data and/or gaining access to data led to an inability to provide practices feedback reports for quality improvement. | “This project would have been very different if we had been able to deliver real-time data back to the practices as we thought we could when we started.” |
| Lessons learned | Recruitment should target practices with specific types of EHRs and/or capabilities. | |
AHRQ, agency for health care research and quality; HVH, heart of virginia health care; EHR, electronic health records.