A Qualitative Analysis of Implementing EvidenceNOW to Improve Cardiovascular Care

Compressed Project Timeline

The project timeline posed one of the largest challenges for the HVH initiative. HVH team members felt the period from grant award notification to the required intervention start was “rushed.” The limited time made it difficult for HVH team members to complete project development tasks, which eventually compromised the original research design and shortened the intense intervention phase from 6 to 3 months. On reflection, most HVH team members believed the short project timeline was “completely unrealistic” and a precursor to other challenges.

Recruitment Challenges

The leadership team reported tremendous challenges recruiting 250 small-to-medium-sized practices, which they earlier thought possible because of their strong connections to the practice community. Recruitment was a challenge due to the short time frame and practice disinterest because of involvement with other government initiatives, particularly programs and requirements by the Centers for Medicare and Medicaid Services. Compromises, therefore, were made to the research design, including expanding the inclusion criteria to allow health system-owned practices to participate. Challenges were later experienced with many health system practices that were less engaged and less aware of the HVH initiative because health system leaders, not individual practice leaders, agreed to participate in the study. As 1 HVH team member explained: “practices who were signed up by health systems it was harder to get them engaged because basically, someone outside of the practice… signed them up.” In addition, according to interviewees, the hierarchical organizational structure of health system practices posed challenges because these practices did not have authority over many of their workflow processes, EHRs, and performance data.

Collaborative Research Team

One of the greatest strengths of the HVH initiative was the collaborative team of researchers, coaches, expert consultants, and evaluators. HVH team members thought of each other as “highly skilled,” “excellent,” and “knowledgeable.” Members of the HVH team stated they “worked well together” and believed the project “benefited significantly from this not being a single institution project.” HVH team members regularly held group meetings and referenced these meetings as pivotal for successful collaboration.

Intervention Flexibility

Intervention flexibility was a key component of research design, considering that participating practices were diverse in demographics, resources, EHRs, and experience with data measurement. This flexibility allowed practices to select intervention strategies that best suited their needs and goals. Many HVH team members viewed this customized approach as positive, whereas evaluation team members desired a more standardized approach for outcome analysis. As 1 evaluation team member pointed out: “It is hard research-wise because the menu they offered was huge…1 practice might have gotten X, and the other 1 might have gotten B, right? So, they are totally, totally different.” Several HVH team members suggested including a before and after assessment of practice transformation status and/or requiring that practice transformation activities serve as a prerequisite to a clinical intervention.

Well-Designed Practice Improvement Toolkit

HVH team members were highly pleased with the “toolkit” of resources developed for participating practices, which included checklists, educational/training materials, and successful case studies on practice transformation and ABCS implementation. The research team, particularly the leadership team, exhibited great pride in the completed toolkit: “What was positive is that we were able to write a ton of educational materials to practices, such as smoking cessation.” Still, despite its strengths, the research team recognized that the toolkit, as it was presented on the on-line platform was underused in the HVH initiative because people were “not logging in and using it.”

Successful Kickoff Event

The kickoff event was a highly positive and successful component of the HVH initiative. Interview participants described participation in the kickoff event as “critical” for initiating a practice's engagement and readiness and expressed viewpoints such as “practices that came and engaged that first day seemed much more likely to stay engaged.” Practices that did not attend the kickoff were believed to be at a disadvantage and required additional orientation to the project. Multiple HVH team members suggested that the kickoff event be a mandatory component of participation in future large-scale quality improvement initiatives.

Complicated Coaching Position

The intense intervention phase was distinguished by direct employment of coaches and availability of expert consultants to participating practices. Both coaches and expert consultants were highly valued components of the initiative. Coaches were responsible for supporting practices in selecting and implementing their customized strategies. Coaches served as the face and facilitator of the HVH initiative for practices and in extreme cases coaches were considered a “lifeline” to “drowning practices.” HVH team members had mixed feelings about the coaching organization's effectiveness in communicating challenges, concerns, and progress with practices. One HVH team member hesitated, then stated “the coaching organization…were not as great…as far as sharing what was going on.” Reflecting on these challenges, numerous HVH team members pointed to factors such as the coaching organization's “closed culture,” a lack of project “guidelines for coaching activities,” and constraints of the coaching organization due to concurrent participation in other initiatives such as the Centers for Medicare and Medicaid Services Transforming Clinical Practice Initiative.

Underused Experts

Expert consultants, regarded as an extremely valuable resource, were “were hardly used” and seen as a “missed opportunity” to help practices with improvement efforts. Several factors were offered as to why expert consultants were underused, including a lack of role clarity, a perceived lack of time or availability, and a lack of involvement with coaches. Both coaches and expert consultants voiced a desire to have expert consultants more involved with practices during the intense intervention phase.

More Intense, Less Remote Facilitation

Multiple HVH team members, particularly the coaches, described the intense intervention phase as “too short” to “transform a practice.” The lack of time in the intense intervention phase left many practices unprepared for the subsequent 9-month maintenance phase. The maintenance phase was designed for practices to independently continue implementing intervention strategies but was not a focal point of the initiative. Some coaches described this period as “hands off,” which frequently resulted in no communication between the HVH coach and practice staff. Most coaches expressed a desire for a longer phase to work closely with practices and a concern that HVH virtual resources, such as on-line materials, were underused. Many HVH team members also voiced concern over the lack of project closure. Numerous coaches interviewed for this study shared a desire for a closing activity that would incorporate strategies for practices to move forward with the intervention after the HVH initiative ended.

External Burden on Practices

HVH team members unanimously agreed that it was difficult to achieve the HVH clinical intervention and practice transformation goals considering the external burden placed on practices. Many coaches also noted a challenge with external demands on practices: “I've got practices that are trying to manage 5 major insurances and trying to do proactive or quality improvement with all their patients for each insurance company. And Medicare, and MIPS, and Meaningful Use, and it is just too much.”

Difficult Data Extraction

The HVH evaluation team worked closely with practices to obtain EvidenceNOW-required clinical outcomes data. However, the team experienced tremendous difficulty extracting data from practice EHRs. A member of the HVH evaluation team stated that getting data were a “6 to 12-month lag, and for some (practices) we cannot get anything unless we physically go get it, and we should not do that more than 2 or 3 times.” Most HVH leadership and evaluation team members indicated that they were not prepared for the challenges of working with a large number of practices with varying types and capabilities of EHRs. “I do not think any collaborative participating in this with AHRQ had any idea that that is how bad the state of reality versus what ONC was telling folks was. There was a huge disconnect.”

Challenges in extracting data differed based on practice ownership, which were mainly due to bureaucracy in health system practices and a lack of trained personnel and expertise in independent practices. The challenges in extracting EHR data from practices ultimately suspended the plan to provide each practice with performance feedback reports.

Missed Opportunity for Practice Feedback

Coaches and other HVH team members voiced frustration in not obtaining performance feedback reports as originally planned in the HVH initiative. Many HVH team members referred to the situation as a “lost opportunity” for practice improvement. The team stated that a clear majority of the participating practices did not have access to their HVH performance data. Multiple team members commented that they would have liked to have practices to be able to successfully integrate evidence-based material into their practice in a sustainable way but that goal was not achieved. As 1 member of the HVH coaching team stated: “we could have done a little more actual QI [quality improvement] work if we had seen that data, but it just really was not feasible.”