Abstract
Background:Long-term sustainment of improvements in care continues to challenge primary care practices. During the 2 years after of our Improving Depression Care collaborative, we examined how well practices were sustaining their depression care improvements.
Methods:Our study design used a qualitative interview follow-up of a modified learning collaborative intervention. We conducted telephone interviews with practice champions from 15 of the original 16 practices. Interviews were conducted during a 3-month period in 2008, and were recorded and professionally transcribed. Data on each of the depression care improvements and the change management strategy emphasized during the learning collaborative were summarized after review of the primary data and a consensus process to resolve differing interpretations.
Results:During the period from 15 months to 3 years since our project began, depression screening or case finding was sustained in 14 of 15 practices. Thirteen practices sustained use of the 9-item Patient Health Questionnaire for depression monitoring, and one additional practice initiated it. Seven practices initiated self-management support and 2 of 3 practices sustained it. In contrast, tracking and case management proved difficult to sustain, with only 4 of 8 practices continuing this activity. Diffusion of use of the 9-item Patient Health Questionnaire to other clinicians in the practice was maintained in all but 3 practices and expanded in one practice. Six of the practices continued to use the change management strategy, including all 4 of the practices that sustained tracking.
Conclusions:Practices demonstrated long-term sustained improvement in depression care with the exception of tracking and care management, which may be a more challenging innovation to sustain. We hypothesize that sustaining complex depression care innovations may require active management by the practice.
“Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime.” —Chinese proverb
This well-known proverb succinctly captures the hypothesis underlying the Improving Depression Care study started in 2005 by the American Academy of Family Physicians’ National Research Network and the American College of Physicians Research Network. Further stated, our goal was to teach practices a change management method for implementing and sustaining quality improvement, in the context of a study about improving depression care processes. We believed that if practices could learn and implement the change management process, ie, “learn to fish,” they would be more successful in implementing and sustaining the depression care improvements. This article builds on findings from our previous report on short-term outcomes,1 and specifically addresses the long-term sustainability of depression care improvements in our participating practices 2 years after the completion of our intervention.
The larger context of our work highlights the difficulty and importance of implementing and sustaining change in primary care. With respect to depression, Wells and colleagues2,3 have published 2 reports based on 9-year follow-up data from the intensive Partners in Care study, which show modest effects at best. Beyond the focus of a single chronic disease, the national push to pursue the patient-centered medical home has captured the imagination of insurers, employers, Medicare, and state governments.4–10 The recent TransforMED National Demonstration Project experience illustrates the challenges of instituting broad change, implicit in patient-centered medical home models, in busy primary care practices.11
We sought a model for intervention in practices that did not require on-site work and that could therefore be scaled; we chose a quality improvement collaborative model for our project. Collaboratives have gained momentum for enhancing clinicians’ ability to plan and test local improvement in health care.1,12–18 Collaboratives typically bring together groups of clinicians, administrators, and top-level leaders from different health care organizations to a series of “learning sessions,” during which participants are exposed to best practice guidelines and the evidence supporting them, as well as change strategies for implementing and sustaining local improvements. During subsequent learning sessions, participants share experiences of making changes in their own settings. Research has not yet established conclusively whether collaboratives consistently lead to sustained improvement changes,19–22 although reports from some collaboratives have been encouraging.12,16,23,24
The goal of this article is to report on both the change management processes and the depression care improvements that practices were using 2 years after the conclusion of our Improving Depression Care intervention. This extends the findings presented in our original article that reported on improvements at 15 months.1 Directly related to our belief that “learning to fish” and “eating for a lifetime” are related, we also seek to illuminate possible relationships between practices’ use of the change management process and their continued depression care improvements.
Methods
The Improving Depression Care project tested the use of a modified improvement collaborative approach to improve depression care in 16 small- to medium-sized primary care practices. Practices were recruited in January 2005 from the American Academy of Family Physicians’ National Research Network and the American College of Physicians Research Network.1 Although the primary purpose of the collaborative was to improve depression care, a great deal of attention was also given to implementing a systematic change management process that would enable the practices to more easily make improvement changes after the collaborative was over. At least 2 champions from each practice, a physician and a nonphysician staff member, both of whom were selected by the practice based on their interest and ability to participate, attended 3 weekend learning sessions held April, June, and November of 2005 in Chicago, IL. During each learning session, principles of both depression care improvement and practice change management were presented. After each learning session, the practice champions returned home to carry out the action plans they designed to implement the concepts taught during each 2-day session.
A key feature of our intervention was the use of the Reflective–Adaptive Process (RAP) model for practice change management.25 The RAP model is based on principles of complex adaptive systems and emphasizes reflective use of rapid-cycle tests of change led by practice improvement teams. We hypothesized that the RAP model would not only enhance practices’ ability to implement improvements in depression care, but would also be used for other improvement changes as chosen by the practices. In a previous report we presented 9- and 15-month outcomes that demonstrated that most practices were able to implement use of the 9-item Patient Health Questionnaire (PHQ-9) for case finding and monitoring, but a minority were able to implement tracking, care management, and self-management support. At 15 months, 6 months after the end of the intervention, nearly all of the improvements that were implemented were sustained.1
In this study, we sought to document and understand how well practices were able to sustain their improvements over a much longer period—nearly 2 years—after the collaborative ended. In late 2007, all 16 practices that completed the original project were invited to participate in follow-up telephone interviews; 15 practices agreed to participate. We sought to interview at least one practice champion from each practice; we interviewed the second champion where we anticipated divergence or needed additional clarity. The interviewees were offered $100 to reimburse them for the time taken up by the interviews, and they provided recorded informed consent at the beginning of each interview. Institutional review boards of the University of Michigan Medical School, the American Academy of Family Physicians, and the University of Colorado, Denver, approved this follow-up study.
Similar to our original study, the study team developed an interview guide and template for the telephone interviews. Our interviews focused on assessing practices’ continued use of the depression care interventions emphasized in the learning sessions: use of the PHQ-9 for screening and severity monitoring, tracking and case management, and self-management support. Because of our hypothesized link between sustainability and change management, we also asked whether and how the practices were still using the RAP-based change management principles.
To enhance reliability of our judgments about sustaining components of depression care, the study team agreed in advance on operational definitions of characteristics for the depression care components. For example, we asked specifically how the PHQ-9 was used and made clear the distinction between screening (in the absence of depression symptoms), confirming a diagnosis (when symptoms were considered to be present or the patient was considered to be at risk), and monitoring severity of depression symptoms during follow-up visits or phone contact. Inquiry about tracking and case management specifically asked about identifying a group of patients with depression to keep track of needed services, whether or not patients made an office visit. We gave credit for tracking a subset of depressed patients (for example, based on severity), and did not require that all depressed patients were tracked. Case management was considered present if practice staff systematically contacted patients between visits to review symptoms and barriers to medication adherence. Contact could be by telephone or e-mail. Mailed reminders to patients about follow-up visits for depression were considered tracking but not case management. Finally, self-management support was assumed to be present if a member of the practice negotiated a specific behavioral goal with the patient and inquired about progress and barriers during a future visit.
Similarly, when assessing maintenance of the RAP process, we inquired about and considered the frequency of meetings, the extent to which the team included individuals from all parts of the practice, and the use of rapid-cycle tests of change. Although we specified the RAP during the learning sessions, we “gave credit” for any evidence of iterative consideration of what was or was not working.
Interviews with at least one champion from each practice were conducted from January through March of 2008, and were performed by a study team member who had acted as one of the practice's coaches during the collaborative. Interviews were audio recorded when possible and were supplemented by recorded field notes. An independent transcriptionist transcribed each recorded interview. All data were added to the AtlasTi (Atlas.ti Scientific Software Development GmbH, Berlin, Germany) dataset from the original project.
We approached the analysis with 3 primary analytic questions. First, What aspects of improved depression care were practices sustaining 2 years after our project completion? Second, Were practices continuing to use the RAP-based change management process? Finally, Was there a discernable relationship between continued use of the change management process and sustainment of depression care improvements? We used a template approach to coding and analyzing the follow-up interview data. We also took advantage of the original coded data for constructing an understanding of the process of change and maintenance of change for each practice. Similarly, a table was constructed that indicated how well practices sustained the RAP model. Last, we also developed a table that indicated how practices had changed on both of these aspects since the final data collection in the original study. Study team members (DN, PN, DG, MJ) conducted the primary data analysis. The practices’ original study team coaches then reviewed conclusions from the primary analysis.
Results
Figure 1 depicts the number of practices using our original project's recommended depression care improvements from completion of the original 9-month intervention, through 15 months, until our follow-up at 3 years. From our 15-month follow-up, 14 of 15 practices continued to use the PHQ-9 for screening and case finding (this aspect could not be verified for one practice); monitoring with the PHQ-9 was sustained by 13 practices and taken up by an additional practice. Self-management activity seemed to increase dramatically: 2 out of 3 practices continued their activity and 7 practices initiated activity during this period. Notably, use of tracking systems and case management decreased from 8 to 4 practices.
Number of practices implementing and sustaining depression care improvements from the end of the original study (9 months) to 36 months.
The RAP model of change management was not sustained in most practices. Although some practices reported remnants of the RAP process, only 6 practices reported vigorous activity consistent with the principles of the RAP change management strategy.
Because most practices continued their use of the PHQ-9, the hypothesized relationship between sustaining the RAP and depression improvements was most apparent for tracking and care management. Figure 2 depicts the complex relationship between implementation and sustainment of the RAP process and tracking and case management over time. At 9 months there was little relationship between tracking and case management and the RAP process. This is depicted by the heterogeneity in the shapes and colors representing the practices in the 9-month cells. However, at 15 months, fewer of the practices with no or only modest RAP use were able to continue tracking and case management, and 2 more practices who did sustain RAP use had implemented tracking and case management. Finally, at 3 years, none of the practices with no RAP use and only one practice with modest RAP use was able to continue tracking and case management. The following case study illustrates, using the findings from one participating practice, how the RAP change management process and depression care improvement were related.
Relationship between vigorous Reflective–Adaptive Process change management strategy and sustained tracking and case management by practice. Numbers within shapes designate individual practices.
Case Study: Sustaining Changes in a Small, Private Practice
Sunnyside Family Health Center is a small, private practice consisting of 2 physicians, a nurse practitioner, an office manager, and 10 full- and part-time support staff (the names of the practice and personnel have been changed to protect anonymity). The practice had been open for 6 years before joining the project. One of the physicians (Dr. Jones) and the nurse practitioner (Heather) participated in the project as practice champions and attended all 3 learning sessions. They were excited at the possibility of improving staff communication during this project. After the first learning session, they formed an improvement team that included Dr. Jones, Heather, the office manager, a billing person, a nurse, and a front desk person. They initially met semimonthly and engaged in some of the recommended team-building exercises. They also addressed implementation of the PHQ-9, and both Dr. Jones and Heather tried it during several RAP cycles. They also identified other issues related to patient flow during the office visit. “We didn’t have anything like this in place before. This was a whole new way of thinking. We had wanted to do something similar, but didn’t know how.”
The office manager was initially supportive of the improvement team, although she was not particularly comfortable with the staff's input that was generated by the process. She had a fairly hierarchical administrative style and by the second learning session it was reported that she had stopped attending the meetings. Dr. Jones and Heather felt that there was a lack of buy-in by the other physician and the office manager and discussed this during the second learning session. They received some suggestions during a facilitated group discussion and returned home with the intent to more fully explain the change management process to the office manager and other physician. This apparently went well and Heather commented later that they should have done that more completely after the first learning session or even included the office manager as a practice champion in the collaborative. “The office manager has come on board; she was the most resistant. But she sees that people are happier, people work together instead of her making all the decisions.”
The improvement team continued to meet and by the project's end the team had fully implemented the PHQ-9 for screening/detection and for monitoring severity. All 3 clinicians actively used the PHQ-9, and the medical assistants (MAs) were involved in administering the PHQ-9 to patients making return visits for depression. “Before we didn’t have a way to do this. Now, if the MA knows the patients have a complaint, they’ll give the PHQ before the provider comes in the room. When the patient comes back for follow-up, we give the PHQ for tracking progress.”
They developed a depression flow sheet and incorporated the PHQ-9 into their electronic medical record (EMR). Heather began a manual tracking system for all the practice's depressed patients. She also began to address patient self-management, negotiating behavioral goals with her own patients using material from the collaborative.
At the 3-year follow-up, all 3 clinicians continued to utilize the PHQ-9 for screening and monitoring severity. Heather continued the manual tracking system for all patients and used self-management support strategies with her own patients. The practice also continued the principles of the RAP process, although they had adapted them for their own practice circumstances. In Heather's words:
“This is a small practice. We set aside 2 hours every month to meet as a full office (including billing, front office, providers, MAs) and call it our ‘happy hour.’ During the month, though, each section of our office (billing/front office, MA staff, and providers) meets and discusses issues that have come up since the last ‘happy hour’ that affect them directly. The main focus of this meeting is not to gripe but to come up with solutions for issues that have come up. This was stressed highly and the office seemed to agree with this approach.”
“After each of these groups meet, they then send a representative from each of the groups to meet in a ‘miniRAP,’ where the issues are brought out and solutions discussed. Generally, the solutions are implemented and when we meet again for the ‘happy hour’ we discuss if these solutions helped and if not we come up with other ways with the whole group. We also use the ‘happy hour’ to do team-building activities or to present different topics that affect our office like confidentiality, [cardiopulmonary resuscitation] CPR, infectious disease control, etc. …”
“On the whole, things have gotten much better since implementing the RAP in this way. Each person in our office has a way to be heard and the office meets as a whole, which has never happened before but was much needed.”
The office members also began to address other chronic disease strategies. For example, they made a checklist for diabetic patients to indicate any problems with medications, foot care, etc, since the last visit. The MA gives the form to patients during the rooming process. “Once we saw how the PHQ made this easier for depression, we applied the same process. Diabetes is really complex, our EMR isn’t great, [and] we still need to work on a better means for tracking.”
Discussion
This study adds additional information to our original work on promoting sustainable depression care improvements in primary care practices.1 During follow-up we found that most improvement changes were sustained more than 2 years after the last learning session. The use of the PHQ-9 seemed to be most durable, and maintenance of a tracking process seemed to be the most fragile. The use of self-management support strategies continued to increase. Although we can only speculate about the differential sustainment among depression care components, it is likely that most practices view the use of the PHQ-9 as less resource intense and requiring less systemic practice-level change. Although most practices reported involving the nursing staff in the use of the PHQ-9, it could be maintained solely by the intention of the individual clinicians. In contrast, tracking systems by their nature are more complicated and, unless embedded seamlessly in an EMR, usually involve specific data entry and monitoring activities that must be assigned, usually as new duties, to someone in the practice. The use of self-management support falls somewhere in between. Although it could be implemented solely at the discretion of the clinician, most clinicians believed that it would consume more of their time and would require some training to prepare MAs to negotiate goals with patients and some systematic processes to follow up during subsequent visits. However, given that self-management use increased over time, we hypothesize that, with PHQ-9 monitoring, clinicians noted that certain patients were more resistant to treatment, and therefore practices began to reach for another tool to assist with depression care.
Consequently, it seems that use of the PHQ-9 is the “low-hanging fruit” in efforts to improve depression care. Several older clinical trials demonstrated that increased detection of unrecognized depressed patients alone would not completely solve the problem of undertreatment and poor outcomes. Two studies suggested that increasing recognition did not improve outcomes,26,27 and another study28 suggested that treatment of newly recognized patients was suboptimal. It is, therefore, very encouraging that all but one of our practices are also monitoring patients’ progress over time with the PHQ-9.
There is a small but growing amount of literature about sustaining quality improvement changes29,30 that remain robust under changing circumstances. Although our ability to draw strong conclusions is limited by the number of practices in the study, we hypothesize that there is an association between a practice maintaining the RAP change strategy and the maintenance of the more complex depression care components that were originally achieved. Maintenance must occur in the face of other constantly evolving demands faced by medical practices. Tracking and case management of depressed patients require the ongoing involvement and interaction of several individuals in a practice. Improvement changes in primary care practices are fragile and can deteriorate because of a variety of circumstances, ranging from staff turnover to attention being focused on another clinical area. For example, implementation of EMRs often cause a huge disruption in a practice and can easily displace previous improvement changes. In this analysis, all 4 practices that were able to continue the tracking system also continued to use the RAP process. We believe that sustainment of complex improvement changes requires a level of active attention29 to hold the gain and integrate it into routine operations in the context of competing priorities and frequent staff turnover.31 Although our data represent an association and cannot be assumed to show a causal relationship, we suspect that the RAP process, with its emphasis on protecting time for reflection on improvement activities,25 might have some of the characteristics needed not only to implement change, but to sustain it as well. Future research needs to examine how a change management strategy works not only to support change initially, but also to sustain change over time.
Limitations of this study include the self-report nature of changes sustained (subject to inaccurate retrospective recall and even optimistic bias) and potential bias of the practice champions. However, given that our sources of information from the practices were the same practice champions during the 3-year period of reporting, any potential optimistic bias should have remained consistent across the time period. Our findings were enhanced by the opportunity to assess the durability of changes made 2 years after the last learning session. In addition, the varied nature of the improvements the practices were asked to undertake made it possible to illuminate possible differences in sustainability and the factors that could impact sustainability. Our ability to explore some of the possible practice-level relationships and interactions in this project was also limited by the small sample size.
Conclusions
At follow-up 2 years after the last learning session we found that most practices sustained their use of the PHQ-9, and some practices continued to make advances in the use of self-management support. The more complex process of tracking and case management, however, showed substantial decline. Although our longitudinal data cannot prove causation, we hypothesize that sustaining depression improvement changes is an active process, enhanced by the change management strategy that made up the core of the original learning sessions.
Acknowledgments
The authors would like to acknowledge the practices that participated in this follow up to the Improving Depression Care program: Baton Rouge Family Practice, Baton Rouge, LA; Carolina Family Health Center, Inc., Wilson, NC; Community Health Center of Snohomish County, Evergreen Branch, Everett, WA; Community Medicine Associates, San Antonio, TX; Deerbrook Medical Associates, Vernon Hills, IL; Hill Country Family Health Specialists, Leander, TX; ICME PSC, Caguas, Puerto Rico; Medicor Associates, Inc., Fredonia, NY; Prime Care Family Practice, Clinton, OK; Salud Family Health Center, Fort Lupton, CO; State University of New York, Upstate, Department of Family Medicine, Syracuse, NY; Torrance Clinical Research, Torrance, CA; University of Illinois at Chicago Primary Care Clinic at Mt. Morris; Valley Medical Center, Newcastle, WA; and Waldron & Holton Internal Medicine, Medford, NJ.
Notes
This article was externally peer reviewed.
Funding: Support for this research study was provided by the American Academy of Family Physicians Foundation through a grant from Forest Laboratories, Inc., and Pfizer, Inc.
Conflict of interest: none declared.
- Received for publication September 3, 2009.
- Revision received May 18, 2010.
- Accepted for publication May 24, 2010.
References
In this issue
Jump to section
Related Articles
Cited By...
- The role of quality improvement collaboratives in general practice: a qualitative systematic review
- Practice Transformation Support and Patient Engagement to Improve Cardiovascular Care: From EvidenceNOW Southwest (ENSW)
- How is the sustainability of chronic disease health programmes empirically measured in hospital and related healthcare services?--a scoping review
- It Takes Two: Using Coleaders to Champion Improvements in Small Primary Care Practices
- The Words We Use, the Procedures We Do, and How We Change (Medical Care Information for Family Physicians)