Abstract
Purpose: The American Board of Medical Specialties' Performance in Practice (“Part IV”) portion of Maintenance of Certification (MOC) requirement provides an opportunity for practicing physicians to demonstrate quality improvement (QI) competence. However, specialty boards' certification of one physician at a time does not tap into the potential of collective effort. This article shares learning from a project to help family physicians work in groups to meet their Part IV MOC requirement.
Methods: A year-long implementation and evaluation project was conducted. Initially, 348 members of a regional family physician organization were invited to participate. A second path was established through 3 health care systems and a county-wide learning collaborative. Participants were offered (1) a basic introduction to QI methods, (2) the option of an alternative Part IV MOC module using a patient experience survey to guide QI efforts, (3) practice-level improvement coaching, (4) support for collaboration and co-learning, and (5) provision of QI resources.
Results: More physicians participated through group (66) than individual (12) recruitment, for a total of 78 physicians in 20 practices. Participation occurred at 3 levels: individual, intrapractice, and interpractice. Within the 1-year time frame, intrapractice collaboration occurred most frequently. Interpractice and system-level collaboration has begun and continues to evolve. Physicians felt that they benefited from access to a practice coach and group process.
Conclusions: Practice-level collaboration, access to a practice coach, flexibility in choosing and focusing improvement projects, tailored support, and involvement with professional affiliations can enhance the Part IV MOC process. Specialty boards are likely to discover productive opportunities from working with practices, professional organizations, and health care systems to support intra- and interpractice collaborative QI work that uses Part IV MOC requirements to motivate practice improvement.
Despite that demonstration of competence in Practice-Based Learning and Improvement (PBLI) has been embraced across the medical education and medical practice continuum for more than a decade,1 development of quality competence and improvement in quality itself continues to lag.2 The requirement for recertification, now embraced by the member boards of all 24 American Board of Medical Specialties,3 presents an opportunity to engage physicians in improving the quality of medical care.4 A recent study showed high rates of family physician participation in the Maintenance of Certification (MOC) process.5
The Part IV portion of MOC, Performance in Practice, requirement engages physicians in assessing their quality of care, participating in a quality improvement (QI) project, and reassessing their individual contribution toward the effort. Part IV MOC provides a strategically important opportunity for QI and for the development and assessment of PBLI competence. However, whereas most QI involves engaging teams within a practice and support across systems and multiple practices,6⇓–8 medical specialty boards certify single physicians. Although there is an emerging use of a small number of institutionally based QI programs that meet Part IV MOC requirements, for the vast majority of physicians, the Part IV MOC opportunity is limited by its one-physician-at-a-time approach.
Therefore, with support from the American Board of Family Medicine Foundation, we undertook a learning initiative to support physicians in working in groups to meet their Part IV MOC requirements. Our purpose was to provide “on the ground” experience through the process of engaging multiple physicians and their practices, and when possible their health care system and community partners, in working together to improve practice while meeting Part IV MOC requirements and to identify opportunities to expand group approaches to QI through Part IV MOC.
Methods
Design
This initiative was deemed exempt by the Case Western Reserve University Institutional Review Board as an educational and QI project. A team consisting of members of a practice-based research and development support unit and an educational and QI consultant worked with a local family physician professional organization, local health care organizations, and individual physicians and practices to conduct and learn from shared Part IV MOC practice improvement projects.
Participant Sampling Procedure
Figure 1 illustrates the 2-phased participant engagement process. The first enrollment path focused on individual physicians by mailing, and then E-mailing, an introductory letter and interest/needs survey to the 348 members of a regional family physician professional organization. Several weeks later, a targeted E-mail was sent by the American Board of Family Medicine to their diplomates within the regional catchment area who were required to complete a Part IV by the end of years 2011 and 2012 to stay on the MOC-Family Physician (MC-FP) pathway.
Participant engagement. *One practice divided into 2 working groups: a Patient Experience Survey (PES) working group and a Diabetes Performance in Practice Module (PPM) working group.
The second enrollment path involved recruiting participants at the group level. Although the majority of the physicians among these groups were contacted in the initial mailing, efforts to enroll physicians through their participation in professional groups focused on (1) a county-wide data sharing collaborative, (2) a large integrated health system, (3) a large independent health system, and (4) a group of rural hospital-affiliated family physicians. Most of the family physicians affiliated with these groups also would have received the individual mailings, but the exact number is not known. Contact with these institutional participants was achieved through attending and presenting the project at already scheduled organizational meetings and learning summits. Simultaneously, several meetings were held with administrative leaders to begin to link the MC-FP Part IV project with ongoing QI initiatives in the systems.
Opportunities for Participants
A facilitation process was established by the project team and followed for each participating physician and practice. An initial consultation was held between physician(s) who expressed interest in the MOC QI Collaborative and a practice coach, an experienced clinical research facilitator cross-trained in an 11-month Primary Care Practice Coach Training Program developed and facilitated by the Institute for Health Care Improvement. The physicians' and practice's priorities for improvement were discussed during this meeting. In addition, an alternative Patient Experience Survey (PES) module was introduced, and interest in using this alternative module versus a standard module was assessed. QI needs and various ways of beginning to meet those needs also were discussed. The QI education framework was adapted from the Health Improvement Skills Center and the Plan, Do, Study, Act cycle adapted from the Institute for Health Care Improvement “How to Improve” website.9 This framework allowed for a basic and fundamental approach to learning QI methods.
For those using the PES alternate module, the practice coaches facilitated survey distribution before and after the improvement project and provided peer-comparison feedback reports. In subsequent improvement project consultations, practice coaches supported the physician(s) and practice colleagues in examining the results of the baseline patient experience survey or the data collected using a standard American Board of Family Medicine module. They discussed possible QI projects, selected one to pursue, and planned a timeline for completion.
During this same meeting, coaches typically supported detailed project planning with an 9-question Quality Improvement Project Plan (Appendix A). When working with the physicians/practices, the practice coach sought to identify and assess the QI skills and knowledge that physician(s) and their colleagues already possessed. This helped the practice improvement team identify QI assistance needs and helped the participants to identify learning opportunities. Types of assistance needs included help with defining the problem or area for improvement, designing the intervention, determining outcome measures for their chosen intervention, and interpreting data generated by the intervention. This support and the practice and meeting structure and processes were documented in project field notes collected by all team members.
Practice members also were encouraged to access online resources and tools10 during the implementation phase of the QI effort. The practice coach was available by telephone and E-mail as well as through face-to-face meetings should additional support be required.
Practice “check-ins” usually were conducted 3 weeks after implementation of the QI project, when participants had had an opportunity to test their intervention. After collecting follow-up PESs (or chart audits for a standard module), participants were asked to complete a QI Project Assessment (Appendix B). This document allowed physician(s) to critically assess the QI project both from the perspective of the team's and their individual work. The final consultation meeting between the practice coach and QI project team was used to discuss the results of the follow-up data, the success of the QI intervention, lessons learned, and next steps in sustaining or altering their efforts.
Data Collection
Multiple data collection mechanisms were built into the project: an interest/needs survey during recruitment, a participant-tracking document, detailed field notes from the consultations, aggregate and individual scores from the PES, and QI project planning and report forms. The project team met routinely to share meeting notes and to discuss common themes and observations throughout implementation of the collaborative. The subset of physicians participating in the alternative PES module was asked to complete an evaluation form that assessed the utility of the alternative module.
Analysis
For each physician and practice participating in the PES module, anonymous summaries of patient responses were created. To identify themes of what worked to engage participants in working together to meet Part IV MOC requirements, the authors periodically reviewed and reflected on field notes and identified common themes. These themes were discussed and refined throughout the project. The level of collaboration was documented throughout the project and results were tabulated. Those physicians participating in the PES alternative module were surveyed, responses were tabulated, and summary statistics were created.
We also used the participant-tracking document data to create a matrix of interactions among participants and then used UCINET (version 6.354) software11 to map interpractice collaborations within the MOC IV collaborative for the purpose of illustrating the various levels of collaboration.
Results
The individual recruitment path yielded 48 physician responses; of these, 12 participated. The group recruitment path yielded 66 additional participating physicians, totaling 78 participants in 20 practices. Forty-one physicians opted for the alternative PES module, whereas the remaining 37 physicians completed a standard Part IV MOC module. In one practice, half the participants used the alternative patient experience and half used a standard module. All participants completed the project (0% attrition).
As shown in Table 1, QI projects motivated by the PES and standard modules included topics surrounding health maintenance and preventive screening; chronic disease self-management and decision support; improving access, workflow and efficiency, care coordination, chronic disease management; and improving the patient/practice experience. Physicians, practices, or both made contact with a practice coach an average of 10.3 times either by telephone, E-mail, or in face-to-face meetings.
Within the 1-year project time frame, intrapractice collaboration (group work) occurred in all participating multiphysician practices. Intrapractice collaboration was considered to occur when more than one physician in a particular practice worked together on a Part IV QI project. Practice teams ranged from an individual to 8 members and frequently included nurses, medical assistants, office managers, and clerical staff.
Interpractice collaboration was said to occur when ≥2 practices met to share their experience with Part IV QI interventions. A smaller amount of interpractice and system-level collaboration took place during the project year, and this continues to evolve. For example, several of the health care systems and a county-wide data sharing and learning collaborative12,13 continue to plan ways to integrate Part IV MOC with their institutional QI efforts. In addition, a rural hospital primary care institute is using the PES alternate module as a framework for collecting discharge information on patients and working with physicians to improve the experience of patients after discharge and reduce rehospitalization. Eight practices continue to meet once per month to support and share their QI learning.
Figure 2 depicts collaborations within the MOC IV collaborative. This figure shows that the practice coach was centrally involved and that the organizations from which we recruited and several physicians within large, integrated health care systems were also central to the process.
Map of interpractice collaborations around Part IV Maintenance of Certification.
Upon completion of the PES alternate module, this subset of physicians was asked to complete a self-assessment survey reflecting their experience with this practice coach–assisted, group approach to QI (Table 2). This survey yielded an 85% response rate. Of the 18 respondents, 78% rated access to a practice coach highly valuable; 47% responded that the role of the practice coach was what they liked most. Of respondents, 100% felt that their QI competency had improved with participation, and 89% believe that they have enough confidence to complete another QI project.
Table 3 summarizes the observations important to facilitating collaborative group participation in Part IV MOC. In particular, the opportunity to work collaboratively with others on QI interventions was widely appreciated, and the practice coach was highly valued.
Discussion
This project is one of the first to describe the utility of supporting work in groups to complete medical specialty board MOC practice improvement (Part IV) projects. Working together to meet Part IV MOC requirements and having the opportunity to engage in a practice-individualized QI approach based on both patient experience data and group practice, input seemed to make the Part IV MOC experience particularly meaningful to participants.
Practice coaches were able to foster intrapractice collaboration by using peer-comparison feedback reports of PES results from the alternative module and similar data from standard modules and by supporting the assessment, QI, and collaborative processes. When participants examined data closely, particularly the PES results, this reflective process facilitated critical thinking and led to improvement projects that were tailored to local patient-identified needs. The process of reflecting on data established a space for open communication and shared learning where coaches routinely were able to help physicians identify what they already know about PBLI and help team members to use, share, enhance, and add to this knowledge. All points of communication, especially face-to-face meetings with the physicians and practice members, were critical in providing a supportive environment.
In early practice meetings, physicians seemed eager to complete a Part IV project quickly to “check it off of their to-do list.” This sentiment became less prevalent as physicians learned that they were able to take part in projects that were meaningful to them and that they were able to apply skills they already had to their QI work. Employing the PES was particularly helpful in fostering individualization of QI projects that were personally meaningful. Tying change efforts to personal meaning has been shown to be important to motivating engagement of busy primary care physicians in QI,14 and a locally tailored approach is an essential characteristic of successful practice change facilitation.14⇓–16
Physicians in practices that engaged other practice staff members as well as fellow physicians were more likely to develop a team approach to the intervention and to begin to establish a culture of practice-based learning. This may have contributed to many physicians' self-assessment that they felt confident about doing future QI projects. In many practices, the initial emphasis on fulfilling the Part IV requirement became secondary to achieving practice priorities and seeing an improvement cycle through to completion.
In the early phase of project planning, teams narrowed choices for possible topics for improvement through discussion and feedback from members as they shared ways in which they individually addressed particular issues or concerns. Involving physicians at a group level in this way established a platform for sharing best practices, which has been shown to be an effective QI approach.13,17 One group developed and refined shared goals for improvement and crafted shared plans for capacity building and task assignment. Practices that established multidisciplinary teams early on exhibited the most success in their efforts and were more likely to communicate with the practice coach on a regular basis, consistent with the emerging literature about the benefits of diverse teams.18⇓⇓⇓⇓–23
While we encouraged the use of supplemental QI tools and learning activities by way of web-based resources, uptake on these suggestions was low. Future efforts might be better advised to require, rather than just recommend, completion of a minimum number of QI learning activities on a phased, “just-in-time” basis. This would provide a more robust, shared mental map of QI principles and methods to team members, enhancing the efficiency, effectiveness, and generalizability of their efforts, and better advance the QI capability development agenda of the MOC Performance in Practice (Part IV) requirement.
Initially we had anticipated supporting interpractice collaboration while simultaneously supporting group work on QI projects within individual practices. What emerged was that practice team members needed to develop their ability to work with each other first. Efforts to support a greater degree of interpractice exchange and collaboration are ongoing. Collaboration across groups of practices often involves negotiating institutional-level agreements, data use and sharing agreements, and use of information technology and institutional QI resources. Supported by our Practice-Based Research & Development Shared Resource24 (a core facility of our National Institutes of Health–supported Clinical and Translational Science Collaborative and Comprehensive Cancer Center), this interpractice Part IV MOC QI work continues to develop and seems to have great potential to institutionalize collaborative QI work. Our ongoing efforts in this realm particularly focus on engaging family medicine practices that serve underserved patient populations and that have been shown to be under-represented in the MOC process.5
A number of physicians have expressed trepidation with MOC, referring to it as both a “burdensome” and “questionable” process.25 Citing time as a barrier to completing MOC requirements, some physicians have described the MOC process as unfair and unneccessary.26⇓–28 Actively encouraging and supporting QI work in groups and providing a practice coach seems to ease this trepidation by helping physicians to see and experience the MOC Part IV requirement as a realistic and approachable objective that can meet larger practice improvement aims.
Strengths and Limitations
Coaching was a key part of the intervention. The availability of a practice coach seemed to be directly related to the success of getting groups of physicians to work together to meet Part IV requirements. Two emerging infrastructures that could provide the facilitation16 necessary to advance use of Part IV MOC requirements to stimulate group QI are (1) the involvement of practice-based research and best practices networks17 and (2) adoption of a Primary Care Health Cooperative Extension Service, as advocated by Grunmbach and Mold,29 included but unfunded in the Affordable Care Act and currently being tested in multiple states.30
The facilitation process happened organically, which contributed to the increased communication and relationship building; however, this sometimes revealed human dynamics that required a deeper level of coaching. This level of complexity should be considered when coaches are introduced to the practice, and they should be equipped with the skills necessary to address these complexities.
We were limited by the 1-year time frame to fully develop interpractice-level collaboration, but development of interpractice collaboration continues. Eight practices continue to meet once per month and are concurrently developing QI projects with their large, integrated regional health system; a rural hospital primary care institute continues to advance use of hospital-collected patient experience data to support care after discharge and reduce readmissions among the independent practices that admit to the hospital. The data sharing and learning collaborative continues to evolve Part IV MOC support among its learning collaborative across family medicine and soon internal medicine practices. While other health systems have shown interest in working across practices, it will take ongoing effort to make this happen. Interpractice-level collaboration benefits from sponsorship at the administrative level as well as the intrapractice level to facilitate widespread support of practice-based learning.14,15 It is also important to note that participants in this project were volunteers and are not necessarily representative of the entire population of family physicians.
Implications/Next steps
Timely, competent coaching of the work of improvement teams, particularly when practices are widely dispersed, is labor intensive. Therefore, for this approach to be replicated widely and, ideally, incorporated into medical specialty board MOC systems, it will be necessary to develop and test a variety of efficient and cost-effective strategies for preparing QI coaches and for scaling up the availability and provision of coaching for those engaged in MOC-related improvement projects. A web-based interface serving as a professional network site for participants might encourage the development of regional partnerships and a culture of shared learning. Attention to independent, solo, rural, and inner-city practices5 will be particularly important to avoid creating a QI divide between practices with access to QI support resources and more isolated practices.
Conclusions
This project demonstrates the potential and the process for group approaches to Part IV MOC to engage physicians and their practice, system, and organizational partners in working together to improve practice. Further work to refine and support group approaches to Part IV MOC seems to be warranted.
Appendix A: Quality Improvement Project Plan
Clinician:
Coach:
Project title:
Date:
Instructions
Please fill out the following Improvement Project Plan. Then revise it as you go, incorporating the ideas that emerge from the work of the improvement project team you assemble for this effort.
In your current work situation, do you have any measures of the quality of your (and your team's) performance that could serve as a basis for selecting an improvement target? If yes, please describe.
What is your target for improvement? Please be as specific as possible, even if you do not have measures of current performance.
List several options for changes you might make that could help you achieve your target for improvement.
Of the options listed in your response to number 3, which specific change in work or activity will you test in an attempt to achieve your target for improvement?
Whom might you need to involve in attempting the change?(Also, who are the other members of your improvement team?)
What might impede or block your intervention?
What additional resources might you need in attempting the change?
What measurement will you perform that will enable you to determine, if change occurs, whether it is an improvement, given your target? Please be specific as to how you will measure your progress.
Describe how this plan uses quality improvement principles and methods.
Appendix B: Quality Improvement Assessment Form
Clinician:
Coach:
Project title:
Date:
Instructions
Please fill out the following Improvement Project Report.
Did you reach your target? Describe the evidence you used to make this judgment.
Describe in more detail what you and your team did to achieve these results. Include specific information about your personal role.
What modifications, if any, did you make in your original change idea as you put it into practice?
Describe any issues or challenges you experienced when measuring the impact of the change idea on your (or your team's) performance?
What did you think you did particularly well in making this change?
What might you do differently the next time?
If your change resulted in an improvement in performance, what are you planning to do to sustain this improvement?
If the change did not result in an improvement, what will you try next?
Acknowledgments
The authors are grateful to the participating physicians, their practice and system partners, the leadership of the Northeast Ohio Academy of Family Physicians, Better Health Greater Cleveland, participating health systems, and the funders, all of whom made this QI and educational work possible. The authors also are grateful to the staff of American Board of Family Medicine for their advice and technical support on this project, to Caroline Carter and Bonnie Hollopeter for their support group work as practice coaches with Better Health Greater Cleveland, to Joshua Terchek and Michelle Hamilton for facilitating the launch of the project, and to Aleece Caron, who served as a QI Consultant.
Notes
This article was externally peer reviewed.
Funding: This project was supported by a grant from the American Board of Family Medicine Foundation and by the Practice-Based Research and Development Core of the Case Western Reserve University/Cleveland Clinic CTSA Grant No. UL1 RR024989 from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH) and NIH Roadmap for Medical Research and the Case Comprehensive Cancer Center, Cancer Center Support Grant P30 CA043703 from the National Cancer Institute. Dr. Stange's time is supported in part by a Clinical Research Professorship from the American Cancer Society. The manuscript contents are solely the responsibility of the authors and do not necessarily represent the official view of ABFM, NIH or ACS. The funders had no role in project design or conduct; data collection, management, analysis, or in interpretation of the data; or preparation, review, and approval of the manuscript.
Conflict of interest: none declared.
- Received for publication September 20, 2012.
- Revision received November 26, 2012.
- Accepted for publication December 3, 2012.
