As the American Board of Family Medicine transitioned to Maintenance of Certification for Family Physicians (MC-FP) from the old recertification paradigm, I made several promises to our diplomates. I assured them that (1) we wanted to develop meaningful, continuous, long-term relationships with them as we worked together to help them deliver the highest-quality care to their patients; (2) we would listen carefully to the feedback they provided to us and act on it accordingly to make completion of their requirements as efficient as possible; and (3) we would endeavor to evolve MC-FP in keeping with the best evidence of assessment, measurement, and quality improvement science.
In keeping with these promises, the American Board of Family Medicine announces several important improvements in MC-FP. These include no longer requiring completion of the clinical simulation component of the Self-Assessment Modules (SAMs) for MC-FP Part II credit; transitioning everyone to the MC-FP point system; adding a new Continuous Knowledge Self-Assessment process to the Part II menu; and instituting a major discount in fees for Diplomates over the age of 70. We also announce new initiatives in practice transformation and physician burnout that we will be undertaking this year.
Unlinking the Clinical Simulation from Knowledge Assessment in the SAMs
We just completed an exhaustive review of all the evaluations that our Diplomates provided after completing their Performance in Practice Modules (PPMs) for Part IV and the SAMs for Part II. We shared with our Diplomates the preliminary results from the very positive feedback that was provided last winter with respect to the PPMs. These data have now been fully analyzed and a peer-reviewed manuscript has been accepted for publication this year. More importantly, our research staff just finished a thorough analysis of the SAM data. Unlike the PPM data, however, these data were more concerning.
While positive feedback with respect to the knowledge assessment portion of the SAMs was apparent, the assessment of the utility of the clinical simulation component was less favorable. Not only were the quantitative evaluations significantly lower, a qualitative analysis of >5 million open-ended feedback comments from 325,000 completed SAMs also revealed several important concerns with respect to technical and navigation issues in the simulations. In an effort to determine whether diplomate familiarity with technology and periodic technical improvements had affected the ratings over time, we analyzed a second data set from almost 100,000 SAMs completed more recently in 2013 and 2014.
The findings were essentially unchanged. Diplomates consistently rated the knowledge assessments more favorably than the clinical simulations. The majority of the negative comments about the clinical simulations revolved around 4 major issues: difficulty in ordering or scheduling tasks; inadequate recognition of questions or language by the simulator; limited medication, treatment, and diagnostic options; and the lack of “realness” in the simulation environment. We provided these data to our board of directors for their review during their April 2015 and October 2015 meetings. Between those 2 meetings, our Clinical Simulation Team, led by Senior Vice President Michael Hagen, undertook the task of making several technical improvements to the clinical simulation interface.
While these changes resulted in improvement in the clinical simulation evaluations during this brief period of time, our directors endorsed unlinking the clinical simulation and knowledge assessment components of each SAM, thereby making the clinical simulations optional, effective this year. Accordingly, the mandatory Part II requirement that at least 1 SAM be completed during each stage of MC-FP will be modified this year to mandate that at least 1 knowledge assessment, to be renamed Knowledge Self-Assessment (KSA), be completed in each stage, in addition to at least 1 Part IV activity. To avoid the confusion of the multiple permutations of Part II and Part IV activities that could be combined to meet our stage requirements, we will also be transitioning all diplomates to a point system to simplify how they meet their stage requirements.
Transitioning All Diplomates to the MC-FP Point System
Those diplomates who have entered continuous MC-FP (those initially certifying or maintaining their certification in 2011 and thereafter) are by now very familiar with this system. For those who are not, it simply requires the accumulation of 50 MC-FP points in each stage, with completion of at least 1 Part IV activity and 1 Part II activity. That mandatory Part II activity will now be a KSA instead of a SAM (knowledge assessment plus clinical simulation); the SAM terminology will no longer be used. Most Part IV activities (but not all) are valued at 20 points; the KSAs will now be worth 10 points each, and the clinical simulations, to be renamed Clinical Self-Assessment, will be valued at 5 points. After completing at least 1 Part IV activity and 1 KSA, diplomates will be able to mix and match any additional Part IV activities, KSAs, or Clinical Self-Assessments to reach the 50-point requirement.
As might be suspected, these major changes to the MC-FP Part II requirement and the conversion of all diplomates to the MC-FP point system will require significant reprogramming of our website and individual physician portfolios. We hope to have all these changes in place by July 1, 2016.
Development of a New Continuous KSA for Part II
Three major events occurring over the past 8 months significantly influenced our plans to revise our Part II platform for MC-FP, beginning this year. The first was the testing summit held by the American Board of Pediatrics that ABFM staff attended late last spring. A number of assessment experts brought the meeting participants up to date on best practices in the clinical assessment arena; a single, high-stakes examination was not at the top of their list. A consensus opinion developed by the end of the summit: frequent, smaller-scale assessment is a much more accurate way to measure knowledge and ability.
The second event was a presentation by the American Board of Anesthesiology (ABA) at the American Board of Medical Specialties (ABMS) meeting held in Chicago in June 2015. They provided results from their Maintenance of Certification–Anesthesiology (MOCA) Minute format that they have used for the past few years as their Part II platform. This format has been overwhelmingly popular with their diplomates; it essentially pushes out a couple of questions per week; their diplomates have 1 minute to answer each. After they submit their answer, they receive immediate feedback on whether they correctly answered the question and the rationale for the correct answer. The ABA has gathered useful information on the knowledge gaps among their diplomates, and they shared this information with their professional society for purposes of continuing medical education development. This activity has been so successful that they are now embarking on a pilot that has been approved by the ABMS Committee on Continuous Certification to use this same format to make summative decisions on the knowledge of their diplomates for Part III. We will be watching the outcome of this pilot closely.
The third event occurred in October 2015, when a blue ribbon task force created by the American Board of Internal Medicine released its white paper on Assessment 2020. This separate group of experts independently arrived at similar recommendations with regard to assessing medical knowledge: multiple, frequent assessments over time are much better than a single, high-stakes examination.
Armed with this information, we sent several members of our staff to the 2015 Maintenance of Certification Summit in late July, hosted by the ABA and ABMS at the ABA offices, to learn more about the ABA's experience with continuous longitudinal assessment. Thirteen other ABMS member boards were in attendance, and most were very interested in learning how this format could be used to replace their high-stakes maintenance of certification examinations. We know that several are considering moving in this direction, most notably the ABA and the American Board of Pediatrics. However, after reviewing all the information from the meeting, we do not believe this format is quite ready for prime time in Part III. The major obstacles in doing so are security, the creation of sufficient secure assessment items, and uncertainty about the psychometric methodology that will underlie summative decision making.
Accordingly, our board of directors recently approved an alternative Part II activity that uses the MOCA Minute format, which we would use to provide targeted feedback to our diplomates about their specific knowledge gaps as well as the probability that they would pass the MC-FP examination. This offering will allow diplomates to choose to complete KSAs or this alternative activity, which will be named “Continuous KSA,” to meet their MC-FP Part II requirements. We are planning to make this option available in early 2017. Over the course of the following 3 years, we hope to use data gathered from diplomates choosing this new Part II offering to assess the feasibility of using this format to replace the current Part III examination.
Discounting the Cost of MC-FP for Diplomates >70 Years Old
We mentioned previously the significant number of diplomates who continue to participate in MC-FP well into their 70s, 80s, and 90s, despite the fact that they are no longer practicing. In recognition of the dedication and commitment that these diplomates have made to our specialty, we are offering each of them a 50% discount on their MC-FP fees if they wish to continue to maintain their certification.
Transforming Clinical Practice Initiative
We would be remiss in not mentioning 2 other important issues that will demand our attention this year. The first was the recent selection of the ABFM as 1 of 39 health care collaborative networks selected to participate in the federal Transforming Clinical Practice Initiative. This initiative was designed to help physicians transform their practices to enhance care coordination and expand information-sharing. We will partner with the American Academy of Family Physicians on this effort and will receive as much as $538,000 to help offer the tools, information, and network support needed to assist physicians in improving the quality of care they provide, increase patients' access to information, and ensure more judicious use of health care dollars. Our clinical data registry will be an integral part of our plan to strengthen quality of care and develop comprehensive quality improvement strategies for those participating in these networks.
Physician Burnout
The final—but not least important—issue is the increasing rate of physician burnout. We are currently collecting data to understand how prevalent this phenomenon is among board-certified family physicians. The findings from the data that we collect will inform our decisions on how we can further enhance MC-FP to create added value and less burden for practicing family physicians, in keeping with the promises that we have made to our diplomates.
Notes
Conflict of interest: The author is president of the ABFM.
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.