We continue to receive a number of inquiries from physicians regarding the low pass rate statistics for recertification candidates for the reporting periods from 2010 to 2012. Many physicians fear the examination has changed in some way, putting those attempting to maintain their certification at a distinct disadvantage. Others worry that the passing score criteria for recertifiers are greater than those of initial certifiers. We appreciate the active interest in ongoing assessment results and welcome the opportunity to clarify the source of these statistical changes. In short, the Maintenance of Certification for Family Physicians (MC-FP) examination has not changed. For those attempting to maintain their certificates, the criteria for a passing score are no more stringent now than in recent years. The apparent pass rate discrepancy for this group in 2010, 2011, and 2012 is a sample bias among recertification candidates created by a policy change that extended certification from 7 to 10 years. Here we attempt to explain this statistical artifact in simple terms and address the concerns expressed by many family physicians.
Certification Extension Policy Change
In January 2006, the American Board of Family Medicine announced an enhancement to the MC-FP process that went into effect in January 2007. The most important enhancement to the process was the opportunity to extend one's certification by 3 years, resulting in a 10-year certification period. Many physicians who would previously have taken the examination during their seventh year successfully met the new MC-FP requirements and did not sit for the MC-FP examination as a result of earning the 3-year extension of their certificate. This option resulted in a much smaller volume of MC-FP (recertification) candidates testing during the 3-year window from 2010 to 2012.
Recent Pass/Fail Statistics for Initial Certifiers and Recertifiers
Examination pass rate statistics are available on the American Board of Family Medicine website.1 We would like to draw attention to the summer statistics for the years 2008 through 2012. In 2008 and 2009, pass rates for initial certifiers were 82.9% and 86.7% and for recertification candidates were 86.8% and 85.7%, respectively. Readers will notice that, in 2010, 2011, and 2012, initial certification pass rates remain quite comparable: 84.0%, 84.2%, and 83.2%, respectively. However, it is with the recertification candidates that the pass rate statistics become misleading. In 2010, recertification candidates passed at a rate of 67.0%, with pass rates of 66.3% and 73.9% in 2011 and 2012, respectively. These values are considerably below recent historical trends. However, when one notes the volume of candidates sitting for the examination during these years, the rest of the story begins to unfold.
The volume of candidates for initial certification has remained quite stable since 2008, with as few as 2236 and as many as 2804 examinees. The volume of recertification candidates, on the other hand, experienced a steep decline. In 2008 and 2009, the volume of recertification candidates ranged between 8356 and 8453. In 2010, this volume dipped to only 1397 examinees, thus resulting in 6959 fewer recertifiers than the previous year. The trend continued in 2011 and 2012: only 1209 and 1395 recertification candidates, respectively, sat for the examination. The certification extension reduced the volume of candidates testing during the transition period. Many candidates who did test during this period were repeating the examination because of a previous failure. A larger proportion of previously unsuccessful candidates combined with an overall smaller number of candidates testing created a dramatically lower passing rate percentage statistic.
Table 1 provides a breakdown of pass rates and mean scaled scores for both initial certifiers and recertifiers based on whether it was examinees' first attempt or a repeat attempt at the MC-FP examination. Results indicate that candidates who successfully pass their first attempt tend to continue to perform well. Candidates repeating the examination because of a previous failure tend to perform less well on subsequent examinations. Significantly fewer recertification candidates sat for the examination during the transition period in which most diplomates earned 3-year extensions on their certificates. A significant proportion of these candidates were attempting because of previous failures. Therefore, pass rate percentage statistics reported during this transition period are not truly comparable to previously reported statistics. In particular, recertification candidates testing during the 3-year extension period are not representative of recertification candidates in normal, high-volume years.
Conclusions
On the surface, pass rate statistics for MC-FP candidates may appear alarming for recent years because they are significantly lower than those from previous years. These candidates will be happy to know that this statistic is misleading; a sample bias has occurred in the current recertification candidate statistics, stemming from a policy change that extended certification from 7 to 10 years. This news should be reassuring to physicians who are approaching recertification and beginning to study and prepare for the November 2013 MC-FP examination.
To further emphasize these points, we have just finished preliminary analysis of the April 2013 examination. The volume of recertification candidates once again resembles the figures from 2008 and 2009 cited earlier, and the characteristics of the sample no longer contain the sample bias of having a significant proportion of recertification candidates testing as a result of a previous failure. Accordingly, the pass rates for candidates taking the examination to maintain their certificate (recertifiers) are similar to the pass rates for this cohort in 2009. These results were expected and should further reassure family physicians that our examinations behave as predicted, since the examinations are standardized and the same passing standard is in effect for the 2013 examination as was the case in 2009.
Notes
Conflict of interest: The authors are from the ABFM.
Reference
- 1.↵