Knowledge Assessment Responses in the ABFM Self-Assessment Modules (SAMs)

The American Board of Medical Specialties (ABFM) introduced self-assessment modules (SAMs) in 2004 with the implementation of Maintenance of Certification for Family Physicians. The SAMs consist of a 60-item knowledge assessment (KA, including multiple choice, multiple true/false, and fill-in-the-blank formats) with references followed by a clinical simulation keyed to the KA content.¹ The KA items are organized according to competencies (eg, pharmacologic therapy, nonpharmacologic therapy, etc) defined during the SAM development process. ABFM currently offers SAMs covering asthma, care of the vulnerable elderly, cerebrovascular disease, early childhood illness, coronary artery disease, depression, diabetes, health behavior, heart failure, hypertension, maternity care, mental health in the community, pain management, preventive care, and well child care. A SAM covering hospital medicine will become available in September.

During the first few months of use, diplomates tended to spend substantial time reading and studying the associated reference material before engaging the KA items. This approach led to quite lengthy SAM sessions (eg, diplomates reported spending an average of nearly 10 hours on the hypertension KA)¹ for a number of diplomates, which led ABFM staff to recommend to participants that they take the KA “cold” (ie, without preparation) the first time through. After this “first pass,” diplomates receive feedback and critiques for the missed items, which facilitates success on subsequent attempts. To successfully complete the KA, diplomates must answer correctly 80% of the items in each competency area.

The ABFM purposefully creates the SAMs to present an in-depth and challenging exposure to the content area. The information technology platform allows the Board to capture and retain success rates on diplomates' first-pass attempts. Information technology staff have recently queried these results for all SAMs completed since 2004. The lowest average (15%) occurred with hypertension; the highest (54%) occurred with heart failure. The median for all the SAMs was 40%. The results for all the KAs appear in the Table.

The results demonstrate that the KAs do indeed represent challenging material, as intended. Interestingly, despite the hypertension module's apparent difficulty, this SAM is the second most popular offering: as of the end of August, diplomates have completed nearly 42,000 hypertension SAMs (the diabetes SAM tops the list at 44,445 modules completed).

Also, the recommended “cold” first-pass approach has anecdotally decreased substantially the time needed to complete the knowledge assessments (Gary Jackson, personal communication, August 31, 2012).

The SAMs represent in-depth coverage of their respective content areas. Over the years since introducing Maintenance of Certification for Family Physicians, the ABFM has created a fairly broad portfolio of topics but has several additional offerings in the planning stages. In particular, we plan to develop in the coming year modules related to care transitions and medical genomics. Rest assured that these additional offerings will continue the tradition of up-to-date, in-depth, and challenging coverage of the subjects!