Listening to the Diplomates: Physicians’ Feedback on Self-Assessment Modules

doi:10.3122/jabfm.18.6.574

The Self-Assessment Module (SAM) is an integral part of the American Board of Family Medicine’s Maintenance of Certification for Family Physicians (MC-FP) process as well as a significant learning opportunity. Contrary to what some may think, the SAMs are not a test, but rather a self-evaluation and self-teaching experience. A SAM consists of two parts: a knowledge assessment and a clinical simulation. Once a Diplomate completes the knowledge assessment portion for the first time, the missed questions present themselves on second and subsequent tries with both references and critiques, making the SAM much more of a learning process than any type of examination. Whenever a physician completes a SAM, we solicit feedback regarding their experience to continually improve these modules. The results of this feedback for our first four modules reflect a strong positive response. In the areas of (1) relevance of information to clinical practice, (2) currency of information, (3) usefulness of information, and (4) overall value of the module, the average ratings fall at 5.4 on a 6-point scale.

Perhaps the most important feedback indicator is whether the information relayed in these educational modules effects positive outcomes by improving practice. In the case of the first two health topics offered in 2004, for which we have at least one year’s record of physician evaluation, 54% of those taking the diabetes module indicated that they would change their practice as a result of participating. Of those persons, 90% volunteered written comments describing what changes they would make. For the hypertension module, these values were within 1% of the diabetes module (nearly identical), providing a strong suggestion that both are high-quality educational materials.

These results bear out the goals established in the development of the SAMs as the primary facet of part II of MC-FP, Self-Assessment and Lifelong Learning. Namely, to enhance family physicians’ knowledge and skills in areas that are of greatest importance to them and to provide continuous opportunity for improvement in the quality of care they deliver to the American public.

Any Comments?

Physicians are also offered the opportunity to make an open response comment at the end of the SAM evaluation. The American Board of Family Medicine has used this feedback—both positive and negative—to modify and update the SAM over the first 18 months to improve both user comfort and the functionality of the SAM itself. Examples of physicians’ comments regarding the relevance of SAM as an educational tool are as follows:

I think that this is an excellent tool for assessing and attaining knowledge in a non-biased format. Excellent update on clinical guidelines, as well as a review on the important basic science aspects of diabetes. I was very impressed with the whole set.
The knowledge assessment portion was quite decent and appropriate. Clearly an educational activity.
I think improvements have already been made since I took the SAM on diabetes last year. I really liked the way the reference came up with the question in the review instead of having to shift from page to page. That reinforces learning in a friendly way.
It was excellent… the immediate critique of missed answers was most helpful.Figure