Skip to main content

Main menu

  • HOME
  • ARTICLES
    • Current Issue
    • Ahead of Print
    • Archives
    • Abstracts In Press
    • Special Issue Archive
    • Subject Collections
  • INFO FOR
    • Authors
    • Reviewers
    • Call For Papers
    • Subscribers
    • Advertisers
  • SUBMIT
    • Manuscript
    • Peer Review
  • ABOUT
    • The JABFM
    • The Editing Fellowship
    • Editorial Board
    • Indexing
    • Editors' Blog
  • CLASSIFIEDS
  • Other Publications
    • abfm

User menu

Search

  • Advanced search
American Board of Family Medicine
  • Other Publications
    • abfm
American Board of Family Medicine

American Board of Family Medicine

Advanced Search

  • HOME
  • ARTICLES
    • Current Issue
    • Ahead of Print
    • Archives
    • Abstracts In Press
    • Special Issue Archive
    • Subject Collections
  • INFO FOR
    • Authors
    • Reviewers
    • Call For Papers
    • Subscribers
    • Advertisers
  • SUBMIT
    • Manuscript
    • Peer Review
  • ABOUT
    • The JABFM
    • The Editing Fellowship
    • Editorial Board
    • Indexing
    • Editors' Blog
  • CLASSIFIEDS
  • JABFM on Bluesky
  • JABFM On Facebook
  • JABFM On Twitter
  • JABFM On YouTube
NewsBoard News

Longitudinal Assessment: Where We Are and Why It Is Important

Warren P. Newton, Kevin Rode, Tom O'Neill, Roger Fain and Elizabeth Baxley
The Journal of the American Board of Family Medicine May 2019, 32 (3) 448-450; DOI: https://doi.org/10.3122/jabfm.2019.03.190096
Warren P. Newton
MD, MPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kevin Rode
BA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tom O'Neill
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Roger Fain
MA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Elizabeth Baxley
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • References
  • Info & Metrics
  • PDF
Loading

On January 4, 2019, the American Board of Family Medicine (ABFM) launched the Family Medicine Certification Longitudinal Assessment (FMCLA) pilot project as an alternative to the 1-day examination. For those eligible—Diplomates in good standing, due for the examination in 2019—FMCLA provides 25 questions online every quarter, which can be completed anywhere, and at any time that is convenient. Four years will be allowed to complete a total of 300 questions. It would be possible to complete the requirement in only 3 years, but flexibility is built in to allow delaying questions, or even an entire quarter, so that personal or professional needs can take priority if necessary. Family Medicine Certification will remain active during this time as long as Diplomates are meaningfully participating in FMCLA and keeping up their regular stage requirements.

FMCLA allows access to reference materials, with up to 5 minutes to complete each question. In addition to increasing convenience and reducing the cost of assessing cognitive expertise, we believe that the format will also support Diplomates' learning. After answering a question, the correct answer is provided, along with a critique explaining why the correct answer is the best answer, and why the other choices are wrong. References are also provided for additional learning opportunities. After the first 100 questions are completed, participants will receive a preliminary performance report that includes their likelihood of passing, as well as feedback on specific gaps in their knowledge, to assist in their selection of further study or continuing medical education opportunities.

What has been the early experience from the FMCLA pilot? There is clearly great interest in the new format. As of this writing, over 90% of Diplomates choosing FMCLA indicated that convenience, the ability to monitor progress, and helping stay up to date were key issues. In addition, nearly two thirds indicated that the expense of board review courses was relevant to their choice. A total of 8418 Diplomates were eligible for the pilot, representing 9% of all Board-Certified family physicians. As of this writing, 73% of eligible Diplomates have chosen FMCLA, while 13% have applied for the April examination. The door to the FMCLA pilot closed in March. Our current intent is to offer both FMCLA and the 1-day examination for the foreseeable future, so that Diplomates can choose their preferred method. While FMCLA seems to be attractive to many, we believe that there are valid personal reasons or preferences why some may choose to take the 1-day examination.

The FMCLA IT platform seems to be performing quite well. Our staff at ABFM is monitoring performance daily and reviewing the feedback comments provided by participants every week. When suggestions for improvement or concerns are identified, these are addressed in real time. Feedback on the questions themselves has also been valuable, leading us to conclude in a few cases that we should not use a particular question for standard setting. Of over 2644 Diplomates who have completed the questions for the first quarter as of this writing, the average time spent per question has been 2 minutes and 20 seconds. Finally, although complete data are forthcoming, many have reported improved relevance and better learning. As 1 American Academy of Family Physicians Board member who is in the FMCLA pilot commented:

The biggest pro for using this format is that it is forcing me to learn something new each week. If I get something wrong, I know immediately and get the critique, which informs my ability to use the information in real life. The other pro is not having to schedule time off to take the examination. And I love the flexibility of choosing when and where to answer questions.1

Over the longer term, we will be getting additional feedback from Diplomates about how we can improve FMCLA and better support learning. We anticipate that Diplomates themselves will become more familiar with the process—an early lesson, from the American Board of Anesthesia, was from Diplomates who reported that it did not work well to answer questions in the grocery line!—and how to use references selectively in real time, ideally in a way similar to practice. It is important for Diplomates to understand the test is challenging, as befits the knowledge needed to set the standard for cognitive expertise in the specialty. Each question has been written by practicing family physicians and pretested on initial certifying physicians and Diplomates, so that the calibrated difficulty of each question is generalizable to the population of over 90,000 family physicians. Answering only about two thirds of the questions correctly is necessary to pass the standard for Board Certification.

A major part of our evaluation will be assessing how well the longitudinal test performs psychometrically to identify the cognitive expertise necessary for Board Certification compared with our standard 1-day examination. The evidence is longstanding, extensive and clear that physicians' ability to assess their own knowledge is limited.2⇓⇓–5 The value of the examination to both Diplomates and to the public is that it provides an independent assessment of cognitive expertise. The first question of our evaluation of the pilot will be how specific questions perform in an “open-book” environment. Given the large number of participants, we will know the answer by the time this article is published. In a little over 2 years, we will know how our standard setting performs at the level of both the individual and the population of test takers, as well as how stable cognitive expertise remains over time. A final major issue will be evaluating how well our emphasis on professionalism and the security steps we have taken have been able to safeguard the identity of individual participants and the integrity of the specific multiple-choice questions. We are committed to maintaining the value of the Certificate for all Diplomates.

The other major question for us is the degree to which FMCLA can improve learning. In keeping with contemporary thinking about how physicians keep up to date6 and the experience of other American Board of Medical Specialties Boards,7 we believe that assessments play a critical role in driving learning. As noted above, we are providing answers, a critique and an article after each question. After the 100th question, each Diplomate will receive an estimated scaled score, representing our best prediction of the likelihood of ultimately passing the examination. The goal is to help Diplomates know where they are—and signal whether they need to do anything differently over the next several years to pass the examination. As we do for the 1-day examination, we will also provide a summary of individual performance by organ system. In addition, we will be experimenting with different types of feedback, geared toward relevance to the Diplomate's practice and their relative confidence in the answers they provide. Evaluation at the end of the first year of the pilot will focus on optimization of helping Diplomates learn. Over the longer term, our goal is to provide a guide for self-directed learning for each Diplomate, while supporting the American Academy of Family Physicians and our other partners as they develop tools for family physicians to keep up to date.

By the end of the summer, we will be able to determine whether we can extend the pilot to those who are due to take the examination in 2020. We are also beginning to plan with other Boards about how to extend longitudinal assessment to Certificates of Added Qualifications, such Sports Medicine, Geriatrics, etc. We will explore how to extend the opportunity for interested Diplomates to begin FMCLA earlier than their 10-year required assessment. Finally, in the spring of 2021, we are scheduled to report formally to American Board of Medical Specialties about the results of the pilot. It is at that time we will share a plan for extending longitudinal assessment to all Diplomates.

Over the long term, FMCLA represents a first expression of ABFM's renewed commitment to seek a new relationship with board-certified family physicians as they serve their patients and communities. We want to develop new programs with your input and with a goal of supporting you in your demonstration of lifelong learning, cognitive expertise, professionalism, and commitment to improving practice. The coming years will see much more in addition to FMCLA.

Notes

  • Conflict of interest: The authors are employees of the American Board of Family Medicine.

  • To see this article online, please go to: http://jabfm.org/content/32/3/448.full.

References

  1. 1.↵
    1. Iroku-Malize T
    . Curious about the ABFM Pilot? Here's my experience so far. Available from: www.aafp.org/news/blogs/leadervoices/entry/20190304lv-fmcla.html. Accessed March 6, 2019.
  2. 2.↵
    1. Gordon MJ
    . A review of the validity and accuracy of self-assessments in health professions training. Acad Med. 1991;66:762–9.
    OpenUrlCrossRefPubMed
  3. 3.↵
    1. Gordon MJ
    . Self-assessment programs and their implications for health professions training. Acad Med. 1992;67:672–9.
    OpenUrlCrossRefPubMed
  4. 4.↵
    1. Davis DA,
    2. Mazmanian PE,
    3. Fordis M,
    4. Van Harrison R,
    5. Thorpe KE,
    6. Perrier L
    . Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA. 2006;296:1094–102.
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Eva KW,
    2. Regehr G
    . Self-assessment in the health professions: A reformulation and research agenda. Acad Med. 2005;80:S46–S54.
    OpenUrlCrossRefPubMed
  6. 6.↵
    1. McMahon GT
    . What do I need to learn today?—The evolution of CME. N Engl J Med. 2016;374:1403–6.
    OpenUrl
  7. 7.↵
    1. Macario A,
    2. Harman AE,
    3. Hosansky T,
    4. Post ME,
    5. Sun H,
    6. McMahon GT
    . Evolving Board Certification—Glimpses of success. N Engl J Med. 2019;380:115–8.
    OpenUrl
PreviousNext
Back to top

In this issue

The Journal of the American Board of Family     Medicine: 32 (3)
The Journal of the American Board of Family Medicine
Vol. 32, Issue 3
May-June 2019
  • Table of Contents
  • Table of Contents (PDF)
  • Cover (PDF)
  • Index by author
  • Back Matter (PDF)
  • Front Matter (PDF)
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on American Board of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Longitudinal Assessment: Where We Are and Why It Is Important
(Your Name) has sent you a message from American Board of Family Medicine
(Your Name) thought you would like to see the American Board of Family Medicine web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
5 + 0 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Longitudinal Assessment: Where We Are and Why It Is Important
Warren P. Newton, Kevin Rode, Tom O'Neill, Roger Fain, Elizabeth Baxley
The Journal of the American Board of Family Medicine May 2019, 32 (3) 448-450; DOI: 10.3122/jabfm.2019.03.190096

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Longitudinal Assessment: Where We Are and Why It Is Important
Warren P. Newton, Kevin Rode, Tom O'Neill, Roger Fain, Elizabeth Baxley
The Journal of the American Board of Family Medicine May 2019, 32 (3) 448-450; DOI: 10.3122/jabfm.2019.03.190096
Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Notes
    • References
  • References
  • Info & Metrics
  • PDF

Related Articles

  • No related articles found.
  • Google Scholar

Cited By...

  • Family Medicine Certification Longitudinal Assessment Becomes Permanent
  • Continuing Board Certification: Seeing Our Way Forward
  • Family Medicine Certification Longitudinal Assessment after One Year
  • Using the Family Medicine Certification Longitudinal Assessment to Make Summative Decisions
  • Improving Continuing Education for Family Physicians: The Role of the American Board of Family Medicine
  • Google Scholar

More in this TOC Section

  • What Assessments Are Being Used in Family Medicine Residencies?
  • ABFM Outreach: A Strategic Approach to Creating Genuine Partnerships with Family Physicians
  • Implementing Competency Based ABFM Board Eligibility
Show more Board News

Similar Articles

Navigate

  • Home
  • Current Issue
  • Past Issues

Authors & Reviewers

  • Info For Authors
  • Info For Reviewers
  • Submit A Manuscript/Review

Other Services

  • Get Email Alerts
  • Classifieds
  • Reprints and Permissions

Other Resources

  • Forms
  • Contact Us
  • ABFM News

© 2025 American Board of Family Medicine

Powered by HighWire