The ABFM National Journal Club: Early Outcomes and Next Steps

On August 11, 2021, the American Board of Family Medicine (ABFM) launched the National Journal Club Pilot. Diplomates select an article of interest, read the article, answer 4 questions, and review brief commentaries on clinical applicability and methodology. Once all questions are answered correctly, the Diplomate receives 1 certification point and 1 continuing medical education (CME) credit from the American Academy of Family Physicians. The National Journal Club is an optional activity available at no additional cost to the Diplomate. We now report on the design, initial outcomes, and planned next steps.

The goals of the ABFM Journal Club are to:

1. Identify the most important empirical articles for family physicians based on relevance to family medicine, potential to change practice, and methodologic rigor.

2. Support family physicians in keeping up to date and in shared decision-making with their patients.

3. Empower family physicians as they engage with health system administrators, subspecialists, and payers on behalf of their patients.

4. Support deeper learning about improvements in practice.

The ABFM National Journal Club has been built on best practices from other certifying boards,¹ with a more structured search strategy, provision of the article in PDF format, greater emphasis on methodology, and a requirement for all assessment questions to be answered correctly. In early 2020, ABFM selected a National Journal Club Committee from nominations from all family medicine organizations. Committee members are practicing family physicians with expertise in evidence-based medicine and represent a diversity of geography and clinical roles. The committee and ABFM staff developed a systematic process for identifying articles from a specified time period from personal nominations and articles recommended by InfoPOEMS, Journal Watch, and the ACP Journal club. The committee ranks each article based on relevance to family medicine, impact on practice, and methodologic rigor appropriate for the clinical question—treatment, diagnosis, prevention, or causation. A blueprint was established, specifying 65% to 70% articles for continuity and outpatient care, 20% to 25% for inpatient and acute care, and 10% to 15% for obstetrics and women's health. For each article selected, the committee wrote 5 questions, including an introductory question that framed the broad clinical questions, 1 on clinical applicability with discussion of related recent literature, and 1 on a methodological feature important to clinical interpretation. To support deeper learning, Diplomates are required to answer all questions correctly. After answering a question, the Diplomate is given immediate feedback and can modify the responses.

The initial release included 44 articles from 2020. As of March 31, 9,774 physicians completed 63,273 article assessments; 3,173 completed more than 10. Overall, feedback was extremely positive. In Likert-type questions, 98.1% found the platform easy to navigate, 96.6% agreed that the introductory question piqued their interest, and 97.4% found the clinical applicability critique helpful. After reviewing 10 articles, 95.3% felt more comfortable reviewing articles and, if appropriate, putting them into practice.

Why did Diplomates participate? Figure 1 lists the most common reasons. Most had more than 1 reason, including getting certification and CME credit, keeping up to date about both problems they see commonly and those they do not, and convenience/easy access to the articles. The median participant spends between 30 and 60 minutes a week reading outside of lookups while seeing patients; Figure 2 lists the sources cited as most valuable for keeping up to date by more than 5% of participants. Unsurprisingly, family physicians have many different sources of information; the most common, but still only a fifth of participants, were the American Family Physician and UpToDate. Figure 3 lists the journals that practice-changing articles came from for 2020. The New England Journal of Medicine and JAMA journals contributed many practice-changing articles, but most journals contributed only 1 or 2 over the year. Only 1 came from a family medicine journal. The dominance of the New England Journal of Medicine in 2020 is impressive and may be related to the initial studies of the COVID pandemic. Interestingly, relatively few participants considered the New England Journal of Medicine one of the most valuable sources of information.

• Download figure
• Open in new tab

Figure 1.

Why are you participating in the ABFM National Journal Club? Abbreviations: CME, Continuing medical education.

• Download figure
• Open in new tab

Figure 2.

In the last 6 months, which of the following resources have been most valuable for keeping up to date and changing your practice? (Chart limited to responses >5%)

• Download figure
• Open in new tab

Figure 3.

Journals with practice-changing articles published in 2020.

What can we learn about the practices of the participants? We believe that the articles chosen represent a Rorschach test for what is most important for family physicians on the front line from August 11, 2021 to March 31, 2022. Figure 4 lists the articles chosen by more than a thousand physicians. Not unexpectedly, a network meta-analysis on treatment of dementia leads the pack, and 4 articles relevant to management of pain are listed. Importantly, articles related to broader scope of care—inpatient/acute care, obstetrics and children—were chosen by many physicians: not all what we are interested in is the management of chronic disease! In addition, notable is the popularity of articles further removed from typical continuity practice, such as the treatment of appendicitis with antibiotics or expectant management of pneumothoraces. This reflects the generalism of family medicine—the recognition that family physicians must constantly consider what they do not see every day. It also reflects the curiosity of practicing family physicians, which we celebrate!

• Download figure
• Open in new tab

Figure 4.

Frequency of article selection in pilot release (some titles abbreviated). Abbreviations: COPD, Chronic obstructive pulmonary disease; US PSTF, U.S. Preventive Services Task Force.

The long-term question is whether this service can speed up the incorporation of evidence into practice. We asked Diplomates whether reading the article would influence their future practice. Figure 5 provides an example of the intended practice change for individual articles. For all of the articles in the initial set, between 60% and 75% agreed with intent to change practice, and another 10% to 15% indicated that a specific article affirmed their current practice. It is important to keep in mind that these are single articles, even if highly selected and evidence framed, and that the gap between intent to change and actual change can be wide, but the feedback is very encouraging.

• Download figure
• Open in new tab

Figure 5.

Intent to change practice after review of articles 1 to 5, titles abbreviated.

Overall, our assessment is that the National Journal Club is promising, with strong early interest and excellent evaluations. In reviewing qualitative feedback, and from conversations at regional and national meetings over the last 8 months, 3 features seem to be most important: the value of curation of important articles, ease of access to the full article, and the clinical applicability commentary, along with certification and CME credit. Based on the pilot results, the National Journal Club will become a permanent part of the ABFM portfolio.

On March 31, 2022, 54 new articles were released. With this release, resident outreach is a major focus. We believe that the application of evidence to the care of individual patients should be at the heart of the residency redesign our specialty is undertaking.^2,3 ABFM is committed to providing about 100 articles per year for Diplomates to choose from. A common request is for more articles, and we are exploring this possibility further in the medium term. Internally, we are measuring and improving the rating and ranking process and exploring what we can learn about participant practices and the bibliometry of practice-changing articles. Over the long term, we will assess the value of our emphasis on “deeper learning” on practice change and explore incorporating Journal Club findings into high-stakes examinations. Most importantly, we will explore how we can better empower family physicians as they provide care and advocate for their patients.

We live in an age in which the outcomes of care are worsening across almost all diseases over the lifespan,⁴ and the US lifespan is dropping, even before COVID.⁵ ABFM believes that robust personal physicians can make a difference, healing patients and community one at a time. The National Journal Club is one way of supporting them. In this context, ABFM staff is immensely grateful to the National Journal Club Committee. These volunteers represent the best of family medicine! Thank you also to the current and future participants: your feedback will help us to improve.