Implementing Competency Based ABFM Board Eligibility

• ACGME
• Competency-Based Education
• Entrustable Professional Activities
• Medical Education
• Specialty Boards

The transition from residency education that emphasizes counts and hours to competency assessment is a major change for Family Medicine. Starting July 1, 2023, it will affect all program directors, faculty, and residents. How should our community support this change?

Keeping in mind the “why” of residency redesign is important. Despite rhetoric of transformation and tech-driven innovation, the outcomes of health care in the US are getting steadily worse in comparison to other affluent countries¹; life expectancy is declining,² even as costs rise unsustainably. Moreover, the pandemic has driven us to rediscover³ disparities and has accelerated burnout and moral injury among family physicians and their teams. To meet the needs of our patients, communities, and health teams, Family Medicine must step up.

We in Family Medicine believe that well trained personal family physicians supported by robust teams and policy can be an antidote to the crisis in health and health care. The goal of the major revision of the ACGME Requirements for Family Medicine is to train the family physicians who can meet these needs. The new requirements represent the most significant changes since our founding and envision many changes in how we train residents, including emphasizing the practice as the curriculum, community engagement to address disparities, flexibility for residencies and residents, participation in residency learning networks, transition to competency-based medical education (CBME) and more faculty time dedicated to education and evaluation.

A first task—and one that will require engagement across the discipline over many years—is the implementation of CBME across the specialty. Of course, CBME, is not new—the WHO described it in 1978—and it has been incorporated into undergraduate medical education and widely across other health professions over the past 20 years.⁴ Now it is coming to graduate medical education, propelled by an ABMS/ACGME collaboration with leadership from Pediatrics, Surgery and Family Medicine.

The challenges of spreading CBME in Family Medicine are great. We have 745+ residencies, distributed across a vast geography, with greatly variable resources in faculty and faculty development, and many have been wounded deeply by the pandemic in terms of finances, support staff and burnout. So how to start? The ABFM believes that we should start with the “end in mind”—the core outcomes we want from family medicine residency education. We use the term “core outcomes” because ABFM research last summer showed that only approximately 40% of family medicine program directors reported they are using the term Entrustable Professional Activities (EPAs), and what they mean by EPAs varies greatly.

From December 2022 through March 2023, the ACGME Family Medicine Review Committee (FMRC) and the ABFM established the “core outcomes” of family medicine residency education, building on the EPAs developed previously in Family Medicine and with input from all the organizations of Family Medicine.⁵ The core outcomes capture the broad scope of practice we want all graduates to be able to do on graduation; they represent observable behaviors that can be improved with deliberate practice. We also believe that both milestones and core outcomes are important. Milestones focus on ACGME core competencies and allow consideration of how residents develop, whereas the core outcomes combine multiple ACGME competencies and underscore the transition to independent practice. We anticipate that all residencies will continue to track both milestones and whether each resident is ready for autonomous practice in each of the 12 core outcomes.

With this shared mental model, the Family Medicine Review Committee has begun to work with the ACGME informatics leadership to redesign the Accreditation Data System and faculty/resident surveys to get the information they will need to monitor the quality of residencies. ABFM’s focus is on competency-based Board Eligibility. Traditionally, ABFM has asked program directors to attest that a resident has finished residency and that they are “ready for autonomous practice.” Starting in June of 2024, we will ask program directors to attest both that each resident has finished their residency and is competent in each of the core outcomes, which represent specific components of readiness for autonomous practice. Our plan is to implement this requirement gradually over the 3 years: 2024, 2025, and 2026. We will not ask for submission of documentation of competence, but rather attestation of competence for individual residents for each core outcome by Program Directors, knowing that they will work with their Clinical Competence Committees (CCC), program faculty and residency administration to determine competence.

Table 1 provides a proposed 3-year schedule, which includes all the core outcomes, with an additional focus on robust continuity of care, the care of children and more specific aspects of the care of pregnant women. In considering the sequence of the 3-year schedule, we sought input from the leadership of the Association of Family Medicine Residency Directors, many current and former RC members and many current faculty and program directors. The final list is based on the 12 core outcomes but adds more detail in competencies related to continuity practice and adds focus to the care of children and the care of pregnant women. As much as possible, the language of the competencies comes from our published list. The final sequence was also informed by our community’s emphasis on supporting broad scope of practice, judgment about the ease of obtaining good assessments, the importance of enhancing the practice as the curriculum and putting core skills such as communication earlier. The ABFM plans to monitor and adjust as necessary as the community and residency practices change in response to the new requirements.

Thus, for June 2024, we will ask that program directors attest that each resident applying for ABFM Board Eligibility has completed residency and is competent in the following core outcomes of residency:

• Practice as personal physicians, providing first contact, comprehensive and continuity care, to include excellent doctor-patient relationships, excellent care of chronic disease and routine preventive care and effective panel and patient management.

• Diagnose and manage acute illness and injury for people of all ages in the emergency room or hospital.

• Provide comprehensive care of children, including diagnosis and management of the acutely ill child and routine preventive care.

• Develop effective communication and constructive relationships with patients, clinical teams, and consultants.

• Model professionalism and be trustworthy for patients, peers, and communities.

What assessments should be used for the first installments? Literature on assessment is burgeoning and is available on the ACGME Learn portal and other settings. ABFM and the FMRC look to family medicine academic organizations for guidance on best assessments as well as support for further development, testing and dissemination of new assessments. While optimizing assessment is critical, however, perhaps most important for the transition to CBME is a shared understanding of outcomes among the faculty who assess and give feedback to residents, and a robust and targeted faculty development program. To meet this need, the STFM Task Force on Assessment is developing recommendations for assessment and faculty development and also mapping the ACGME competencies to the core outcomes. We look forward to their wisdom.

Our hope is that assessment of the outcomes targeted for 2024 will be readily attainable for most residencies using existing approaches with minor changes. The first outcome is competence in practice as a personal family physician, with excellent doctor patient relationships, excellence in care of chronic disease and clinical prevention and effective practice management. We believe that many programs already measure quality of chronic disease and/or preventive care and manage efficiency and billing and coding. Some programs also address resident effectiveness in managing clinic teams and other higher order competencies. In addition, preceptors and behavioral health faculty already have opportunities to directly assess residents’ doctor patient relationships. Faculty may be able to provide end-of-precepting shift assessments such as those developed by Emergency Medicine for end of shift assessments. We hope that program directors developing their assessment strategy will also consider the aspects of continuity care we emphasize in later years, such as measured rates of continuity and appropriate referrals, cost of care, competence in common procedures, care for special populations, management of medications, and care for patients with undifferentiated symptoms or multimorbidity. We believe that the right kinds of assessments will help create the personal physicians our society needs.

How should diagnosis and management of acute illness in the hospital be assessed? Broadly speaking, this outcome addresses resident competence with assessing and managing complex and acutely ill adults in the hospital or emergency department. Hospital rotations allow assessment of many of the core ACGME competencies beyond patient care and knowledge, to include communication with patients, colleagues, and other health professionals, systems-based practice in sign outs and discharges, and problem-based learning. In designing a rotation assessment related to the core outcomes, it is also important to consider what the key elements of inpatient care include—such as assessments of efficiency and thoroughness of initial patient evaluation, ongoing inpatient management and managing discharges. Like all assessments, rotation evaluations should be anchored in specific behaviors, summarize multiple assessments, and conclude with whether the resident has met the outcomes of residency. Depending on the duration of rotations and continuity of teachers, hospital assessments should also address how well residents respond to feedback, a key component of professionalism.

The third core outcome for attestation in 2024 is comprehensive care of children, both the acutely ill and the well child. Because resources vary widely by program, the new residency requirements allow considerable flexibility in how residencies train residents in the care of children. These variations will shape the assessment strategy for any specific residency. Even so, we believe that in most cases, existing resident rotation evaluations provide a good starting point, if anchored in observed behaviors, and with emphasis on management of childhood emergencies, key procedures, and communication with patients, families and other professionals on the team.

Effective communication and constructive relationships with patients, clinical teams, and consultants are foundational to future residency education, and were therefore also prioritized for 2024. We think that this competency can be assessed in almost every rotation assessment as well as in special curricula on behavioral health. One challenge will be to develop the data systems that can allow assessments of resident communication skills to be separated by source—patients, team members, staff—and also aggregated across settings. An important lesson from Canada is that a key role of CCCs should be to ensure that assessment of competence is being performed across the continuum of care—and, if necessary, to request additional assessments as necessary.

The final important priority for 2024 is foundational to many others, and is also 1 that should be assessed across multiple settings: modeling professionalism and being trustworthy for patients, peers, and communities. As with communication skills, professionalism should be embedded in many different rotation assessments. An important first step is for faculty to consider what they think the key components of professionalism are—including confidentiality, commitment to patients, learning from feedback, trustworthiness to patients and peers—and to assess each of these explicitly.

Of course, all assessments, whether they be rotation evaluations, direct observations, multisource feedback or new assessments developed by the specialty, should be integrated into an assessment system for each residency which collates and summarizes the quantitative data and makes all comments easily available to residents, their advisors or coaches and the CCC. Most residencies already have some version of this system, as CCCs currently assess milestones to track trajectories of resident development; systems like the M3⁶ app also facilitate this process. We hope that the specialty can act together to develop the systems necessary to make the work of CBME easier. We believe that full implementation of CBME will require a significant increase in the volume of assessments, along with handheld digital tools that facilitate real time feedback. The Canadian experience has demonstrated how important this is. Ultimately, of course, portfolios will need to become more than digital file cabinets and include data analytics.

We look forward to working with the specialty to shape our future.