Practice Adjustments Made by Family Physicians During the COVID-19 Pandemic
===========================================================================

* David W. Price
* Aimee R. Eden
* Elizabeth G. Baxley
* Ann Williamson
* Warren P. Newton

## Abstract

*Introduction:* COVID-19 pandemic-related health care disruptions necessitated rapid adaptation among family physicians to safely meet patient needs while protecting themselves and their staff. On April 1, 2020, the American Board of Family Medicine (ABFM) introduced a COVID Performance Improvement (PI) activity for physicians to report on and receive Family Medicine certification credit for practice adjustments they made during the early stages of the pandemic. We aimed to understand the types of interventions implemented, and lessons physicians learned from the efforts.

*Methods:* We analyzed data from COVID-PI activities submitted by self-selected family physicians between April 1 and June 30, 2020. We summarized the COVID-related topics chosen for improvement and performed a qualitative content analysis on a random sample of open-text responses about lessons learned.

*Results:* The most common practice changes among 1259 unique COVID-PI activity submissions related to virtualization of patient visits, implementing new workflows, developing screening protocols, and obtaining and preserving personal protective equipment. We identified 12 themes regarding lessons learned, most commonly regarding patient and staff safety, modified practice processes and workflows, positive perceptions of and future plans for virtual visits, access to care, and patient satisfaction. Most submitters noted early successes with their interventions.

*Conclusion:* A PI activity template designed for continuous board certification allowed family physicians to report on how they successfully implemented short term practice changes during the early stages of the COVID-19 pandemic. Reflections from this subset of physicians regarding lessons learned may prove useful in informing future COVID-19 related practice changes.

*   Certification
*   COVID-19
*   Family Medicine
*   Family Physicians
*   Health Services Accessibility
*   Pandemics
*   Workflow

## Introduction

The COVID-19 pandemic has caused massive disruptions in health care delivery, including at the level of the primary care practice. In the early phases, face-to-face office visits were dramatically decreased or suspended as care became focused on emergency, inpatient, and critical care services in response to dramatic increases in hospitalizations. Ambulatory care visits declined 50–60% from February through April 2020;1,2 as of August 2020, in-person visits remained 10% below prepandemic levels.2 These significant decreases in visit volumes resulted in an increased level of unpaid work; furloughs and layoff of staff; early retirements; and both real and threatened practice closures.3,4 These changes have also been associated with increased levels of burnout, depression, anxiety, and stress among family physicians.5 These impacts are likely to produce long-lasting effects on the primary care workforce, the delivery of preventive, acute, and chronic disease care provided by family physicians, thus potentially impacting the health of the population.6,7,8

In the United States, as well as many other countries, primary care practices rapidly increased the amount of care delivered remotely by telephone, e-mail, and videoconferencing.29,10,11,12 Practices also needed to develop and implement new workflows for screening and triage of patients, including those potentially infected with COVID-19, while implementing safety precautions for protecting physicians, staff, and other patients.13,14,15,16 Family physicians working in the urgent care, emergency department, hospital, and long-term care settings were also forced to respond in unique ways.17

Assessment of performance in practice has been part of the American Board of Family Medicine (ABFM) recertification program since the board's inception in 1969. Article-based practice audits were replaced by online topic-based performance improvement (PI) activities in 2003. In an effort to provide a pathway for physicians to report on the improvement efforts they are already doing in practice, ABFM introduced a template for data-driven self-directed PI activity in 2014. The self-directed option provides additional opportunities for physicians across practice types, locations, and scopes of practice to earn certification credit for improvement work relevant to their needs and the needs of their patients. A streamlined self-directed activity template has been online since 2018.

In the spring of 2020, several ABFM diplomates began using the self-directed template to submit COVID-19 related practice changes to meet their PI activity requirement. In response, ABFM developed a modified version of its self-directed activity template to help more family physicians capture their learnings and earn continuing certification credit for rapid and ongoing COVID-19 related practice adjustments. Rather than focusing on a single change such as tele-visits, the COVID-PI template provided flexibility to help diplomates identify and test many types of practice changes that could evolve throughout and beyond the pandemic. Although baseline measurement was encouraged, it was not required for testing of interventions that were never or seldom used before the pandemic. Thus, the template facilitated quicker implementation and streamlined the reporting of COVID-related interventions. The process recognizes the challenges family physicians faced during this crisis; provides value in the form of certification credit while minimizing reporting burden; and helps ABFM identify and curate learnings and successful practices as a resource to help other physicians who desired to make similar practice changes. The COVID-PI activity was launched on April 1, 2020 and was rapidly adopted by a large number of diplomates. In this article, we report findings from a sample of submissions from self-selected diplomates through June 30, 2020, aiming to understand the types of interventions implemented and lessons physicians learned from the efforts.

## Methods

The ABFM COVID-19 self-directed clinical pilot activity template asked physicians to identify the need(s) for adapting their practice to the pandemic; select improvement topic(s) from a checklist; identify practice gaps, measures, and aim statements for their improvement; describe their intervention; submit follow-up and baseline data when available; and describe lessons learned from the effort.

Data from the online templates were compiled into a Microsoft Excel spreadsheet for analysis. Duplicate submissions were removed. Submissions were cross-referenced with other ABFM data to assess certain demographic and practice characteristics of diplomates who submitted performance improvement activities using the COVID-PI template from April 1 through June 30, 2020.

We compiled frequencies of the checklist improvement topics selected by these diplomates. We used a team-based inductive approach to thematic analysis18 in an exploratory analysis to characterize free text comments physicians made regarding lessons learned from completing the PI activity. Using Microsoft Excel (Microsoft Corporation, Redmond, Washington), 2 researchers (DP and AE) first read through a random sample of 250 comments to identify key words and initial themes. Four authors (DP, AE, EB, and AW) then met to refine and finalize the codes and their definitions, and to ensure that the codes could be accurately applied to the comments by all team members. An additional random sample was extracted, and a total of 633 comments (half the total sample) were analyzed. Each comment was read and coded by 1 team member, categorized into at least 1 code. Open-text comments that contained multiple themes were assigned more than 1 code. A second coder checked a random sample of comments to ensure inter-rater agreement. The team determined that thematic saturation was achieved after reviewing this half of the sample. Theme frequencies were calculated to provide a sense of the most commonly cited lessons learned.

Diplomate demographic information was determined from the ABFM physician portfolio. Practice characteristics were gathered from a survey completed by diplomates before ABFM examinations; due to different survey versions, data were only available for diplomates whose most recent examination occurred after 2017. Ethical approval was granted by the American Academy of Family Physicians Institutional Review Board.

## Results

Between April 1 and June 30, 2020, a total of 2054 COVID-PI activity submissions were submitted. After removing 267 duplicate single diplomate submissions and 528 duplicate group activity submissions, 1259 submissions were included in the analytic pool. Submitting diplomates were more likely to be female compared with nonsubmitting ABFM diplomates. diplomate years of experience, faculty status, primary practice site, practice size, specialty mix, and practice ownership were similar between submitting diplomates compared with the general population of ABFM diplomates. (Table 1).

View this table:
[Table 1.](http://www.jabfm.org/content/35/2/274/T1)

Table 1. 
Characteristics of Diplomates Submitting COVID-PI Activities, April 1–June 30, 2020 (n = 1259)

The most common COVID-PI activity chosen for practice change (Figure 1) related to virtualization of patient visits (n = 935), followed by adaptations in practice workflow (n = 583). Other commonly identified areas included COVID-19 screening (n = 452) and obtaining and maintain supplies of personal protective equipment (n = 418). Areas selected less frequently by diplomates included hand hygiene (n = 262), COVID-19 diagnosis (n = 183) and treatment (n = 159), and home visits (n = 35). 641 of submissions (50.9%) included 2 or more topic areas. Virtual visits was the topic most often chosen alone (67% of 618 single topics). Furthermore, 44.3% of the 943 virtual visit submissions did not include other topics.

![Figure 1.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/35/2/274/F1.medium.gif)

[Figure 1.](http://www.jabfm.org/content/35/2/274/F1)

Figure 1. 
COVID-19 Performance Improvement Topic Areas From Diplomates Submitting COVID-PI Activities, April 1–June 30, 2020 (n = 1259). Abbreviation: PPE, personal protective equipment.

We identified 12 cross-cutting themes about lessons learned, regardless of PI activity topic selected. While challenges and concerns were noted, most reflections were positive about the initial practice adaptations. Some comments were specific to a unique theme. Comments that contained elements of more than 1 theme (for example, safety and workflows or virtualization and access to care) were counted in each applicable category. Thus, the total frequency of themes exceeds the total number of comments. Below, we present comments illustrating 4 of commonly identified themes: patient/staff/personal safety, intent to continue virtual visits, continuing to adapt workflows, and patient satisfaction.

Patient and staff/personal safety was the most common theme found in diplomate reflections on lessons learned from participating in the COVID-PI activity. Submissions included keeping patients safe at home (virtualization), safe in the office, and self and staff protection (PPE, workflow changes, and hand hygiene). One diplomate wrote that “[t]hese changes significantly improved our ability to deliver safe and effective [care] but [*sic*] limiting exposure from staff and patients who may be likely to spread COVID-19 to our most vulnerable patient population.”

Many diplomates noted intention to continue to offer virtual visits to patients after the pandemic, as exemplified by the following reflection: “We hope to keep this going forward for patients with transportation issues, who cannot take off work or who need a family member present.”

As stressful as these initial change processes were, there was evidence of the resilience of family physicians on the frontlines of care. The necessity to continue process and workflow adaptations after initial changes were made was also a commonly cited lesson learned: We have had ups and downs determining what appointments/concerns are best suited for remote visits–some have needed to be changed from a scheduled Telehealth visit to an in-person visit–remotely as well as patients resisting our requests to have them seen in person for more severe illnesses. We are learning!

Indeed, patient satisfaction with implemented practice changes, especially virtualization, was another common theme, and demonstrated patient willingness to adapt to pandemic safety: Patients felt they were able to reach their providers and felt more comfortable knowing they were not stranded or having to jeopardize their own health to reach us. I learned to roll with changes and that patients are willing to adapt as needed.

In addition, the uncertainty and rapidity with which large scale changes were required was felt to have brought physicians, their staff, and in some cases their employer or affiliated organization, closer together as a team and resulted in enhanced communication: We learned that during a pandemic we must slow down. We must work as a team to look at the details of our process in order to keep everyone involved safe. Small changes made an immense difference in our team's safety.

Table 2 presents the 12 emerging themes in descending order of frequency, with definitions and illustrative quotes representing each theme.

View this table:
[Table 2.](http://www.jabfm.org/content/35/2/274/T2)

Table 2. 
COVID-19 Performance Improvement Lessons Learned by diplomates Submitting COVID-PI Activities, April 1–June 30, 2020

## Discussion

### Practice Implications

The brisk response to ABFM's COVID-19 self-directed clinical pilot activity opportunity potentially indicates its perceived value and relevance to diplomates during the early, confusing stages of the pandemic. Results demonstrated the creativity and flexibility of family physicians in implementing practice changes to meet their needs of their patients.

The practice improvement topics, adjustments, and reflections we identified in this cross sectional analysis are consistent with COVID-19 adaptations noted in several smaller studies.12⇓⇓⇓–16,19 Respondents in this study were generally successful in transitioning to telehealth. They found telehealth useful for maintaining patient access (at least in the short term) while attending to patient and staff safety. They indicated changes in office workflows and screening procedures to incorporate telehealth in their practice and mitigate COVID-19 risks from in-person visits to other patients, their staff, and themselves. Participants also noted the importance of team communication in making rapid adaptations to change.

The majority of reflections indicated that diplomates achieved some early successes with their interventions, although we did not specifically analyze the degree of improvement. While the majority of submissions commented on high patient and staff satisfaction with changes made in practice, several diplomates expressed anxiety about the impact of these changes on the longer term health of their patients or practice. Several participants commented on practice or patient challenges in implementing and adapting to new technology. Others expressed concerns about challenges in obtaining vital signs or conducting physical examinations, highlighting possibilities for future physician, staff, or patient education as tele-visits evolve. Only 12 of the 1259 submissions noted concerns about stress or burnout. Prepandemic concerns about physician burnout and reported increases during the pandemic5 are likely a reflection of the self-selected nature of respondents and their focus on early response during the initial phase of the pandemic.

### Implications for Board Certification

In recent years, the value of including performance improvement as a part of Board Certification has been questioned by some.21,22 The COVID-PI activity has been one of the most popular PI activities in ABFM history; its rapid, broad, and continuing uptake demonstrate the power of being relevant to what family physicians are facing in practice. Our pilot demonstrates how board certification activities can be aligned with physician needs to address contemporary, critical health care issues. The activity's flexibility—allowing diplomates to choose their target—has been very important. The breadth of subject and ideas has been impressive, as family physicians applied the module to very different communities and aspects of care. Target options and flexibility are also emphasized in ABFM's self-guided and health equity PI activities. This flexibility, which supports diplomate's intrinsic motivation to improve areas they have prioritized, has important implications for the design of future PI activities. In addition, lowering burden by not requiring an audit before starting telehealth seemed important, even though most diplomates did. Some structure is necessary for learning and improvement and to avoid “check-boxism”—but just enough. These lessons may represent a foundational principle for the next generation of PI work by the ABMS boards community: adapating principles of quality improvement by increasing flexibility and reducing administrative burden can facilitate relevant and timely participation and meaningful learning from improvement activities by large numbers of physicians across different practice settings.

### Limitations

We did not use intentional representative sampling in our study. Our sample consisted of family physicians participating in continuous certification who self-selected and voluntarily completed and submitted a PI Activity to ABFM. However, the size, national reach, and variety of different family medicine practices in our sample suggests that our findings may be applicable across a variety of primary care practices. Due to participation bias, our results should not be interpreted as generalizable to all family physicians, especially since many practices, physicians, and staff are known to have struggled in adapt during early stages of the pandemic.3⇓⇓⇓⇓–8 Rather, they are best viewed as examples of positive deviance20 which can provide a road map for other practices considering undertaking these types of changes.

### Future Directions

Virtual visits may continue to serve as an important means of providing patient access to primary care services in small or large practices. We believe the learnings from responding diplomates, particularly learnings around flexibility and team communication in the face of practice change, may be relevant in the post-acute phase of the pandemic, and can inform potential future, sustainable practice improvements.

At the time of this writing (September 2021), ABFM has received over 15,000 submissions reporting on practice changes related to COVID-19, including return to practice with a hybrid of in person and remote care, as well as vaccine promotion and administration. We intend to analyze these later submissions to describe how initial changes evolved over time and attempt to compare these practice changes with quality measures from registry data. Analysis of later submissions will also allow us to identify other new adaptations as the pandemic evolved, potentially including family physicians' role in purveying accurate scientific information to patients and communities, combatting COVID-19 misinformation, providing COVID-19 vaccinations and addressing vaccine hesitancy, and ongoing education about the role of nonpharmacologic means (masking, physical distancing) of COVID-19 mitigation. It will also be important to monitor and explore ongoing challenges (particularly stress and burnout), or the emergence or of new barriers to future practice adaptations in later stages of the pandemic. Analysis of practices who were unable to quickly adapt to the pandemic may also be instructive.

The COVID-PI activity form was structured to collect information about the focus (topics) of attempted interventions, and high-level reflections on diplomate learning. Given the importance of minimizing reporting burden during a highly stressful time, we did not ask respondents to extensively comment about the specifics of how they implemented changes or overcame barriers. Further elucidation of the specifics of how diplomates implemented their adaptations and overcame barriers to change could help others translate the learnings into practice.

## Conclusion

The ABFM COVID-19 self-directed clinical pilot activity provided a mechanism for family physicians to reflect on short-term practice changes they implemented during the early stages of the COVID-19 pandemic. It also provided a means for ABFM diplomates to earn continuous certification credit for the relevant, immediate process changes they were making to provide safe, continuous, compassionate care to their patients under extreme circumstances.

Diplomate reflections may prove useful in informing future COVID-19 related practice changes. Preliminary learnings may inform the post-COVID-19 role of virtual care. Early improvements in team communication and function, workflows, and, and safety practices should be built on and sustained. We believe lessons learned from diplomate submissions can apply to small and medium sized independent practices and larger health system networks. They can also inform learning collaboratives for physicians to receive feedback and more widely share successful practices.23 Hopefully, these efforts will lead to continued improved and satisfying care delivery long after the COVID-19 pandemic has passed.

## Acknowledgments

The authors thank Lars Peterson, MD, MPH, Matt Wilhoite, and Ketu Vagdiya for their help accessing participant demographic data.

## Notes

*   This article was externally peer reviewed.

*   *Funding*: This article was not funded by any organization, community, or group.

*   *Conflict of interest*: The authors are employees of the ABFM. Dr. Newton is the president of ABFM.

*   To see this article online, please go to: [http://jabfm.org/content/35/2/274.full](http://jabfm.org/content/35/2/274.full).

*   Received for publication September 27, 2021.
*   Revision received November 8, 2021.
*   Accepted for publication November 10, 2021.

## References

1.  1.Alexander GC, Tajanlangit M, Heyward J, Mansour O, Qato DM, Stafford RS. Use and content of primary care office-based vs telemedicine care visits during the COVID-19 pandemic in the US. JAMA Netw Open 2020;3:e2021476.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2020.21476&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fjabfp%2F35%2F2%2F274.atom) 

2.  2.The impact of the COVID-19 pandemic on outpatient visits: Changing patterns of care in the newest COVID-10 hot spots [Internet]. Commonwealth Fund; 2020 [cited 2021 May 21]. Available at: [https://www.commonwealthfund.org/publications/2020/aug/impact-covid-19-pandemic-outpatient-visits-changing-patterns-care-newest](https://www.commonwealthfund.org/publications/2020/aug/impact-covid-19-pandemic-outpatient-visits-changing-patterns-care-newest).
    
    

3.  3.Gausvik C, Jabbarpour Y. COVID-19 timeline: Centers for Medicare and Medicaid services (CMS) changes and primary care support were not enough to prevent practice losses. J Am Board Fam Med 2021;34:S7–S9.
    
    [FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6MTY6IjM0L1N1cHBsZW1lbnQvUzciO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzM1LzIvMjc0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

4.  4.Filippi MK, Callen E, Wade A, et al. COVID-19's financial impact on primary care clinicians and practices. J Am Board Fam Med 2021;24:189–497.
    
    

5.  5.Ofei-Dodoo S, Loo-Gorss C, Kellerman R. Burnout, depression, anxiety, and stress among family physicians in Kansas responding to the COVID-19 pandemic. J Am Board Fam Med 2021;34:522–30.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMzQvMy81MjIiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzM1LzIvMjc0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

6.  6.Primary care and COVID-19: Week 21 survey [Internet]. Primary Care Collaborative; 2020 [cited 2021 May 21]. Available at: [https://www.pcpcc.org/2020/10/01/primary-care-covid-19-week-21-survey](https://www.pcpcc.org/2020/10/01/primary-care-covid-19-week-21-survey).
    
    

7.  7.Phillips RL, Bazemore A, Baum A. The COVID-19 tsunami: the tide goes out before it comes in [Internet]. Health Affairs Blog; 2020 [cited 2020 October 13]. Available at: [https://www.healthaffairs.org/do/10.1377/hblog20200415.293535/full/](https://www.healthaffairs.org/do/10.1377/hblog20200415.293535/full/).
    
    

8.  8.Barnett ML, Mehrotra A, Landon BE. Covid-19 and the upcoming financial crisis in health care. NEJM Catalyst 2020;CAT.20.0153. [https://catalyst.nejm.org/doi/full/10.1056/CAT.20.0153](https://catalyst.nejm.org/doi/full/10.1056/CAT.20.0153).
    
    

9.  9.Huston P, Campbell J, Russell G, et al. COVID-19 and primary care in six countries. Br J Gen Pract Open 2020;4:bjgpopen20X101128.
    
    

10. 10.The impact of the COVID-19 pandemic on outpatient visits: practices are adapting to the new normal. Commonwealth Fund; 2020 [cited 2021 May 21]. Available at: [https://www.commonwealthfund.org/publications/2020/jun/impact-covid-19-pandemic-outpatient-visits-practices-adapting-new-normal](https://www.commonwealthfund.org/publications/2020/jun/impact-covid-19-pandemic-outpatient-visits-practices-adapting-new-normal).
    
    

11. 11.Jetty A, Jabbarpour Y, Westfall M, Kamerow DB, Petterson S, Westfall JM. Capacity of primary care to deliver telehealth in the United States. J Am Board Fam Med 2021;34:S48–54.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6MTc6IjM0L1N1cHBsZW1lbnQvUzQ4IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8zNS8yLzI3NC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

12. 12.Eight ways practices have changed workflow during COVID-19 [Internet]. University of Colorado Anschutz Medical Campus; 2020 [cited 2021 May 21]. Available at: [https://medschool.cuanschutz.edu/docs/librariesprovider32/snocap-covid-19/practice-flow-stories\_snocap-covid-surveys-4-6.pdf?sfvrsn=a926d2b9\_2](https://medschool.cuanschutz.edu/docs/librariesprovider32/snocap-covid-19/practice-flow-stories_snocap-covid-surveys-4-6.pdf?sfvrsn=a926d2b9_2).
    
    

13. 13.Careyva BA, Greenberg G, Kruklitis R, Shaak K, Stoeckle JJ, Stephens J. Key factors promoting rapid implementation of virtual screening modalities for the COVID-19 pandemic response. J Am Board Fam Med 2021;34:S55–S60.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6MTc6IjM0L1N1cHBsZW1lbnQvUzU1IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8zNS8yLzI3NC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

14. 14.Gomez T, Anaya YB, Shih K, Tarn DM. A qualitative study of primary care physicians' experiences with telemedicine during COVID-19. J Am Board Fam Med 2021;34:S61–S70.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6MTc6IjM0L1N1cHBsZW1lbnQvUzYxIjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8zNS8yLzI3NC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

15. 15.Knierim K, Palmer C, Kramer ES, et al. Lessons learned during COVID-19 that can move telehealth in primary care forward. J Am Board Fam Med 2021;34:S196–S202.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6MTg6IjM0L1N1cHBsZW1lbnQvUzE5NiI7czo0OiJhdG9tIjtzOjIwOiIvamFiZnAvMzUvMi8yNzQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

16. 16.Highlighting the success and challenges of practices during COVID-19 [Internet]. University of Colorado Anschutz Medical Campus; 2020 [cited 2021 May 21]. Available at: [https://medschool.cuanschutz.edu/docs/librariesprovider32/snocap-covid-19/success-and-challenges-stories\_snocap-covid-surveys-4-6.pdf?sfvrsn=e424d2b9\_2](https://medschool.cuanschutz.edu/docs/librariesprovider32/snocap-covid-19/success-and-challenges-stories_snocap-covid-surveys-4-6.pdf?sfvrsn=e424d2b9_2).
    
    

17. 17.Healthcare facilities: managing operations during the COVID-19 pandemic [Internet]. Centers for Disease Control and Prevention; 2021 [cited 2021 May 21]. Available at: [https://www.cdc.gov/coronavirus/2019-ncov/hcp/guidance-hcf.html](https://www.cdc.gov/coronavirus/2019-ncov/hcp/guidance-hcf.html).
    
    

18. 18.1.  Guest G, 
    2.  MacQueen KM
    
    McLellan-Lemal KM, MacQueen E. Team-based codebook development: structure, process, and agreement. In: Guest G, MacQueen KM eds. Handbook for Team-Based Qualitative Research. Latham/Altimira; 2008;119–135.
    
    

19. 19.Silver SL, Lewis ML, Ledford CJW. A stepwise transition to telemedicine in response to COVID-19. J Am Board Fam Med 2021;34:S152–S161.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6MTg6IjM0L1N1cHBsZW1lbnQvUzE1MiI7czo0OiJhdG9tIjtzOjIwOiIvamFiZnAvMzUvMi8yNzQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

20. 20.Bradley EH, Curry LA, Ramanadhan S, et al. Research in action: using positive deviance to improve quality of health care. Implement Sci 2009;4:25.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1186/1748-5908-4-25&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=19426507&link_type=MED&atom=%2Fjabfp%2F35%2F2%2F274.atom) 

21. 21.MOC requirements modified for internal medicine physicians. AMA PRA credit system. 2015. Available at: [https://www.ama-assn.org/education/ama-pra-credit-system/moc-requirements-modified-internal-medicine-physicians](https://www.ama-assn.org/education/ama-pra-credit-system/moc-requirements-modified-internal-medicine-physicians). Accessed September 26, 2021.
    
    

22. 22.Cooney TG, Moyer DV, Abraham GM, Chick D. American College of Physicians Response to ABMS Draft Standards for Continuing Certification - Call for Comments. (Title of letter). June 22, 2021. Available at: [https://www.acponline.org/acp\_policy/letters/acp\_comment\_letter\_abms\_draft\_standards\_june\_22\_2021.pdf](https://www.acponline.org/acp\_policy/letters/acp\_comment\_letter\_abms\_draft\_standards_june_22_2021.pdf). Accessed September 26, 2021.
    
    

23. 23.Price DW, Campbell CM. Rapid retooling, acquiring new skills, and competencies in the pandemic era: Implications and expectations for physician continuing professional development. J Contin Educ Health Prof 2020;40:74–5.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1097/CEH.0000000000000297&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=32433321&link_type=MED&atom=%2Fjabfp%2F35%2F2%2F274.atom)