A Stepwise Transition to Telemedicine in Response to COVID-19
=============================================================

* Sabrina L. Silver
* Meghan N. Lewis
* Christy J. W. Ledford

## Abstract

*Introduction:* With the emergence of COVID-19, many primary care offices closed their physical space to limit exposure. Despite decades of telemedicine in clinical practice, it is rare to find it used in small-metro and academic settings. Following the decision to limit face-to-face care, we tracked our practice’s transition to telemedicine.

*Methods:* This was a prospective quality improvement project following Plan-Do-Study-Act (PDSA) cycles to optimize the use of telemedicine (both telephone and video in this practice) encounters. Central to the PDSA cycles was the use of a post-encounter questionnaire to track patient, appointment, and physician factors. Throughout the cycles, inferential statistics were used to inform process improvement.

*Results:* In Cycle 2, a logistic regression model showed length of encounter, need for physical examination, and physician satisfaction correctly predicted a physician’s preferred medium (χ2(3) = 40.56, *P < .001).* In Cycle 3, a χ2 test showed the reason for visit predicted the preferred medium (χ2(4) = 47.30, *P < .001*). In cycle 4, week of telemedicine, need for physical examination, length of encounter and physician satisfaction predicted the preferred medium (χ2(9) = 172.52, *P < .001*).

*Discussion:* Using the variables that predicted preference for telemedicine, we were able to adjust our processes through PDSA cycles.

*Conclusion:* Early use of the PDSA cycle allows for informed quality improvement at the local level. Our findings highlight factors to consider when implementing telemedicine such as need for physical examination and type or length of encounter. In addition, physician satisfaction can encourage use of telemedicine, and tools for learning and practicing telemedicine should be available.

*   Chi-Square Test
*   COVID-19
*   Logistic Models
*   Personal Satisfaction
*   Physical Examination
*   Primary Health Care
*   Prospective Studies
*   Quality Improvement
*   Surveys and Questionnaires
*   Telemedicine

## Introduction

The emergence of the novel coronavirus (COVID-19) drastically changed how we provide primary care. To avoid spreading the virus, many primary care clinics shuttered their physical offices and transitioned to telemedicine to protect their most vulnerable patients and staff from exposure.1⇓⇓⇓–5

Before March 2020, some physicians embraced technology in their practices, but few utilized telemedicine to its fullest potential.6⇓–8 Telemedicine is more than a simple switch from 1 communication medium to another; telemedicine leverages communication technology to assess and address patients’ acute and chronic concerns from a distance.9⇓⇓–12 Telemedicine encompasses a variety of communication media – voice-only telephone calls, video consultations, and mobile application enabled interactions.

Despite decades of acknowledging the importance of telemedicine in clinical practice, its use is limited by physician, patient, and policy factors.12⇓⇓–15 Few physicians receive formal training in telemedicine, and there is a lack of readily available literature for learning such skills.16⇓⇓–19 In addition, obstacles such as payment and malpractice coverage exist for most clinics. Our clinic, like many others in academic or rural settings, discovered a lack of access to adequate business infrastructure and technology that provided a unique challenge in the transition to telemedicine catalyzed by COVID-19.20⇓⇓⇓⇓–25

Following a decision to limit face-to-face clinical care, we recognized a need to document our clinic’s transition to telemedicine. The purpose of this project was to identify visit characteristics that are suitable for telemedicine; focusing on patient, appointment, and physician factors. In our practice, telemedicine encapsulates clinical medicine delivered by synchronous technology. This term is synonymous with “virtual visits,” specifically referring to conducting a visit over the telephone (voice only) or using a cellular video platform such as Facetime or Google Duo (voice and face). The term telemedicine has been used throughout this article to refer to this care delivery.

## Methods

With design and statistical support from our practice-based research network, this prospective quality improvement project followed Plan-Do-Study-Act (PDSA) cycles, to optimize the use of telemedicine encounters, at a family medicine residency clinic in a northwest Florida community-based hospital. The hospital is located in a small metro county as classified by the Centers for Disease Control and Prevention.26

The PDSA process involves continuous cycles of incremental change, assessment of progress toward the objective, and reflection on lessons learned.27 The project team maintained detailed field and process notes to document decision making and changes. Central to PDSA cycles was the use of a post-encounter questionnaire. Completed by family physicians immediately following telemedicine appointments, the questionnaire documented patient factors (age, sex), appointment factors (mode of visit, reason for visit [categorized in 1 of 4 options], number of reasons, length of appointment), and physician perceptions (need for a physical examination, physician satisfaction, preferred mode for visit). At the end of the questionnaire, we asked 1 open-ended item: “What do we need to know from this encounter that can inform decisions about telemedicine appointment processes?”

The previously adapted and validated physician version of the Patient Satisfaction Questionnaire (P-PSQ) measured physician satisfaction.28 The P-PSQ consists of 5 items measuring physicians’ satisfaction in the following categories: (1) how well they addressed patient needs, (2) patient involvement in the interaction, (3) adequacy of information given by physician, (4) satisfaction with emotional support given by physician, and (5) the interaction in general. Answers were given on Visual Analog Scales ranging from 0 to 100. An overall satisfaction score was calculated by averaging responses to the questions. Internal reliability (*Cronbach’s α*) of the P-PSQ in this project was 0.94.

From March 16 to June 12, 2020, 22 physicians completed 361 questionnaires. Throughout the cycles, we ran inferential statistics to inform process improvement.

## Results

Table 1 describes PDSA cycles conducted in the first 12 weeks of COVID response. Each cycle followed a PDSA process, which was informed by the physician experience surveys and process notes of the team.

View this table:
[Table 1.](http://www.jabfm.org/content/34/Supplement/S152/T1)

Table 1. 
Plan, Do, Study, Act (PDSA) Cycle Summary

Table 2 presents the incremental, descriptive findings as they accumulated through each cycle. Across the cycles, mean physician satisfaction with telemedicine increased and physicians increasingly reported telephone as the preferred communication medium.

View this table:
[Table 2.](http://www.jabfm.org/content/34/Supplement/S152/T2)

Table 2. 
Descriptive Data as It Accumulated through Each Cycle

At the end of Cycle 2, the logistic regression test showed length of encounter, need for physical examination, and physician satisfaction predicted a physician’s preferred medium. The model was significant (χ2(3) = 40.56, *P < *.001), and correctly predicted 74.7% of cases. Physicians preferred in-person clinical visits when the encounter was longer or they perceived the need for a physical examination. A 1-way analysis of variance showed that at 14 minutes physicians preferred in-person visits. Physician satisfaction was not significant in the model.

By Cycle 3, preliminary review of open-ended feedback indicated that patient age was a likely determinant for physician preferred medium; a regression did not show this. A χ2 test detected a significant relationship between reason for visit and preferred medium (χ2(4) = 47.30, *P < *.001). When the reason for the visit was to get imaging or lab results, physicians reported a preference for telephone in 90.3% of cases.

At the end of Cycle 4, we ran a final regression to better understand the physician’s preferred medium. For this test, we limited the sample to encounters with adults and collapsed age into 2 categories: younger than 65 years or 65 years and older. Table 3 shows the results of a logistic regression analysis identifying variables that are associated with preferring in-person encounters for the visit. Four variables were significantly associated with wellbeing in the final model: week into telemedicine implementation, perception that physical examination was critical, length of encounter, and physician satisfaction (χ2(9) = 172.52, *P < *.001). Two variables – more weeks into telemedicine implementation and higher physician satisfaction – were associated with preference for telephone as a medium. Perception of the critical role of physical examination and longer telemedicine encounters were associated with a preference for in-person clinical encounters.

View this table:
[Table 3.](http://www.jabfm.org/content/34/Supplement/S152/T3)

Table 3. 
Summary of Logistic Regression Analysis for Variables Predicting Physician Preference for in-Person Visit as Compared to Telemedicine Encounter (n = 278)

To better understand why age and results as a reason for visit were not significant in the model, we tested individual relationships among model variables. An ANOVA revealed that encounters with patients 65 years and older (mean, 17.11 minutes) were significantly longer than appointments with patients younger than 65 years (mean, 14.51 minutes) with *F*(1, 314) = 3.82 and *P = *.05. Therefore, we expect length of the appointment in the model explains the age difference. Similarly, chi-square test results revealed that physician perception of not needing a physical examination was significantly associated with presenting results (χ2(1) = 22.52, *P < *.001), and with evaluating treatment (*χ*2(1) = 6.30, *P < *.05). Physician perception of needing a physical examination was significantly associated with an encounter for increased symptoms (χ2(1) = 42.17, *P < .*001).

## Discussion

Using the variables that predicted telephone medium, we adjusted clinic processes throughout the PDSA cycles. While the influencing factors are somewhat intuitive, they ultimately served as the basis of a flowchart (Figure 1) developed for nurses, appointment line, and self-booking when we started to bring face-to-face care back to our clinic. Notably, patients age greater than 65 years were preferentially booked into face-to-face appointments. Fortunately, by 12 weeks into the process, systematic changes had been implemented to limit patient exposure allowing us to balance the risk of face-to-face care with the risk of poor telephone communication.

![Figure 1.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/34/Supplement/S152/F1.medium.gif)

[Figure 1.](http://www.jabfm.org/content/34/Supplement/S152/F1)

Figure 1. 
Flowchart created for use by booking staff. This flowchart was created in Cycle 4 based on feedback from nurses to help facilitate appropriate appointment booking. The basis of the flowchart was data collected in the physician survey and analyzed weekly. Abbreviations: MSK, Musculoskeletal pain; F2F, face-to-face.

The normative acceptance and familiarity with doing telemedicine influenced preference for telemedicine. Throughout the PDSA cycles, we made changes to help normalize telemedicine as a platform for care. In Cycle 2, a telemedicine curriculum was provided to the physicians including learning points on HIPAA compliance, telephone communication skills, physical examination skills, and coding/payment. In Cycle 3, a post-telemedicine clinic huddle was implemented to discuss feedback from the day. These process improvements served to normalize telemedicine as a delivery of care model and to establish best practices. Despite these improvements, the use of video did not gain much traction in our resource-limited practice (Table 2). We suspect this is related to the ease with which a telephone can be attained for providing care compared with the challenge of tracking down the 1 clinic-owned iPhone approved for video care.

Results should be interpreted through a communication theory lens. Findings align with the principles of the medium richness model,29 which provides a theoretical framework for what types of communication tasks should occur in person in contrast to within a mediated-environment. Physicians recognize the need for multiple cues, including physical presence, voice inflection, and body gestures,30 to address some patient complaints. When physicians can only use a lean medium, like telephone, some of the information cannot be transmitted and the communication will be less effective. In addition, physicians recognized that task complexity31 is a determinant of effective telemedicine. Simple tasks such as lab results are a good fit for telephone, whereas undifferentiated concerns like joint pain are evaluated better face-to-face. Physicians also recognized not only the information-carrying capacity but the symbol-carrying capacity of the medium.32 Each channel inherently carries a symbolic message to the recipient. Although people historically associated mediated technologies as less personal,33 in times like COVID-19 pandemic, patients may perceive that reaching out to them using technology is more considerate (as in Cycle 3).

Our study has several limitations. Responses only included physicians and excluded support staff. Furthermore, the project did not collect patient-oriented outcomes. We also recognize that this project did not address other common limitations to virtual care such as payment models and malpractice coverage that must be considered in telemedicine development. Future research should look at outcomes such as patient satisfaction or frequency of follow-up, and it should include impact of established doctor-patient relationships on outcomes. More work is also needed to develop and test physician training in telemedicine and to better understand the role of specific telemedicine modalities in patient care. Specific to our site, the next PDSA cycle is continued growth of hybrid (both telemedicine and face-to-face care) half-day templates and use of the developed algorithm.

## Conclusion

Early use of the PDSA cycle allows for informed quality improvement at the local level. Factors to consider when implementing telemedicine include length of visit (linked to patient age and type of visit) and need for physical examination. In addition, physician satisfaction can encourage use of telemedicine, and tools for learning and practicing telemedicine should be available.

## Acknowledgments

We thank the staff of the Military Primary Care Research Network, including Lauren Cafferty, Hannah Ledford, and Jeremy Jackson, for their support in data management and editorial assistance.

## Notes

*   This article was externally peer reviewed.

*   *Funding:* No funding was sought or used in this collection or manuscript creation.

*   *Conflict of interest:* Dr. Ledford is Associate Editor of JABFM. It could be noted that she was not at the time this article was accepted. All other authors declare no conflict of interest.

*   *Disclaimer:* The views expressed within this publication represent those of the authors and do not reflect the official position of the US. Air Force, Uniformed Services University of the Health Sciences, the US Government, or the Department of Defense at large.

*   To see this article online, please go to: [http://jabfm.org/content/34/Supplement/S152.full](http://jabfm.org/content/34/Supplement/S152.full).

*   Received for publication July 12, 2020.
*   Revision received September 19, 2020.
*   Accepted for publication September 22, 2020.

## References

1.  1.WHO statement on cases of COVID-19 surpassing 100,000 [press release]. World Health Organization. Available at: [https://www.who.int/news/item/07-03-2020-who-statement-on-cases-of-covid-19-surpassing-100-000](https://www.who.int/news/item/07-03-2020-who-statement-on-cases-of-covid-19-surpassing-100-000). Published March 7, 2020. Accessed 17 September 2020.
    
    

2.  2.NIH clinical trial of investigational vaccine for COVID-19 begins. National Institutes of Health. [https://www.nih.gov/news-events/news-releases/nih-clinical-trial-investigational-vaccine-covid-19-begins](https://www.nih.gov/news-events/news-releases/nih-clinical-trial-investigational-vaccine-covid-19-begins). Published 2020. Updated March 16, 2020. Accessed March 26, 2020.
    
    

3.  3.Garrett L. COVID-19: the medium is the message. Lancet 2020;395:942–3.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/S0140-6736(20)30600-0&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 

4.  4.Shearn IT. COVID-19 pandemic pushes NJ to step up reliance on telemedicine. NJ Spotlight. Available at: [https://www.njspotlight.com/2020/03/covid-19-pandemic-pushes-nj-to-step-up-reliance-on-telemedicine/](https://www.njspotlight.com/2020/03/covid-19-pandemic-pushes-nj-to-step-up-reliance-on-telemedicine/). Published 2020. Accessed March 27, 2020.
    
    

5.  5.Nasca TJ. ACGME response to the coronavirus (COVID-19). Accreditation Council for Graduate Medical Education. Available at: [https://acgme.org/Newsroom/Newsroom-Details/ArticleID/10111/ACGME-Response-to-the-Coronavirus-COVID-19](https://acgme.org/Newsroom/Newsroom-Details/ArticleID/10111/ACGME-Response-to-the-Coronavirus-COVID-19). Published 2020. Accessed March 27, 2020.
    
    

6.  6.Phillips RL Jr.., Bazemore AW, DeVoe JE, et al. A family medicine health technology strategy for achieving the triple aim for US health care. Fam Med 2015;47:628–35.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=26382121&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 

7.  7.Neuhauser L, Kreps GL, Morrison K, Athanasoulis M, Kirienko N, Van Brunt D. Using design science and artificial intelligence to improve health communication: ChronologyMD case example. Patient Educ Couns 2013;92:211–7.
    
    

8.  8.Klasnja P, Pratt W. Healthcare in the pocket: mapping the space of mobile-phone health interventions. J Biomed Inform 2012;45:184–98.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/j.jbi.2011.08.017&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=21925288&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 

9.  9.Sood S, Mbarika V, Jugoo S, et al. What is telemedicine? A collection of 104 peer-reviewed perspectives and theoretical underpinnings. Telemed J E Health 2007;13:573–90.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1089/tmj.2006.0073&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=17999619&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000251086000026&link_type=ISI) 

10. 10.Kosower E, Inkelis SH, Seidel JS. Telephone T.A.L.K.: a telephone communication program. Pediatr Emerg Care 1991;7:76–9.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=2047312&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 

11. 11.Reisman AB, Brown KE. Preventing communication errors in telephone medicine. J Gen Intern Med 2005;20:959–63.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1111/j.1525-1497.2005.0199.x&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=16191150&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000232049300021&link_type=ISI) 

12. 12.Peabody MR, Dai M, Turner K, Peterson LE, Mainous AG. Prevalence and factors associated with family physicians providing e-visits. J Am Board Fam Med 2019;32:868–75.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMzIvNi84NjgiO3M6NDoiYXRvbSI7czozMDoiL2phYmZwLzM0L1N1cHBsZW1lbnQvUzE1Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

13. 13.Santos MV, Oliveira DC, Novaes Mde A. A telehealth strategy for increasing adherence in the treatment of hypertension in primary care. Telemed J E Health 2013;19:241–7.
    
    

14. 14.Russo JE, McCool RR, Davies L. VA telemedicine: an analysis of cost and time savings. Telemed J E Health 2016;22:209–15.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1089/tmj.2015.0055&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=26305666&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 

15. 15.Moore MA, Coffman M, Jetty A, Klink K, Petterson S, Bazemore A. Family physicians report considerable interest in, but limited use of, telehealth services. J Am Board Fam Med 2017;30:320–30.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMzAvMy8zMjAiO3M6NDoiYXRvbSI7czozMDoiL2phYmZwLzM0L1N1cHBsZW1lbnQvUzE1Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

16. 16.Bulik RJ, Shokar GS. Integrating telemedicine instruction into the curriculum: expanding student perspectives of the scope of clinical practice. J Telemed Telecare 2010;16:355–8.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1258/jtt.2010.090910&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=20643847&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 

17. 17.Brown SB, Eberle BJ. Use of the telephone by pediatric house staff: a technique for pediatric care not taught. J Pediatr 1974;84:117–9.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/S0022-3476(74)80570-6&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=12119932&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=A1974R958400025&link_type=ISI) 

18. 18.Lamb MP. Telephone precepting: the development of a curriculum. Teach Learn Med 2004;16:276–8.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1207/s15328015tlm1603_10&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=15388385&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000223666800010&link_type=ISI) 

19. 19.Caralis P. Teaching residents to communicate: the use of a telephone triage system in an academic ambulatory clinic. Patient Educ Couns 2010;80:351–3.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/j.pec.2010.07.028&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=20688456&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 

20. 20.Lowery CL, Bronstein JM, Benton TL, Fletcher DA. Distributing medical expertise: the evolution and impact of telemedicine in Arkansas. Health Aff (Millwood) 2014;33:235–43.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjMzLzIvMjM1IjtzOjQ6ImF0b20iO3M6MzA6Ii9qYWJmcC8zNC9TdXBwbGVtZW50L1MxNTIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

21. 21.Greenhalgh T, Koh GCH, Car J. Covid-19: a remote assessment in primary care. BMJ 2020;368:m1182.
    
    [FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE4OiIzNjgvbWFyMjRfMTUvbTExODIiO3M6NDoiYXRvbSI7czozMDoiL2phYmZwLzM0L1N1cHBsZW1lbnQvUzE1Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

22. 22.Car J, Sheikh A. Telephone consultations. BMJ 2003;326:966–9.
    
    [FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjYvNzM5Ni85NjYiO3M6NDoiYXRvbSI7czozMDoiL2phYmZwLzM0L1N1cHBsZW1lbnQvUzE1Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

23. 23.Flannery MT, Moses GA, Cykert S, et al. Telephone management training in internal medicine residencies: a national survey of program directors. Acad Med 1995;70:1138–41.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1097/00001888-199512000-00018&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=7495460&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 

24. 24.Hannis MD, Hazard RL, Rothschild M, Elnicki DM, Keyserling TC, DeVellis RF. Physician attitudes regarding telephone medicine. J Gen Intern Med 1996;11:678–83.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=9120654&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=A1996VX83900005&link_type=ISI) 

25. 25.Toon PD. Using telephones in primary care. BMJ 2002;324:1230–1.
    
    [FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMjQvNzM0OC8xMjMwIjtzOjQ6ImF0b20iO3M6MzA6Ii9qYWJmcC8zNC9TdXBwbGVtZW50L1MxNTIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

26. 26.Ingram DD, Franco SJ. 2013 NCHS urban-rural classification scheme for counties. Vital Health Stat 2 2014;1–73.
    
    

27. 27.Leis JA, Shojania KG. A primer on PDSA: executing plan–do–study–act cycles in practice, not just in name. BMJ Qual Saf 2017;26:572–7.
    
    [FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI2LzcvNTcyIjtzOjQ6ImF0b20iO3M6MzA6Ii9qYWJmcC8zNC9TdXBwbGVtZW50L1MxNTIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

28. 28.Zandbelt LC, Smets EM, Oort FJ, Godfried MH, de Haes HC. Satisfaction with the outpatient encounter: a comparison of patients' and physicians' views. J Gen Intern Med 2004;19:1088–95.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1111/j.1525-1497.2004.30420.x&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=15566437&link_type=MED&atom=%2Fjabfp%2F34%2FSupplement%2FS152.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000224864900003&link_type=ISI) 

29. 29.Daft RL, Lengel RH. Organizational information requirements, media richness, and structural design. Manag Sci 1986;32:554–70.
    
    

30. 30.Daft RL, Lengel RH, Trevino LK. Message equivocality, media selection, and manager performance: implications for information systems. Manag Inf Syst Q 1987;11:355–66.
    
    

31. 31.Sheer VC, Chen L. Improving media richness theory. Manag Comm Q 2004;18:76–93.
    
    

32. 32.Sitkin S, Sutcliffe KM, Barrios-Choplin JR. A dual-capacity model of communication media choice in organizations. Human Commun Res 1992;18:563–98.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1111/j.1468-2958.1992.tb00572.x&link_type=DOI) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=A1992HY39000003&link_type=ISI) 

33. 33.Johnson MA. Public relations and technology: Practitioner perspectives. J Public Relat Res 1997;9:213–36.