Article Figures & Data
Figures
- Figure 1.
Flowchart created for use by booking staff. This flowchart was created in Cycle 4 based on feedback from nurses to help facilitate appropriate appointment booking. The basis of the flowchart was data collected in the physician survey and analyzed weekly. Abbreviations: MSK, Musculoskeletal pain; F2F, face-to-face.
Tables
Plan (For Delivery of Care) Do Study (Adjustments to Data Collection and Results from Data) Act Pre-Cycle: COVID started and clinic responded by shutting down clinic, physicians forced to adapt Mar 16 to 20 COVID-19 concerns arise, need for decrease in face-to-face appointments to reduce patient and physician exposures, cleaning room times, and maintaining availability for patient care Canceling all face-to-face appointments and transitioning to telemedicine and ambulatory clinic Transition normal clinic operations to telemedicine clinic staffed with 4 physicians per day Mar 23 to 27 4 physicians per day. Normal operations fully shifted to only respiratory or telemedicine care Started to discuss need for process improvement Defined need for ability to gather data to assess feedback and areas of improvement from physicians regarding appropriateness of telemedicine appointments and process of mass-scheduling telemedicine care Cycle 1: Started utilizing telemedicine care + data gathering about new clinic process and expanding telemedicine capabilities Mar 30- Apr 3 4 physicians per day with 20 minutes appointments Created physicians survey (version 1) for use with telemedicine encounters. Incorporation of residents into telemedicine care to expand operations Started survey collection Need additional methods to provide full-scope telemedicine care including video capabilities. Small group of physicians self-trained on video telemedicine programs and options. Apr 6 to 10 2 physicians per day with 20 minutes appointments Developed ways to promote video telemedicine (Facetime, Google Duo with clinic iPhone). Researched methods for physician education about telemedicine care Added video as an option to survey. Free text response question about telemedicine delivery experience Need additional training of physicians administering telemedicine care Cycle 2: Telemedicine education + need for inclusion of chronic care Apr 13 to 17 2 physicians per day with 20 minutes appointments Offered video telemedicine as method of providing care. Created telemedicine curriculum for resident education Began to more closely track free text response of reason for visit on survey Based on free test responses of reason for visit, a needs assessment of missing aspects of care showed a need to perform chronic care and annual wellness visits. Began reaching out to patients with chronic diseases. Apr 20 to 24 2 physicians per day with 20 minutes appointments Telemedicine curriculum disseminated to all residents Defined 5 categories in which to characterize reason for encounter on survey. Noted appointment length influenced recommended type of care Need for assistance with scheduling appointments more appropriately (ie chronic disease needs longer appointment; lab call back, annual workplace physicals can be shorter). Need assistance with patient intake questions to improve efficiency Cycle 3: Implementing chronic disease + need for efficiency Apr 27-May1 2 physicians per day with adjusted appointment template* Created 30 minutes appts for future appointments (chronic disease management), continued 20 minutes appointments for acute complaints, and 10 minutes appts for annual workplace physical and result call-backs, based on data from surveys Added free text question for curriculum feedback to survey With increasing number of telemedicine appointments per day, need assistance with efficiency of encounters May 4 to 8 3 to 4 physicians per day with adjusted appointment template* Added targeted chronic disease care and provided documentation templates. Paired technician† with physician to help with 'intake' information to run encounters more efficiently Free text responses often requested communication and physical exam skills Provided feedback directly from curriculum on communication and telemedicine physical exam skills May 11 to 15 3 to 4 physicians per day with adjusted appointment template* Attempted to collect patient medical history via patient portal platform to allow techs to update patient intake Patient portal platform was exceedingly difficult to use and never took off, some free text responses showed appreciation for tech doing intake Need for coordinated communication between technician†, admin staff, and physician. Encouraged daily 'huddles' between technician† and doc Cycle 4: Increasing teamwork and communication for well-run telemedicine clinic May 18 to 22 3 to 4 physicians per day with adjusted appointment template* Implemented telemedicine care huddles at end of day for all techs and physicians Continued to have free text feedback on appropriateness of appointments Need to communicate with nurses as they are the ones booking appointments in the acute COVID transition phase. Invited nurses to daily huddle. May 25 to 29 3 to 4 clinicians per day with adjusted appointment template* Nurses attend huddle Results began to show factors that made an appropriate telemedicine appointment (results only, age <65, physical exam not needed) Need increased F2F clinic appointments Jun 1 to 5 3 to 4 physicians doing telemedicine care per day with adjusted appointment template*. 2 to 4 physicians F2F encounters Increased F2F care with 20 minutes appointments. Results continue to show same factors that made an appropriate telemedicine appointment (results only, age <65, physical exam not needed) Need for better integration of F2F and telemedicine. Need to have flowsheet for the registered nurse, appointment line, and Tricare online self-book for which patients to book F2F vs telemedicine June 8 to 12 3 to 4 physicians per day doing telemedicine care with adjusted appointment template*. 2 physicians per afternoon doing F2F. 1 physician per morning doing hybrid (telemedicine and F2F) Created half-day clinics templated for telemedicine care and F2F care. Flowsheet for RNs and appointment line Survey collection concluded Need to continue development of hybrid clinic Abbreviation: F2F, face-to-face appointment.
↵*Adjusted appointment templates = 10 minutes for annual workplace physical and results call back; 20 minutes for acute complaints; 30 minutes for future and chronic disease management.
↵†Technicians = certified medical assistants who perform patient intake. In the telemedicine setting, they reviewed medical history, medications, and appropriate screenings as they would in a F2F encounter.
Cycle Cumulative Data Points through Cycle Mode of Visit N (%) Mean Number of Reasons for Visit ± SD Mean Length of Visit*± SD Physician Perception of Need for Physical ExamN (%) Mean Physician Satisfaction ± SD Physician Recommended Ideal Visit Type N (%) 1 n = 49 Telephone 46 (93.9%) 1.14 ± 0.41 14.35 ± 6.65 Yes 20 (40.8%) 76.78 ± 21.25 Telephone 18 (36.7%) Video 3 (6.1%) In person 31 (63.3%) Video 0 2 n = 135 Telephone 124 (93.9%) 1.36 ± 0.72 14.39 ± 7.07 Yes 59 (44%) 78.94 ± 18.65 Telephone 55 (40.7%) Video 8 (6.1%) In person 75 (55.6%) Video 1 (0.7%) 3 n = 229 Telephone 212 (95.9%) 1.33 ± 0.69 14.69 ± 7.15 Yes 91 (40.3%) 79.55 ± 17.58 Telephone 97 (42.4%) Video 9 (4.1%) In person 107 (46.7%) Video 12 (5.2%) 4 n = 361 Telephone 329 (91.1%) 1.32 ± 0.69 14.88 ± 8.69 Yes 146 (40.9%) 80.07 ± 16.84 Telephone 164 (45.4%) Video 15 (4.2%) In person 159 (44.0%) Video 16 (4.4%) Abbreviation: SD, standard deviation.
*Length of visit in minutes.
- Table 3.
Summary of Logistic Regression Analysis for Variables Predicting Physician Preference for in-Person Visit as Compared to Telemedicine Encounter (n = 278)
Step 1 Step 2 Step 3 Step 4 β (95% CI) β (95% CI) β (95% CI) β (95% CI) Week of telemedicine implementation 0.88*** (0.81, 0.96) 0.84** (0.75, 0.94) 0.84** (0.75, 0.94) 0.85** (0.75, 0.95) Patient age ≥ 65 1.43 (0.71, 2.89) 0.59 (0.23, 1.51) 0.48 (0.18, 1.26) 0.46 (0.17, 1.26) Type of visit: -Routine visit type 1.41 (0.30, 6.57) 0.98 (0.14, 6.82) 0.83 (0.11, 6.21) 0.76 (0.10, 5.63) - Type of visit: Evaluate new symptom visit 1.63 (0.38, 6.93) 1.63 (0.28, 9.59) 1.55 (0.25, 9.49) 1.55 (0.25, 9.49) - Type of visit: Increase of symptoms visit 5.21* (1.26, 21.57) 2.03 (0.35, 11.81) 1.98 (0.33, 11.90) 2.04 (0.34, 12.33) - Type of visit: Results visit 0.47 (0.10, 2.27) 0.68 (0.11, 4.41) 0.75 (0.11, 5.01) 0.72 (0.11, 4.84) Physical exam critical to encounter 34.69*** (15.19, 79.22) 31.78*** (13.80, 73.20) 23.87*** (10.17, 56.03) Length of encounter 1.07* (1.01, 1.13) 1.07* (1.01, 1.13) Physician satisfaction 0.97** (0.94, 0.99) Note: R2 = 0.23 for Step 1, ΔR2 = 0.35 for Step 2 (P < .001), ΔR2 = 0.02 for Step 3 (P < .05), ΔR2 = 0.02 for Step 4 (P < .01). *P < .05, **P < .01, ***P < .001.
CI, confidence interval.
