Abstract
Introduction: Telemedicine has been implemented in many health systems by necessity, yet evidence is sparse about its appropriate use for the delivery of primary care. We sought to understand what clinicians and patients consider to be appropriate use of telemedicine in primary care to inform future development of a framework that should be valuable to diverse stakeholders.
Methods: We conducted in-depth, structured interviews of patients, clinicians who deliver primary care, and other select informants. They were asked to discuss optimal, acceptable, and suboptimal uses of telemedicine for delivering care relative to in-person care delivery. Audio was transcribed and paired reviewers analyzed the content to identify the key concepts that motivated the informants. The reviewers did thematic analysis to organize the concepts into unifying themes.
Results: Our 18 key informants generated 103 unique concepts. The unique concepts aggregated into themes suggesting the clinical situations in which telemedicine is appropriately used in primary care and clinical situations in which it should be avoided. We also learned of motivators toward expanded, or at least continued, use of telemedicine and motivators away from telemedicine's continued use. The informants expressed their expectations regarding decision making about telemedicine use and who should make these decisions.
Discussion: These key concepts and themes are expected to be a valuable starting point for the development of a framework to inform appropriate use of telemedicine in primary care.
The pandemic of COVID-19 provided many clinicians and patients with their first experience of telemedicine.1⇓–3 Telemedicine, specifically the synchronous clinician-patient interactions occurring over telephone or video, is being broadly implemented in many health systems out of necessity, yet its continued use may be premature in the absence of evidence about its effectiveness, acceptance, and value.4⇓–6 The widespread, continued use of telemedicine requires evaluation of its impact on primary care outcomes relative to in-person care, including on clinical outcomes, patients' satisfaction with care,7 clinician satisfaction,8 and costs.9 This method of care delivery almost certainly increases patients' access to care, yet it may increase health care utilization and spending,10⇓–12 and have varied effects on health outcomes.13,14
Telemedicine for primary care delivery is more than a conversion to a new setting of care; it is a significant departure from traditional clinical practice. It requires new technology, different work flows, altered triage processes, clinician commitment, and patient acceptance.15,16 For interventions that do not have a strong, supporting evidence base, adopters of a new practice learn for themselves how to use the innovation—they learn for which clinical conditions, for which patients, at which point in the disease trajectory its use results in best outcomes.17 Through such in situ learning, appropriate and inappropriate use is defined, which is later confirmed or refuted by evidence from trials or rigorous observational studies of outcomes.
Although the outcomes from telemedicine use will most likely be evaluated in comparison to in-person care delivery, there are alternative comparisons: relative to the absence of care or relative to care delivered in an emergency department or in an urgent care center. During the COVID-19 pandemic, care was scarcely delivered in other settings,2 and telemedicine's effectiveness might only be measured relative to the absence of care. All these comparisons generate important information; however, the priority for evidence generation depends on who needs this information to make decisions about ongoing use of this innovation. Here, we were explicitly interested in how to best use telemedicine for primary care delivery, in comparison to in-person interactions. This was among the most pressing questions for health systems and payors as new workflows were established and reimbursement policies were considered.
Given the absence of criteria for appropriate use of telemedicine, we expect there is a need for a framework for allocating patients' clinical needs into telemedicine appropriateness categories. We anticipate that this will have utility in evaluation, research, clinician and patient education, triage, and possibly for payment. In this work, we sought to understand what patients and clinicians consider to be appropriate and high-value use of telemedicine. We consider high value care to be that which achieves best patient outcomes and highest patient satisfaction for the investment.18 Our framing assumption is that, for primary care delivery, in some clinical situations, care delivered by telemedicine is of higher value than an in-person encounter. Similarly, in some clinical situations, telemedicine is of equivalent value to in-person care, and in other situations, it is inferior, meaning of lower value than in-person care. These premises are consistent with established approaches for considering the appropriateness of medical services.19
Methods
Design
This was formative, qualitative work. We conducted in-depth interviews with diverse key informants to begin to learn the appropriate and inappropriate uses of telemedicine for delivering primary care relative to in-person care delivery, using a theory-informing inductive approach.20
Informants
We considered the primary key informants to be patients and clinicians, specifically clinicians who bill independently for the delivery of primary care (physicians, nurse practitioners, and physician assistants in some states). To these groups, we also added key informants who were payors and clinical managers. We selected key informant clinicians from 3 health systems in our region and used snowball sampling to identify other clinicians practicing primary care and administrators who were expected to be valuable informants.
Interview Guide
We reviewed published literature by searching for key articles describing telemedicine implementation, appropriateness, and existing theoretical frameworks about telemedicine use.21⇓⇓⇓⇓⇓⇓⇓–29 We then developed a preliminary list of concepts exemplifying our hypothesized key determinants of appropriate use of telemedicine for primary care delivery. These concepts informed the development of an interview guide, including prompts to encourage the respondent to address these concepts in depth. We conducted pilot interviews with 2 patients and 1 clinician to assure the clarity of the interview guide and iteratively revised the guide (see the Appendix). The interview guide included both open-ended questions about the appropriateness of telemedicine, generally, and brief clinical scenarios to spark conversations to elicit rich content about telemedicine use. We provided definitions of optimal, acceptable, and suboptimal use of telemedicine to ground the conversations: “optimal” means that telemedicine use in a given situation is better than in-person visits, for the patient, “acceptable” means that telemedicine is about as good as in-person visits, and “suboptimal” means that telemedicine is not as good as in-person care.
Interviews
Two researchers, working independently, conducted 1-on-1 interviews which lasted up to 1 hour using Zoom conferencing software (Zoom Video Communication, San Jose, CA). Audio recordings were transcribed using computer-assisted technology (Sonix, San Francisco, CA).
Synthesis
We synthesized the information qualitatively. Our goal was to identify and then organize the concepts expressed by the participants. During the coding of each transcript, the reviewers identified additional concepts as they emerged with this inductive approach. Each transcript was coded in duplicate. As this process was meant to capture the breadth of concepts discussed in the interviews, we were not directed toward coming to consensus about consistent use of the codes between reviewers. The research team reviewed the concepts and came to consensus on unifying themes. We display in the results select exemplar quotes from the patients (P), clinicians (C), and other informants (O) that illustrate some of the concepts, organized by theme.
Results
We conducted 18 key informant interviews, ceasing interviews when we were not uncovering new concepts. The informants included 7 patients of diverse ages, sex, race (White, Black, and South Asian), and medical complexity; 3 family physicians caring for low income urban residents, 4 internal medicine physicians practicing primary care in hospital-based and community-based clinics, 1 medicine/pediatric trained primary care physician, 1 primary care triage nurse practicing in an urban setting, 1 administrator from a commercial insurer, and 1 outpatient internal medicine clinic administrator from a hospital-based practice.
We identified 103 unique concepts that summarized the expressions of the key informants, with the clinicians providing a greater number of unique concepts than the other informants. There were both overlapping and unique concepts expressed by the clinicians, patients, and other informants. (Table 1). The key concepts clustered into the following themes: (1) potentially appropriate use of telemedicine, (2) potentially inappropriate use of telemedicine, (3) motivators toward using telemedicine, (4) motivators away from using telemedicine, and (5) decision making about the use of telemedicine. The highest counts of unique concepts were for describing appropriate use of telemedicine and motivators toward using telemedicine for primary care delivery.
Key Concepts About Appropriateness and Motivators Expressed by Clinicians and Patients
In our thematic analysis, we found that the clinicians, patients, and other informants appeared to have largely overlapping beliefs about the appropriate use of telemedicine. Many of the respondents expressed that telemedicine is appropriate when the condition is mild and when the symptoms can be described completely (Table 2). Telemedicine was thought to be highly appropriate for chronic care management when a specimen or measurement is not needed and appropriate for information gathering, such as after hospital discharge. Telemedicine was thought not to be appropriate for severe conditions or when physical examination is necessary for a diagnosis and inappropriate when a procedure is needed, such as vaccination. Many of the respondents noted the efficiencies that are expected with telemedicine including savings of time and elimination of the costs of transportation. They also described the barriers that arise from technological challenges and the privacy concerns with use of telemedicine. Across participants, there was less consistency about who should be making decisions about the use of telemedicine. Participants described an expectation that algorithms could be developed to assist with triage or that nurses or other primary care office personnel should determine the visit type. Patients believed that selection of telemedicine for a visit either already was or should be preapproved by clinicians.
Quotes Illustrating Select Key Concepts in Each Thematic Category
Discussion
Our diverse respondents provided rich content that informs how telemedicine might be used in primary care delivery, when face-to-face care is also available. We found in our interviews that our clinician informants had remarkably consistent expectations about appropriate use of telemedicine in primary care. The differences seemed to be driven by the clinicians' degrees of tolerance of uncertainty, although the literature supports generally high diagnostic accuracy within telemedicine visits in primary care.30 The patients' expectations about telemedicine and its appropriate and inappropriate use were more varied. There seemed to be agreement that minor clinical concerns, preventative care, and scheduled assessment of chronic conditions in primary care can be done well with a telemedicine visit. However, many contextual factors affect patients' preferences about telemedicine and in-person care.
We recognize limitations to our findings. We did not probe in-depth for the respondents to separately address audio-only visits, and we recognize that these are a different experience for patients and clinicians.31 Although we recognize that there are differences in what can be accomplished during a visit with video in addition to audio 2-way communication; in practice, appointments that are scheduled as video appointments often result in telephone consultations due to technology failures. If a decision is being made at the point of scheduling or triage, the decision is between in-person care and remote care. It is a secondary consideration as to whether this will be a video or telephone consult and that is often driven by the availability of the technology and patient comfort. In addition, clinicians are rapidly gaining skills in telemedicine, such as use of digital stethoscopes and otoscopes to allow distant examination—this could not be addressed at this early stage of discussion with our informants.
This preliminary work was done with stakeholders in a single city, although interacting with different health systems and different practice sites. We expect that our observations are relevant broadly in settings where telemedicine is being implemented for primary care delivery. This present work was done with US stakeholders, and we cannot say with certainty how this translates to international settings, and those investigations would be appropriate for future research.
We expect that these insights can be a valuable starting point for the development of a framework that will inform processes to maximize the value of care delivered to patients in the context of primary care. A framework may also inform processes for generating evidence about the effectiveness, safety, and cost-effectiveness of telemedicine for primary care delivery. We are unaware of existing frameworks describing appropriate use of telemedicine in primary care, although we presume that many health systems have established their own processes to guide use of these services.
There is much work to be done to understand the best use of telemedicine in primary care and in other clinical settings. Primary care delivery in rural areas is different from in urban areas; primary care for a patient without an established relationship with a clinician is different from for a patient with such. Different systems of care delivery have different incentives which may alter how telemedicine is deployed. Additional insights may come with the input of stakeholders representing other payors, including the Centers for Medicare and Medicaid Services in the US.
We except that telemedicine can provide high value care, but this will need to be demonstrated. We look forward to developing a framework for appropriate use of telemedicine, which we anticipate may build on the concepts and themes identified in this qualitative work. This should be valuable for developing optimized processes in primary care for the continued use of telemedicine and for the evaluation of the relative value of telemedicine versus in-person care.
Acknowledgments
Kathryn McDonald, phD for helpful edits.
Appendix
Interview Guide for PATIENTS
Modified Oral Consent [EXEMPT proposal and does not require even oral consent.]
Thank you for your willingness to talk to me about telemedicine. The purpose of this research study is to begin to develop rules to guide the use of telephone or video for doing clinic visits.
You will be interviewed for about an hour and I will ask you at the end if you would like to participate in a second session that will happen a few months from now. If you get tired or bored, we can stop at any time.
As part of this research, we are making an audio recording so we can remember what we talked about. You can request that the recording be stopped at any time. If you agree to allow the recording and then change your mind, you can ask us to destroy the recording immediately. No 1 outside of our small research team will listen to the conversation and it will be destroyed in 6 months.
As we discussed, we will compensate you for your time with a $50 gift card.
Are you comfortable with this or do you have any questions about privacy?
Great – let us begin.
Increasingly, doctors and other clinicians are having telephone and video visits with patients. For the rest of this discussion, I will call this TELEMEDICINE. During the COVID pandemic, these visits needed to happen because we were not allowed to do face-to-face visits. We expect that there will continue to be some use of telemedicine. Everyone's goal is to use telephone and video visits wisely. Right now, we are not certain in which situations these visits are smart and in which situations they are not. I am part of a team working to develop rules for knowing when telemedicine should be used and when it should not.
We suspect that there are clinical situations in which telemedicine is GREAT – that is better than in-person visits; there are situations in which telemedicine is OK – about as good as in-person visits, and situations in which telemedicine is NOT AS GOOD as in-person visits. This is what we are trying to figure out.
To start – I would love your thoughts on this topic. Again, we are focusing on situations where telemedicine really works well and where it does not. There are no right answers. What are your thoughts?
[Interviewer Should Guide the Discussion to Specific Clinical Situations and Avoid Discussion about Technological Challenges
Below Are Prompts to Facilitate Discussion. The Interviewer Does Not Need to Systematically Go through These If the Participant is Supplying Information That Informs These Questions without Prompting.]
Perhaps you might ….
Tell me about situations in which telemedicine is BETTER than in-person visits with your doctor.
Tell me about situations in which telemedicine is WORSE than in-person visits with your doctor.
What if you were meeting a doctor for the first time?
What if you had a very private issue to discuss?
What if it is an emotionally hard topic? (Like the doctor has bad news for you)
When do you think a doctor must touch you or examine you?
What if your doctor wants to follow up your chronic conditions – like your blood pressure and diabetes control?
What if you were well and just needed a checkup to refill medicines and schedule cancer screening tests?
What if you were very sick like with the flu?
What if there was a frightening new problem like bad stomach pain or dizziness?
When do you prefer to stay home?
When do you prefer to come in?
Are other situations that make telemedicine work very well for you?
What is the weather is bad, or you do not have transportation, or you do not want to pay for parking? Do those affect how you feel about telemedicine?
Are other situations that make telemedicine work very badly for you?
What if you do not have somewhere private to talk, or a bad internet or phone connection?
Interview Guide for Non-Patient Participants (CLINICIANS/ADMINSTRATORS/PAYORS)
Thank you for your willingness to talk to me about telemedicine. The purpose of this research study is to begin to develop rules to guide the use of telephone or video for doing clinic visits.
You will be interviewed for about an hour and I will ask you at the end if you would like to participate in a second session that will happen a few months from now. If you get tired or bored, we can stop at any time.
As part of this research, we are making an audio recording so we can remember what we talked about. You can request that the recording be stopped at any time. If you agree to allow the recording and then change your mind, you can ask us to destroy the recording immediately. No 1 outside of our small research team will listen to the conversation and it will be destroyed in 6 months.
As we discussed, we will compensate you for your time with a $50 gift card.
Are you comfortable with this or do you have any questions about privacy?
Great, let us begin.
Increasingly, doctors and other clinicians are having telephone and video visits with patients. For the rest of this discussion, I will call this TELEMEDICINE. During the pandemic, these visits needed to happen. We expect that there will continue to be some use of telemedicine. Everyone's goal is to use telephone and video visits wisely.
I suspect that there are clinical situations in which telemedicine is OPTIMAL – that is better than in-person visits; there are situations in which telemedicine is ACCEPTABLE – about as good as in-person visits, and situations that are SUBOPTIMAL – not as good as in-person visits. I am working to develop a framework and rules for putting clinical situations into the right bucket.
To start – I would love your thoughts on this topic. Again, we are focusing on situations where telemedicine really works well and where it does not. What are your thoughts?
[Interviewer Should Guide the Discussion to Specific Clinical Situations and Avoid Discussion about Technological Challenges
Below Are Prompts to Facilitate Discussion. The Interviewer Does Not Need to Systematically Go through These If the Participant is Supplying Information That Informs These Questions without Prompting.]
Prompts to Facilitate Discussion:
Perhaps tell me…
Tell me what you think might be situations where telemedicine is BETTER than in-person visits for your patients.
Tell me what you think might be situations where telemedicine is worse than in-person visits for your patients.
Are there situations that seem risky for telemedicine? How do you define risk?
When do you need to lay hands on a patient?
I am going to suggest some specific scenarios that I would like you to talk about:
What if it is an emotionally challenging topic or you are delivering bad news?
What if the doctor and patient are meeting for the first time?
What if the patient is acutely ill with an exacerbation of a chronic condition [for example, COPD]?
What if the patient needs routine care for medication refills and scheduling cancer screenings?
What is the patient needs a Medicare Health Risk Assessment?
What if there is a need for an immediate diagnosis and action?
What is there is a diagnostic dilemma?
What if the patient is stable but with multiple chronic conditions requiring care coordination?
What if the patient is referred for preoperative evaluation?
What if the patient needs transitional care management after a hospitalization?
I am going to ask you about certain types of patients:
Are there patients who do particularly well with telemedicine and ones who do less well? (setting aside the access or technical challenges)?
Do you think that patients with less education fare better or worse with telemedicine relative to in-person visits?
Do you think that older patients fare better or worse with telemedicine relative to in-person visits?
Do you think that patients with anxiety or depression fare better or worse with telemedicine relative to in-person visits?
Do you think that patients who are active substance users fare better or worse with telemedicine relative to in person visits?
Notes
This article was externally peer reviewed.
Conflict of interest: None.
Funding: Institutional funding was provided by the Hopkins Business of Health Initiative.
To see this article online, please go to: http://jabfm.org/content/35/3/507.full.
- Received for publication June 1, 2021.
- Revision received August 5, 2021.
- Revision received October 22, 2021.
- Revision received November 2, 2021.
- Accepted for publication November 16, 2021.
