Abstract
Objectives: We have improved access to specialist care and decreased wait times in our region through the development and implementation of the Champlain BASE (Building Access to Specialists through eConsultation) service. This secure, web-based tool allows primary care providers (PCPs) quick access to specialist advice for their patients and often helps to avoid the need for a face-to-face referral. Our successful implementation of eConsult in our region provides a unique opportunity to examine PCPs' satisfaction and overall perspective on using the service.
Methods: Following the closure of each case, PCPs completed a short survey with multiple-choice and open-ended questions regarding the eConsult. All eConsults submitted between April 15, 2011, and December 31, 2013, were analyzed. We calculated satisfaction scores from the survey and conducted a constant-comparison thematic analysis on those cases where the PCP elected to leave a text response.
Results: We analyzed 2,052 eConsults completed during the study period. In 91% and 93% of eConsults, PCPs reported a high value for their patients and themselves, respectively. In 554 eConsults, PCPs elected to leave a written response. Three major themes emerged: PCP appreciation of the eConsult service, perceived benefits for the quality of patient care, and attitudes towards using a new health technology. High satisfaction was expressed with quick response times, helpfulness of responses, and reassurance reported. Most PCPs felt eConsult had a positive impact on patient care by also providing reassurance to patients, reducing burden of time and travel, and offering educational opportunities to PCPs applicable to future cases.
Conclusion: PCPs showed a high level of satisfaction with eConsult's quick turnaround time and quality of specialist advice. Our results illustrate the advantages of using asynchronous virtual platforms to increase access to specialty care from a PCP perspective.
Excessive wait times for accessing specialist care is one of the most significant problems in health services delivery both in Canada and beyond.1⇓⇓⇓–5 The impact of waiting for access to specialist care is significant for patients, with lengthy delays affecting daily activities, increasing anxiety and pain, and often leading to a general deterioration in overall health.3 Furthermore, delayed access to specialist care can result in delays in diagnosis, duplication of services, and dissatisfaction among providers.2
There is an opportunity to improve access to specialist care through the use of innovative e-health platforms such as electronic consultation (eConsult). eConsult links primary care providers (PCPs)—a group that includes family physicians (MDs) and nurse practitioners (NPs)—and specialists through electronic means and enables specialist advice to be given directly, often without the need for a face-to-face visit.4,6,7
We developed and deployed an eConsult service in our health region in 2010; this service is known as the Champlain Building Access to Specialists through eConsultation (BASE) eConsult service. The project was launched to address issues with excessive wait times in our region, which, depending on the specialty, exceeded 9 months for nonurgent appointments. The system was developed, implemented, and evaluated in several stages: proof of concept,4 pilot phase,8 and, more recently, broad-scale implementation with ongoing collection of utilization and outcomes data.
There is a need to understand the perspectives of the users of new technological approaches, especially in the case of eConsultation, where adoption and utilization are entirely dependent on the physician. The adoption of new technology for health care providers can be challenging, and factors such as the relative advantage and observable benefits, complexity, and compatibility must be considered when implementing new systems.9 In addition, the engagement and buy-in of the provider is critical, especially when they are being asked to participate directly, as in the case of eConsults.10 PCPs who see the innovation as having a relative advantage over current practice will more readily commit to and champion the project.10
There is very little published in this area because eConsult systems are new, and although some initial work looking at user satisfaction with the systems has been done,11 there is no research that gives an in-depth PCP perspective. Because our system has been in operation since 2010 and we collect routine feedback from PCPs, this presents a unique opportunity to examine the perspectives of PCPs who currently use the service. Here we present a mixed-methods study of the results of a satisfaction survey and an in-depth look at the qualitative comments from PCPs. Specialists' perspectives will be presented separately.
As innovative new technologies continue to be introduced into health care, studies such as this, which provide the unique perspectives of the health care providers who must adopt and use new systems, are critical. Our results will help inform others who are considering implementing similar systems.
Methods
The eConsult Service
The eConsult service has been reported previously.4,8 In brief, the Champlain BASE eConsult service is an asynchronous, web-based system that allows PCPs to submit a patient-specific question to a specialist. PCPs log onto the system and complete a 4-field electronic form with the option to upload any pertinent files that may help the specialist (eg, laboratory results, images, data from electronic medical records). PCPs can select from 48 different specialty groups. Specialists respond to questions within 1 week. Communication between PCPs and specialists is iterative, allowing for both parties to request additional information/clarification from one another as needed. The PCP ultimately decides when to close the case, at which point a permanent record of the discussion is created that may be downloaded into the patient's electronic health record. A standard computer with Internet access is the only requirement. Registration, including an orientation session, takes ≤30 minutes.
At the conclusion of each eConsult, PCPs must complete a mandatory 4-question closeout survey that includes optional free-text fields (Figure 1). The first 2 questions are multiple choice and inquire about the outcome of the eConsult for the patient and whether a face-to-face visit is required. Questions 3 and 4 assess the overall value of the eConsult to the patient and provider, respectively, and are answered on a 5-point Likert scale (5 = excellent, 1 = minimal). A final, optional free-text question asks for any additional comments the PCP would like to provide.
Closeout survey administered upon the completion of each eConsult.
Study Design and Setting
This was a mixed-methods study of PCPs' responses to the closeout survey administered at the completion of each eConsult. The setting is a large health region in eastern Ontario, Canada.
Participants
Participants included any PCP registered to use the eConsult service between April 15, 2011, and December 31, 2013, who had completed at least 1 eConsult.
Data and Analysis
We tabulated the responses from questions 3 and 4 of the PCP closeout survey to assess the perceived benefit of the service for patients and PCPs. We considered Likert scores of 4 or 5 as high value. We extracted the written comments from those eConsults when the PCP elected to write a response to questions 1, 2, or 5 of the closeout survey. These data then were exported into NVivo 10 for analysis.
We conducted a thematic analysis of PCPs' text responses using a constant comparison approach.12,13 Two members of the research team (PD and JJ) independently reviewed all the responses and broadly coded them, working without preconceived themes.14 Researchers were blinded to the identity of the respondents. Team members met regularly to compare themes and group them into codes. Emerging themes and subthemes were systematically refined, tested, and revised, and new codes were developed accordingly. The thematic analysis was facilitated by discussions with the research team to ensure concordance and identify disconfirming data. The entire analysis team (CL, PD, JJ, and EK) reviewed all the comments to reduce the chance of bias. The analysis team met on 4 separate occasions over the course of 6 months to review the analysis and to gain a consensus on the emerging themes and subthemes.
Results
A total of 2052 eConsults directed to 27 different specialty groups were completed during the study period. There were a total of 559 written comments reported from 137 different PCPs, of whom 78.1% were family physicians and 73.7% were women (Table 1).
Five comments were excluded because of duplication or lack of relevance. We therefore analyzed qualitative data from 554 comments directed to 25 different specialty groups (Figure 2 and Table 2).
Flow of electronic consults (eConsults) completed between April 15, 2011, and December 31, 2013, through qualitative analysis.
Perceived Value to Patients and PCPs
The results from the closeout survey demonstrated that PCPs highly valued the service both for their patients and themselves. In 91% and 93% of eConsults, PCPs reported high value of the service to their patients and themselves, respectively (Figure 3).
Chart of primary care physician (PCP) responses to questions 3 and 4 of close-out survey, completed between April 15, 2011, and December 31, 2013, enquiring about the perceived value of the electronic consult (eConsult) service for their patients and for themselves. Questions were answered on a 5-point Likert scale (5 = excellent, 1 = minimal).
Thematic Analysis
Through the analysis of the written comments from PCPs, we identified 3 overarching themes: PCP appreciation of the eConsult service, perceived benefits to the quality of patient care, and attitudes toward using a new health technology (Table 3).
Theme 1: PCP Appreciation of the eConsult Service
Satisfaction.
Nearly all PCPs reported satisfaction with and appreciation for the service. PCPs appreciated the quality of specialists' responses, how helpful they were, and how much their patients valued having access to the service. Users regularly expressed gratitude toward the specialists who answered their questions as well as to the service itself, and they stated how appreciative they were of the eConsult system's ability to have a direct effect on patient care.
“Many thanks to endocrinology—for this elderly frail patient I appreciated not having to wait for her to see an endocrinologist. Very clear guidance.” (ID 346, MD, female)
Similarly, PCPs often remarked on how the quality of the response helped guide the management of their patient.
“Very thorough reply! I have a lot of patients [who] come back with this result and I am never sure what to do with them. Now I have a clear course of action when I get similar results in the future. Thanks!” (ID 426, MD, female)
“An extremely helpful consultation! Plotted out the entire course of action I can follow over the long run including [a] safety hatch for me!” (ID 295, MD, female)
In a few instances, however, PCPs were not satisfied with the depth of the answer that was provided. Some providers were looking for more detail, whereas others felt their questions were not adequately addressed: “An example of what anticonvulsant to start her on would have [made] this consult more useful.” (ID 204, MD, female)
Response Time.
In addition to noting the ease with which users can access specialist advice, many providers noted how appreciative they were for the timely response. The average specialist response time (from the time a case was submitted by the PCP to when the specialist first responded) during the study period was 18 hours, and many PCPs expressed how helpful it was to receive a quick answer.
“Fantastic to be able to get great advice within hours of sending the consult. This is so helpful for me in my rural practice. Great service!!” (ID 322, MD, male)
“I have had several excellent quick responses from radiology through eConsult, that have saved me a lot of time trying to reach [a radiologist,] which usually takes a lot of time.” (ID 27, MD, female)
Reassurance to the PCP.
In many cases, PCPs reported that the eConsult service reassured them when dealing with a challenging or unfamiliar case. The advice they received often served to limit doubt, provide support in unfamiliar situations, or help confirm a diagnosis.
“[This patient] has had the issue for almost a year—I was unsure [of the] diagnosis, and without [a] consult, would have tried the wrong course of action for several weeks before sending to Derm for advice.” (ID 199, MD, female)
Cost Savings.
A few PCPs reported the value of the eConsult service for the health care system as a whole, noting that providing prompt access to specialists saves the system money.
“That is what an eConsult service should be[:] to help reduce health care services load by a simple and sound advice.” (ID 397, MD, female)
“Please continue with e-consult services as it will save on health [dollars] in the long run and will assist in improvement of patient care.” (ID 293, MD, female)
Theme 2: Perceived Benefits for the Quality of Patient Care
Most PCPs felt that eConsult had a positive impact on the care they provided to patients. Specifically, PCPs cited educational benefits, improved access to specialist knowledge, and an increased ability to reassure patients in their treatments and reduce the burden these treatments placed on their daily lives.
Educational Benefits of Using eConsult
Many PCPs viewed the eConsult service as a learning opportunity. By engaging in iterative conversations with specialists in which problems are presented and diagnosed, PCPs felt they gained valuable knowledge in new medical disciplines, discovered specialities or diagnoses they were previously unfamiliar with, and refreshed their knowledge of areas they had not worked in for some time.
“Thank you very much for your detailed and very helpful response. It is great learning for me and I have shared it with a couple of my colleagues as well! I am glad that I will now be able to recommend against testing and treating the children with more evidence behind me and I will feel much more confident standing up for my now-more-educated opinion.” (ID 402, MD, female)
PCPs felt this new knowledge often translated to better care because they could now manage their patient's care in a more confident and informed manner. Furthermore, some PCPs noted that the knowledge they gained while using eConsult for specific cases could be applied to their practice more generally by guiding their management of all subsequent patients presenting similar conditions.
“Thank you to Dr. X for the excellent advice. This will also help me manage patients with similar profiles in the future.” (ID 329, MD, male)
Face-to-Face Referral Still Needed
Even when a face-to-face referral was initiated, PCPs appreciated the opportunity to provide further workup for their patient or give the consulting specialist more information before the patient even stepped into their office. PCPs noted that this initial exchange of information can reduce the urgency of referrals, lead to more effective visits, and help avoid wasting health system resources and patients' time.
“I will be able to provide more information to the consultant, for a more effective visit.” (ID 143, MD, female)
Rerouting Referrals
In some cases specialists responded to PCPs by recommending they approach a different speciality, which the specialist thought could provide more relevant answers. PCPs often found this clarification helpful because it ensured that their patients were directed to the appropriate resources.
“It is good to be told the most useful people to refer to, specifically. Thank you.” (ID 316, MD, female)
Access to Specialist Knowledge
A few PCPs expressed appreciation for the eConsult service's ability to improve access to specialist knowledge. PCPs noted that eConsult was particularly helpful for rural physicians and allowed NPs to submit a referral without having to get an MD to cosign their referral form. In addition, a small number of PCPs felt the service offered a useful alternative to the traditional “hallway consult,” in which physicians and specialists would meet informally to discuss specific cases.
Reassurance for the Patient
A few PCPs reported instances where the eConsult service reassured patients by providing them with prompt access to a second opinion. By verifying the PCP's diagnosis, eConsults reduced patients' anxiety regarding their diagnoses or treatment plans.
“This was very helpful to me and allowed me to reassure my patient that our plan of care is acceptable.” (ID 280, NP, female)
Reduced Patients' Burden
PCPs recognized that specialist appointments can place a significant burden on some patients, especially those who have mobility issues or live far from urban centers where specialist clinics tend to operate. Costs associated with attending specialist appointments can include child care, transportation, parking, meals, and lost wages. By using eConsult to avoid unnecessary referrals, PCPs were able to spare their patients these expenses.
“This likely saved my [patient] an extra trip to Ottawa, and paying for a driver.” (ID 316, MD, female)
Theme 3: Attitudes Toward Using a New Health Technology
Adoption
Some PCPs described their personal experience with the service platform when submitting an eConsult. These comments touched on how easy the platform was to use, experiences filling out the necessary forms or attaching documents, and requests for additional functionality within the system. Comments relating to technical aspects of the service were largely positive; PCPs appreciated the ability to upload relevant laboratory tests, images, or videos.
“Great program, very user friendly!!” (ID 113, NP, male)
“It was easier than I expected and very fast response. Thank you.” (ID 173, MD, female)
Challenges
While most comments addressed eConsult's ease of use, a small number of PCPs experienced technical issues when submitting eConsults or uploading relevant photographs or documents. These challenges often involved technical problems or misunderstandings of what file types were supported by the system.
“When I added a follow up question it never seems to go through and the consult disappeared. I had to request a new consult with my follow up question.” (ID 115, MD, male)
“Unfortunately [the] specialist was not able to open attachments; would be helpful to have clear guidelines as to which file types are acceptable. Thanks!” (ID 225, MD, female)
However, such instances were infrequent and reflect constraints typically seen when using eConsult type systems.
Request Additional Specialities
A few PCPs felt their questions were more applicable to a specialty or subspecialty not available in our list of options and requested that these groups be added to the service. Since the completion of our data collection, the majority of these specialties have joined the service.
Discussion
The introduction of new technology in health care can be challenging and completely ineffective if the systems are not adopted and used by the providers on an ongoing basis. Our study of PCPs' perspectives on the Champlain BASE eConsult service provides a unique view of not only the high levels of satisfaction but also the reasons for them, which are revealed through our qualitative study of the free-text comments about our large, multispecialty eConsult service. Our PCPs overwhelmingly reported high levels of satisfaction with the service, noting that the prompt responses from specialists addressed the common problem of long wait times for speciality care. Few negative comments were reported, and those that were often arose during initial implementation with the limited number of specialities available or had to do with technological challenges.
Earlier articles of single specialty eConsultation systems also have reported high levels of satisfaction with smaller samples of users (n = 19–53): McFarland et al15 reported 71% of PCPs were satisfied with the overall service, Whited et al16 noted 92% satisfaction, and 63% of providers in the study by Weinstock et al17 rated the system as excellent or good. Two studies evaluated the eReferral system at the San Francisco General Hospital and found high satisfaction ratings.18,19
Our results provide an in-depth explanatory look at these satisfaction ratings. The overarching themes of PCP appreciation of the eConsult service, perceived benefits to the quality of patient care, and attitudes toward using a new health technology illustrate the breadth of impact the service is having in our health region. Through using the service, PCPs report feeling more confident and reassured when treating their patients while appreciating the educational aspect of the eConsult service that, over the long-term, will guide the future management of their patients. In turn, the service is aiding in building capacity over the long term to increase the quality of care. In accordance with our findings, Straus et al18 noted the educational benefit PCPs received in using the service.
In addition, PCPs who use the service are helping to improve communication between themselves and specialist physicians. The traditional referral process is marred with breakdowns in communication between providers, leading to overall dissatisfaction with the referral process.20 The iterative nature of the discussion within eConsult gives PCPs and specialists the opportunity to clarify or ask follow-up questions, or provide additional information, thereby helping to build a better relationship between providers.
Our results are consistent with the key findings from a systematic review by Greenhalgh et al9 that evaluated the spread and diffusion of innovation in health service delivery. Our eConsult service offers a relative advantage over the traditional referral process and is easily compatible with PCPs' values and norms, a requirement for the successful adoption of a new technology.9,10
The limited disaffirming comments within our study were centered around PCPs voicing their concern over not having access to other specialty services, shortcomings in the detail of the response from the specialist, and minor challenges using the system. Studies evaluating similar services noted that using an eConsult service may or may not increase the workload of PCPs in terms of ordering new tests.11,18 Our analysis did not reveal any dissent regarding the increased workload for PCPs, although we are not able to rule this out completely because we did not explicitly ask this question.
Our study has several strengths. It is the first to report the scope of the impact of a multispeciality eConsult service and illustrates that the benefits extend beyond avoidance of face-to-face specialist referrals. Second, because the survey is administered immediately at the conclusion of each eConsult rather than months later, it is less susceptible to recall bias. To our knowledge, it is the first study to systematically analyze open-ended comments from a closeout survey, giving rich insight into the perspectives of PCPs right after they complete each eConsult. This approach yields specific feedback from each individual eConsult rather than sampling PCPs at a later date.
Limitations
Our study has some limitations. Despite analyzing comments from 137 different PCPs, our sample represents 68.9% of the total number of PCPs who submitted an eConsult, which may lead to selection bias. In addition, in only 27% of eConsults did the PCP elect to leave a written response, which could potentially further bias our results. Reports of patient satisfaction are not drawn from patients directly, but rather from PCPs' perceptions of patient satisfaction.
Though analyzing responses from open-ended questions yields rich information, it does not allow for an opportunity to follow up with participants and clarify responses. Future work should investigate the perspectives of PCPs further, potentially through in-depth or semistructured interviews. In addition, work is required to better understand the perspectives of the specialist physicians who currently use the eConsult service.
Conclusion
We reported the results of our highly successful eConsult service from the perspectives of its active PCP users. Our results illustrate the advantages of using secure, web-based platforms for asynchronous communication to increase access to specialty care and will help PCPs, health care leaders, policy makers, and other health care stakeholders to appreciate the implications of using eConsult services to reduce wait times for specialist care.
Acknowledgments
The authors thank both the specialist physicians and the primary care providers who participated in the service. In addition, the authors extend their gratitude to the Winchester District Memorial Hospital, which hosts the servers, and to Lois Crowe for her assistance with qualitative analysis.
Notes
This article was externally peer reviewed.
Funding: This study was supported by the Ministry of Health and Long-Term Care, the Ottawa Hospital Academic Medical Organization Innovation Fund, eHealth Ontario, and the Champlain Local Health Integration Network.
Conflict of interest: none declared.
- Received for publication September 2, 2014.
- Revision received December 4, 2014.
- Accepted for publication January 15, 2015.
References
In this issue
Jump to section
Related Articles
Cited By...
- COVID-19 and beyond: virtual consultations in primary care--reflecting on the evidence base for implementation and ensuring reach: commentary article
- A comparison of faxed referrals and eConsult questions for rheumatology referrals: a descriptive study
- Does total triage and remote-by-default consulting impact vulnerable groups: A pilot study
- Patients Assess an eConsult Models Acceptability at 5 US Academic Medical Centers
- Assessment of the Generalizability of an eConsult Service through Implementation in a New Health Region
- Primary Care Clinician Adherence to Specialist Advice in Electronic Consultation
- Development of a virtual benign hematology consultation service: results of a pilot project involving 5 medical centers
- Barriers and facilitators for implementation of electronic consultations (eConsult) to enhance specialist access to care: a scoping review protocol
- Using the Quadruple Aim Framework to Measure Impact of Heath Technology Implementation: A Case Study of eConsult
- Improving Access to Rheumatologists: Use and Benefits of an Electronic Consultation Service
- Use of an electronic consultation system in primary care: a qualitative interview study
- Use of a primary care online consultation system, by whom, when and why: evaluation of a pilot observational study in 36 general practices in South West England
- Improving access to allied health professionals through the Champlain BASE™ eConsult service: a cross-sectional study in Canada
- Supporting Better Access to Chronic Pain Specialists: The Champlain BASE™ eConsult Service
- Prevention of delayed referrals through the Champlain BASE eConsult service
- What are the cost savings associated with providing access to specialist care through the Champlain BASE eConsult service? A costing evaluation
- Electronic Health Record Innovations for Healthier Patients and Happier Doctors