Implementation of Technology-based Patient Engagement Strategies within Practice-based Research Networks
========================================================================================================

* Beth Careyva
* Kyle Shaak
* Geoffrey Mills
* Melanie Johnson
* Samantha Goodrich
* Brian Stello
* Lorraine S. Wallace

## Abstract

*Background:* Technology-based patient engagement strategies (such as patient portals) are increasingly available, yet little is known about current use and barriers within practice-based research networks (PBRNs). PBRN directors have unique opportunities to inform the implementation of patient-facing technology and to translate these findings into practice.

*Methods:* PBRN directors were queried regarding technology-based patient engagement strategies as part of the 2015 CAFM Educational Research Alliance (CERA) survey of PBRN directors. A total of 102 PBRN directors were identified via the Agency for Healthcare Research and Quality's registry; 54 of 96 eligible PBRN directors completed the survey, for a response rate of 56%.

*Results:* Use of technology-based patient engagement strategies within PBRNs was limited, with less than half of respondents reporting experience with the most frequently named tools (risk assessments/decision aids). Information technology (IT) support was the top barrier, followed by low rates of portal enrollment. For engaging participant practices, workload and practice leadership were cited as most important, with fewer respondents noting concerns about patient privacy.

*Discussion:* Given limited use of patient-facing technologies, PBRNs have an opportunity to clarify the optimal use of these strategies. Providing IT support and addressing clinician concerns regarding workload may facilitate the inclusion of innovative technologies in PBRNs.

*   Decision Support Techniques
*   Electronic Health Records
*   Health Services Research
*   Patient Participation
*   Practice-based Research
*   Privacy
*   Registries
*   Risk Assessment
*   Surveys & Questionnaires
*   United States Agency for Healthcare Research and Quality
*   Workload

Practice-based research network (PBRN) directors are uniquely poised to answer questions regarding the pragmatic implementation of evolving patient engagement technologies within PBRNs. Given the increasing availability of technology to communicate with patients, it was anticipated that PBRN directors may have valuable experience to share in using patient engagement technologies within practice-based research. PBRNs offer a rich research laboratory across multiple settings, ownership models, and organizational models, and even more diverse populations within ambulatory primary care practices.1 Primary care researchers within PBRNs have unique opportunities to inform the implementation of patient-facing technology, thereby translating research findings into practice.2,3

Patient engagement strategies encompass information exchange and shared decision making to promote patient activation and self-management.4 Using technology—including videoconferencing, web-based tools, texting, and patient portals—for patient engagement may be instrumental in facilitating patient-centered care and improving both health-related outcomes and workflow.5⇓⇓–8 Technology-based patient engagement strategies include activities such as the transmission of clinical questions, biometric data, patient surveys (eg, decision aids and patient-reported outcomes), and risk calculators. Real-world examples include the use of videoconferencing to facilitate patient visits in rural areas, the use of portals to collect patient-reported outcomes, and Internet-connected scales to communicate daily body weight.

With the combination of an aging population and a primary care workforce shortage, there is a greater-than-ever need to explore innovative technology strategies that empower patients to be active participants in their health and health care.9,10 Primary care research is needed to inform the optimal use of these technologies to meet the tenets of the triple aim of better care, better quality, and lower cost.11,12 However, little is known about current research efforts related to the use of technology to engage patients and, more important, which barriers need to be overcome to optimize the use of such innovations. Technology has the potential to improve systems of care and health outcomes; therefore, further research is needed to inform the optimal implementation and evaluation of outcomes moving forward.8,13,14

Clinician acceptance has been shown to be a major factor in the uptake of new technologies in practice.15 In a prior survey of clinicians, while most felt positive about technology use for patient engagement, concerns about workload and patient safety also surfaced.16 While the issue of clinician attitudes toward technology use has been examined previously, to our knowledge, no studies have queried technology use and acceptance among those leading PBRNs. To address this gap in the literature, the purpose of this study was to assess the current use of and perceived barriers to patient engagement technologies within practice-based research among a sample of PRBN directors throughout the United States and Canada.

## Methods

Data for this study were collected via the Council of Academic Family Medicine Educational Research Alliance's (CERA) 2015 survey of PBRN directors. CERA is a joint initiative of all 4 major US academic family medicine organizations (Society of Teachers of Family Medicine, North American Primary Care Research Group, Association of Departments of Family Medicine, and Association of Family Medicine Residency Directors). Detailed information about CERA has been published elsewhere.17 All procedures used in this study were approved by the American Academy of Family Physicians Institutional Review Board.

### Data Collection Procedures

Between September and October 2015, PRBN directors (n = 102) identified within the Agency for Healthcare Research and Quality's PBRN registry received an electronic invitation to complete a 10-minute survey. The survey was introduced as part of an electronic message that included a personalized greeting, a letter signed by each CERA organization president, and a link to the survey. Nonrespondents were sent 3 follow-up E-mails encouraging participation. Five E-mail invitations were returned (“bounced back”) and 1 individual indicated that the PBRN was no longer active; therefore, the useable sample included a total of 96 PBRN directors. Overall, 54 PBRN directors completed the survey, yielding a 56% overall response rate.

A collaborative team of researchers developed original content to design the 10-question survey. The survey included a collection of close-ended items addressing (1) demographic characteristics related to the scope and functioning of each director's PBRN, (2) current use of technology-based patient engagement strategies, (3) perceived barriers to the use of patient engagement technologies, and 4) factors affecting participant practice recruitment. Sample items included: “What proportion of practices within your PBRN have access to patient portals that could be used for research purposes,?” “In your opinion, which of the following are the 3 most important factors in determining which technology-based strategy to incorporate into your practice-based research?” and “How likely are each of the following characteristics to affect physician participation in studies using technology-based patient engagement strategies?” The questions pertaining to technology-based patient engagement strategies are included in the Appendix. Survey items aimed to evaluate the impact of PBRN scope (local, state, regional, national) on the use of technology-based patient engagement strategies within PBRNs, the role of patient portals in the overall use of technology-based patient engagement strategies, and the influence of familiarity with Health Insurance Portability and Accountability Act (HIPAA) regulations for technology use. Hypotheses being tested included the following: (1) regional/national PBRNs are more likely to be using technology-based patient engagement strategies than local/state PBRNs, and (2) PBRNs with increased access to patient portals in member practices are more likely to be engaged in studies involved patient engagement technologies.

### Data Analysis

Descriptive statistics (frequencies, percentages) were used to depict characteristics of PBRNs in terms of current practices, barriers and facilitators, and attitudes toward the use of technology-based patient engagement strategies. For survey items that included count data, χ2 tests were used to compare PBRNs in terms of current practices, barriers and facilitators, and attitudes toward the use of technology-based patient engagement strategies. All data analyses were conducted using the SPSS version 22.0 (SPSS, Inc./IBM, Chicago, IL). Response rates varied per survey question, as represented in the tables.

## Results

Overall PBRN characteristics are presented in Table 1. Survey respondents included representation from local (n = 11), state (n = 18), regional (n = 16), and national (n = 9) PBRNs. Almost half (46%) of PBRN directors were leading PBRNs that had been in existence a decade or longer.

View this table:
[Table 1.](http://www.jabfm.org/content/29/5/581/T1)

Table 1. 
Practice-Based Research Network Characteristics

The proportion of studies within each PBRN using technology-based patient engagement strategies is listed in Table 2. Of the 49 PBRN directors who responded to this question, 39 (80%) indicated using technology-based patient engagement strategies in fewer than 25% of their studies in the past 5 years. When comparing the percentage of studies using technology-based patient engagement strategies in the past 5 years, no significant differences were observed based on the scope of the PBRN (local/state vs regional/national; χ2 = 3.91; *P* > .05).

View this table:
[Table 2.](http://www.jabfm.org/content/29/5/581/T2)

Table 2. 
Percentage of Studies Including Technology-Based Patient Engagement Strategies among Practice-Based Research Networks

Technology-based patient engagement strategies used in the past 5 years by PBRNs are outlined in Table 3. Risk assessments or decision aids administered by web-based tools (n = 20), waiting room kiosks/computers (n = 13), or smartphone/personal tablet applications (n = 13) were most commonly used, whereas videoconferencing for patient visits (n = 4) and texting to transmit clinical information (n = 4) were used least frequently.

View this table:
[Table 3.](http://www.jabfm.org/content/29/5/581/T3)

Table 3. 
Practice-Based Research Network Use of Technology-Based Patient Engagement Strategies in the Past 5 Years

In terms of importance in determining which type of technology-based patient engagement strategy to incorporate into research studies, PBRN directors most frequently identified ease of use for practices (n = 33), ease of use for patients (n = 30), and ease of use for clinicians (n = 23). Factors least likely to affect the type of technology selected included cost (n = 16) and patient population (n = 5).

Directors' rankings of the top barriers to incorporating technology into research studies are reported in Table 4. Low rates of portal enrollment was selected as a top barrier, though only 47% PBRNs (n = 22) had access to patient portals in >50% of their practices. PBRN directors reported access to patient portals (Table 5). Among PBRN directors reporting technology-based patient engagement strategies in 0%, 1% to 50%, and >50% of studies, those who reported access to portals in ≥50% of the practices were not significantly more likely to have engaged in studies involving technology-based patient engagement strategies than PBRNs with access to portals in <50% of practices (χ2 = 1.92; *P* > .05).

View this table:
[Table 4.](http://www.jabfm.org/content/29/5/581/T4)

Table 4. 
Barriers to Incorporating Technology-Based Patient Engagement Strategies into Practice

View this table:
[Table 5.](http://www.jabfm.org/content/29/5/581/T5)

Table 5. 
Practices with Patient Portals in Each Practice-Based Research Network

In terms of familiarity with HIPAA regulations pertaining to the transfer of electronic information, PBRN directors described practices being most familiar with E-mail, followed by patient portals, texting, and videoconferencing (Figure 1). There were no significant differences between familiarity with HIPAA regulations (not at all familiar/slightly familiar/somewhat familiar vs moderately familiar/extremely familiar) and use of E-mail (χ2 = 0.12; *P* > .05), portal use (χ2 = 0.037; *P* > .05), texting (χ2 = 1.14;*P* > .05), or videoconferencing (χ2 = 1.42; *P* > .05).

![Figure 1.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/29/5/581/F1.medium.gif)

[Figure 1.](http://www.jabfm.org/content/29/5/581/F1)

Figure 1. 
Familiarity with Health Insurance Portability and Accountability Act regulations for the transmission of electronic data.

Of the 39 PBRN directors who reported the use of technology-based patient engagement strategies, 35 responded to the survey question regarding the ease of recruitment for studies using these strategies. Of these, 54% reported ease of recruitment as good (n = 15) or very good (n = 4). A total of 34% of responding directors (n = 12) reported ease of recruitment as fair and 11% (n = 4) reported it as poor.

PBRN directors' perceptions of factors that may affect physician participation in studies are shown in Figure 2. Overall, workload was the most concerning factor identified, followed closely by practice leadership. Practice models identified as most likely to participate in PBRN studies were individual practitioner or group practice (n = 14), hospital-owned and university-affiliated (n = 11), federally qualified health center (n = 7), hospital-owned community-based practice (n = 2), and federal/national government health care agency (n = 2).

![Figure 2.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/29/5/581/F2.medium.gif)

[Figure 2.](http://www.jabfm.org/content/29/5/581/F2)

Figure 2. 
Factors that may affect physician participation in studies including technology-based patient engagement strategies.

## Discussion

This study aimed to evaluate the current use of, barriers to, and clinician acceptance of patient engagement technologies within studies conducted in PBRNs. Based on the increasing availability of new technologies, it was anticipated that PBRN directors would have experience with studies encompassing many different technology-based modalities for patient engagement. Instead, some of the most common technology-based tools (risk assessments or decision aids transmitted by web-based tools, kiosks, smartphones/tablets, or portals) were used by less than half of the PBRNs in the past 5 years. Decision aids have been found to improve knowledge and decrease decisional conflict when administered through different modalities and settings, which may explain their routine use within practice-based research.18 Further study is needed to clarify the optimal use of technology-based tools (such as videoconferencing, texting, and portals) to transmit other types of clinical data, including the contexts in which they may be most beneficial.19⇓–21

It was expected that regional and national PBRNs would be more likely to have experience with technology-based patient engagement strategies because of increased access to technology within diverse practice sites, but no significant differences were noted based on scope of PBRN (local/state vs regional/national). This may be related to variability in the number of practices and resources within each PBRN, irrespective of geographic coverage, as well as increased probability of a greater number of technology platforms for regional and national PBRNs. The overall limited use of patient engagement technologies in practice-based research identified in this survey is consistent with a recent evaluation of patient engagement technologies in the inpatient setting, in which a systematic review highlighted limited use and gaps in the literature surrounding optimal implementation.22

In terms of factors influencing the inclusion of technology in practice-based research studies, PBRN directors reported ease of use (for patients, practices, and clinicians, in descending order) as the most frequently cited factor in determining which technology-based patient engagement strategy to use in research. User-friendly technology has been found to be critical to successful implementation in the practice setting.23 Cost and the availability of tools in multiple languages were cited less frequently as factors in choosing which type of technology to implement. While a prior study noted cost as a barrier,24 this may be less relevant to practice-based researchers who are seeking to optimize the use of existing technologies in practices rather than purchasing new systems.

Linked to electronic medical records, patient portals are an example of existing technology available in an ever increasing number of practices for research purposes. However, most directors in this sample reported access to portals in less than half of practices within their PBRN. Given the potential for portals to be used to transmit decision aids, risk assessments, and clinical information, and to maximize the study sample and standardize data collection methods, it was expected that PBRNs with portal access in a greater number of practices may use technology-based patient engagement strategies more frequently than PBRNs with a patient portal in fewer practices. The lack of significant differences noted between PBRNs with varying degrees of access to patient portals and studies involving technology may be the result of both low rates of enrollment and a limited understanding of optimal implementation of patient portals for both clinical and research purposes. Further efforts to facilitate patient enrollment and to evaluate who might benefit most from patient engagement via patient portals may facilitate their use within practice-based research.

In addition to establishing current use of technology-based patient engagement strategies within PBRNs, directors were queried about barriers to technology implementation. Information technology (IT) support was the most frequently cited barrier to incorporating technology in practice settings. It is unclear whether the IT barrier is limited availability, lack of IT personnel trained in implementation science, institutional regulations, or something else. Further study is needed to clarify the type of IT support needed to optimize the implementation of technology for patient engagement. Similar to other studies, practice champions were also found to be particularly critical to successful implementation.25 Technology-based tools in multiple languages were cited as a barrier less frequently than anticipated based on a prior study, though this is undoubtedly dependent on the patient populations of each PBRN.26 Access to smartphones was 1 of the least commonly cited barriers, which corresponds to a recent study indicating that 55% of primary care patients use smartphones, and 70% of those patients with smartphones use them for health-related purposes.27

An additional potential barrier to studies involving technology-based patient engagement strategies is clinician acceptance. Clinician acceptance is critical to the successful recruitment of practices and has been found to facilitate study implementation.28 We aimed to determine factors that were most critical for physician participation in studies involving technology for patient engagement. All but 1 PBRN director cited workload as a major factor in physician participation, whereas patient safety and patient privacy were cited less frequently than in prior studies.16,29 This may be in part the result of evolving experience and institutional policies regarding the management of patient privacy and patient safety within patient-facing technology-based interventions. Further study is needed to clarify whether workload is a greater concern for studies involving patient engagement technologies than for other types of practice-based research.

In light of expected concerns with patient privacy, PBRN directors reported practice familiarity with HIPAA regulations for transmission of clinical data. The directors reported variable practice familiarity with HIPAA regulations; practices were most familiar with regulations regarding E-mail and least familiar with those regarding videoconferencing. To be HIPAA compliant, E-mail, texting, portals, and other forms of patient-facing technology need to include safeguards for the transmission of protected health information, potentially including encryption and patient logins.30,31 There is more debate about adequate safeguards for videoconferencing, which is reflected in the PBRN directors' perceptions of decreased familiarity with HIPAA regulations for videoconferencing within practices. While it was thought that PBRNs reporting increased familiarity with HIPAA regulations may be more likely to use various technology-based tools to transmit clinical information, no significant differences were noted with respect to HIPAA familiarity and the reported percentage of studies using technology for patient engagement. This may be linked to other findings within this study indicating that patient privacy was not a major factor affecting physician participation. In addition, a more recent study confirmed the feasibility of protecting patient privacy while harnessing mobile technology to enhance patient and family engagement32; however, questions still remain regarding adequate protection of patient privacy within other modalities such as videoconferencing.33

### Limitations

The findings generated from this study should be considered within the context of several limitations. First, generalizability of the study findings is limited by the response rate (56%) of PBRN directors polled. Second, as with all studies that rely on self-report, response bias remains a possibility. Third, the cross-sectional nature of the study limits causality. Last, reporting is second-hand; individual PBRN-member clinics and clinicians may have different perceptions of technology strategies than the PBRN director, particularly with regard to knowledge of HIPAA regulations.

## Conclusion

Decision aids and risk assessments were the most commonly used types of technology-based patient engagement strategies within the PBRNs. Even in practices with access to portals and other technologies, few PBRNs are engaging in studies of technology implementation or related outcomes. IT support was the top-ranked barrier to incorporating technology into practice, though practice champions were also cited as being critical to successful implementation. For participant practices, workload and practice leadership were cited as major factors in physician participation in studies involving technology-based patient engagement strategies—more so than concerns about patient safety and patient privacy. Optimizing IT support and addressing workload concerns may facilitate increased use of technology-based patient engagement strategies within practice-based research.

## Acknowledgments

The authors thank Jacqueline Grove for assistance with editing and manuscript preparation.

## Appendix

#### Questions Regarding Technology-Based Patient Engagement Strategies

1.  Below are some examples of technology-based patient engagement strategies. Has your PBRN conducted studies that relied on any of the following technology-based patient engagement strategies in the past 5 years? (Please select all that apply.) 
    
    *   Patient visits with primary care physicians remotely by audio/video
    
    *   Electronic transmission of vital signs or clinical data (eg, blood pressure, weight, blood glucose)
    
    *   Waiting room kiosk computers/tablets for risk assessments or decision aids
    
    *   Web-based risk assessments or decision aids
    
    *   Smartphones/personal tablet applications for risk assessments or decision aids
    
    *   Electronic medical record portal-based risk assessments or decision aids
    
    *   Texting to patients (eg, reminders, educational materials)
    
    *   Texting from patients (eg, transmission of vital signs or clinical data)
    
    *   E-mail

2.  What proportion of studies within your PBRN have relied on technology-based patient engagement strategies in the past 5 years? 
    
    *   0%
    
    *   1% to 25%
    
    *   26% to 50%
    
    *   51% to 75%
    
    *   76% to 99%
    
    *   100%

3.  What proportion of practices within your PBRN have access to patient portals that could be incorporated for research purposes? 
    
    *   0%
    
    *   1% to 25%
    
    *   26% to 50%
    
    *   51% to 75%
    
    *   76% to 99%
    
    *   100%

4.  In your opinion, which of the following are the 3 most important factors in determining which technology-based patient engagement strategy to incorporate into your practice-based research? 
    
    *   Clinical utility
    
    *   Ease of use for patients
    
    *   Ease of use for clinicians
    
    *   Ease of implementation for practices
    
    *   Availability of a particular technology
    
    *   Cost
    
    *   Patient population

5.  How familiar are your practices with HIPAA regulations surrounding the transmission of electronic data pertaining to: 
    
    *   Patient portals
    
    *   E-mail
    
    *   Texting
    
    *   Videoconferencing
    
    *Responses to question 5 were rated on a Likert scale (1 to 5): not at all familiar, slightly familiar, somewhat familiar, moderately familiar, extremely familiar.*

6.  In your experience as a PBRN director, what are the 3 most critical barriers to successfully incorporating technology-based patient engagement strategies into clinical practice? 
    
    *   Lack of patient portals within electronic medical records
    
    *   Low rates of portal enrollment among patients
    
    *   Access to smartphones
    
    *   Cost of health care technology
    
    *   Staff comfort with use of technology
    
    *   Patient willingness to use technology
    
    *   Tools that interface with electronic medical records
    
    *   Availability of tools in multiple languages
    
    *   IT support
    
    *   Lack of practice champions

7.  In your experience as a PBRN director, what are the 3 most critical facilitators to successfully incorporating technology-based patient engagement strategies into clinical practice? 
    
    *   Patient portals within electronic medical records
    
    *   Portal enrollment
    
    *   Access to smartphones
    
    *   Cost of health care technology
    
    *   Staff comfort with use of technology
    
    *   Patient willingness to use technology
    
    *   Tools that interface with electronic medical records
    
    *   Availability of tools in multiple languages
    
    *   IT support
    
    *   Practice champions

8.  Across all studies incorporating technology-based patient engagement strategies in your PBRN, please describe the overall ease of recruitment of participant practices for the studies. 
    
    *   Poor
    
    *   Fair
    
    *   Good
    
    *   Very good
    
    *   Excellent

9.  How likely are the following to affect physician participation in studies using technology-based patient engagement strategies? 
    
    *   Comfort with technology
    
    *   Training
    
    *   Practice leadership
    
    *   Concerns about workload or time
    
    *   Concerns about patient safety
    
    *   Concerns about patient privacy
    
    *   Sustainability of interventions
    
    *Responses to question 9 were rated on a Likert scale (1 to 5): extremely unlikely, unlikely, neutral, likely, extremely likely.*

10. In your opinion, what practice ownership model is most likely to participate in studies using technology-based patient engagement strategies? 
    
    *   Individual practitioner or group practice
    
    *   Hospital-owned, university-affiliated practice
    
    *   Hospital-owned, community-based practice
    
    *   Federally qualified health center
    
    *   Federal or national government health care agency
    
    *   Not sure

## Notes

*   This article was externally peer reviewed.

*   *Funding:* none.

*   *Conflict of interest:* none declared.

*   Received for publication January 28, 2016.
*   Revision received April 18, 2016.
*   Accepted for publication April 22, 2016.

## References

1.  1.Agency for Healthcare Research and Quality. Primary care practice-based research networks. Available from: [http://www.ahrq.gov/research/findings/factsheets/primary/pbrn/index.html](http://www.ahrq.gov/research/findings/factsheets/primary/pbrn/index.html). Updated October 2013. Accessed May 11, 2015.
    
    

2.  2.Heintzman J, Gold R, Krist A, Crosson J, Likumahuwa S, DeVoe JE. Practice-based research networks (PBRNs) are promising laboratories for conducting dissemination and implementation research. J Am Board Fam Med 2014;27:759–62.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjcvNi83NTkiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI5LzUvNTgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

3.  3.Krist AH, Green LA, Phillips RL, et al. Health information technology needs help from primary care researchers. J Am Board Fam Med 2015;28:306–10.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjgvMy8zMDYiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI5LzUvNTgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

4.  4.James J. Health policy brief: Patient engagement. Available from: [http://www.healthaffairs.org/healthpolicybriefs/brief.php?brief\_id=86](http://www.healthaffairs.org/healthpolicybriefs/brief.php?brief_id=86). Updated February 14, 2013. Accessed May 15, 2015.
    
    

5.  5.Mosen DM, Schmittdiel J, Hibbard J, Sobel D, Remmers C, Bellows J. Is patient activation associated with outcomes of care for adults with chronic conditions? J Ambul Care Manage 2007;30:21–9.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1097/00004479-200701000-00005&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=17170635&link_type=MED&atom=%2Fjabfp%2F29%2F5%2F581.atom) 

6.  6.Riva G, Baños RM, Botella C, Wiederhold BK, Gaggioli A. Positive technology: using interactive technologies to promote positive functioning. Cyberpsychol Behav Soc Netw 2012;15:69–77.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1089/cyber.2011.0139&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=22149077&link_type=MED&atom=%2Fjabfp%2F29%2F5%2F581.atom) 

7.  7.Finkelstein J, Knight A, Marinopoulos S, et al. Enabling patient-centered care through health information technology. AHRQ publication no. 12-E005-EF. Available from: [http/www.effectivehealhcare.ahrq.gov/reports/final.cfm](http://http/www.effectivehealhcare.ahrq.gov/reports/final.cfm). Rockville, MD: Agency for Healthcare Research and Quality; 2012.
    
    

8.  8.Ricciardi L, Mostashari F, Murphy J, Daniel JG, Siminerio EP. A national action plan to support consumer engagement via e-health. Health Aff (Millwood) 2013;32:376–84.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjMyLzIvMzc2IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8yOS81LzU4MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

9.  9.Kvedar J, Coye M, Everett W. Connected health: a review of technologies and strategies to improve patient care with telemedicine and telehealth. Health Aff (Millwood) 2014;33:194–9.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjMzLzIvMTk0IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8yOS81LzU4MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

10. 10.Petterson SM, Liaw WR, Phillips RL Jr., Rabin DL, Meyers DS, Bazemore AW. Projecting US primary care physician workforce needs: 2010–25. Ann Fam Med 2012;10:503–9.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTAvNi81MDMiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI5LzUvNTgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

11. 11.Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff (Millwood) 2008;27:759–69.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjI3LzMvNzU5IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8yOS81LzU4MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

12. 12.Phillips RL Jr., Bazemore AW, DeVoe JE, et al. A family medicine health technology strategy for achieving the triple aim for US health care. Fam Med 2015;47:628–35.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=26382121&link_type=MED&atom=%2Fjabfp%2F29%2F5%2F581.atom) 

13. 13.Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med 2010;363:501–4.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1056/NEJMp1006114&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=20647183&link_type=MED&atom=%2Fjabfp%2F29%2F5%2F581.atom) 

14. 14.Ahern DK, Woods SS, Lightowler MC, Finley SW, Houston TK. Promise of and potential for patient-facing technologies to enable meaningful use. Am J Prev Med 2011;40:S162–72.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/j.amepre.2011.01.005&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=21521591&link_type=MED&atom=%2Fjabfp%2F29%2F5%2F581.atom) 

15. 15.Ward R, Stevens C, Bretnall P, Briddon J. The attitudes of health care staff to information technology: a comprehensive review of the research literature. Health Info Libr J 2008;25:81–97.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1111/j.1471-1842.2008.00777.x&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=18494643&link_type=MED&atom=%2Fjabfp%2F29%2F5%2F581.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000256100400002&link_type=ISI) 

16. 16.Davidson E, Simpson CR, Demiris G, Sheikh A, McKinstry B. Integrating telehealth care-generated data with the family practice electronic medical record: qualitative exploration of the views of primary care staff. Interact J Med Res 2013;2:e29.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.2196/ijmr.2820&link_type=DOI) 

17. 17.Mainous AG, Seehusen D, Shokar N. CAFM Educational Research Alliance (CERA) 2011 residency director survey: background, methods, and respondent characteristics. Fam Med 2012;44:691–3.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=23147999&link_type=MED&atom=%2Fjabfp%2F29%2F5%2F581.atom) 

18. 18.Stacey D, Légaré F, Col NF, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev 2014;1:CD001431.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=24470076&link_type=MED&atom=%2Fjabfp%2F29%2F5%2F581.atom) 

19. 19.Irizarry T, DeVito Dabbs A, Curran CR. Patient portals and patient engagement: a state of the science review. J Med Internet Res 2015;17:e148.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.2196/jmir.4255&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=26104044&link_type=MED&atom=%2Fjabfp%2F29%2F5%2F581.atom) 

20. 20.Goldzweig CL, Orshansky G, Paige NM, et al. Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med 2013;159:677–87.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.7326/0003-4819-159-10-201311190-00006&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=24247673&link_type=MED&atom=%2Fjabfp%2F29%2F5%2F581.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000328732700016&link_type=ISI) 

21. 21.Fanning J, Mullen SP, McAuley E. Increasing physical activity with mobile devices: a meta-analysis. J Med Internet Res 2012;14:e161.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.2196/jmir.2171&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=23171838&link_type=MED&atom=%2Fjabfp%2F29%2F5%2F581.atom) 

22. 22.Prey JE, Woollen J, Wilcox L, et al. Patient engagement in the inpatient setting: a systematic review. J Am Med Inform Assoc 2014;21:742–50.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiamFtaW5mbyI7czo1OiJyZXNpZCI7czo4OiIyMS80Lzc0MiI7czo0OiJhdG9tIjtzOjIwOiIvamFiZnAvMjkvNS81ODEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

23. 23.Vuononvirta T, Timonen M, Keinänen-Kiukaaniemi S, et al. The compatibility of telehealth with health-care delivery. J Telemed Telecare 2011;17:190–4.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BqdHQiO3M6NToicmVzaWQiO3M6ODoiMTcvNC8xOTAiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI5LzUvNTgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

24. 24.Walker J, Whetton S. The diffusion of innovation: factors influencing the uptake of telehealth. J Telemed Telecare 2002;8:73–5.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BqdHQiO3M6NToicmVzaWQiO3M6MTI6Ijgvc3VwcGxfMy83MyI7czo0OiJhdG9tIjtzOjIwOiIvamFiZnAvMjkvNS81ODEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

25. 25.Shaw EK, Howard J, West DR, et al. The roles of the champion in primary care change efforts: from the State Networks of Colorado Ambulatory Practices and Partners (SNOCAP). J Am Board Fam Med 2012;25:676–85.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjUvNS82NzYiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI5LzUvNTgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

26. 26.Dawson J, Doll H, Fitzpatrick R, Jenkinson C, Carr AJ. The routine use of patient reported outcome measures in healthcare settings. BMJ 2010;340:c186.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE2OiIzNDAvamFuMThfMS9jMTg2IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8yOS81LzU4MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

27. 27.Bauer AM, Rue T, Keppel GA, Cole AM, Baldwin LM, Katon W. Use of mobile health (mHealth) tools by primary care patients in the WWAMI Region Practice and Research Network (WPRN). J Am Board Fam Med 2014;27:780–8.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjcvNi83ODAiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI5LzUvNTgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

28. 28.Wade VA, Eliott JA, Hiller JE. Clinician acceptance is the key factor for sustainable telehealth services. Qual Health Res 2014;24:682–94.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BxaHIiO3M6NToicmVzaWQiO3M6ODoiMjQvNS82ODIiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI5LzUvNTgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

29. 29.Beasley JW, Wetterneck TB, Temte J, et al. Information chaos in primary care: implications for physician performance and patient safety. J Am Board Fam Med 2011;24:745–51.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjQvNi83NDUiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI5LzUvNTgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

30. 30.US Department of Health and Human Services. Does the HIPAA Privacy Rule permit health care providers to use e-mail to discuss health issues and treatment with their patients? Available from: [http://www.hhs.gov/hipaa/for-professionals/faq/570/does-hipaa-permit-health-care-providers-to-use-email-to-discuss-health-issues-with-patients/index.html](http://www.hhs.gov/hipaa/for-professionals/faq/570/does-hipaa-permit-health-care-providers-to-use-email-to-discuss-health-issues-with-patients/index.html). Updated December 15, 2008. Accessed December 23, 2015.
    
    

31. 31.Office of the National Coordinator for Health Information Technology. Guide to privacy and security of health information. Chapter 4: Understanding electronic health records, the HIPAA security rule, and cybersecurity. Available from: [https://www.healthit.gov/sites/default/files/pdf/privacy/privacy-and-security-guide-chapter-4.pdf](https://www.healthit.gov/sites/default/files/pdf/privacy/privacy-and-security-guide-chapter-4.pdf). Accessed January 23, 2016.
    
    

32. 32.Gordon CR, Rezzadeh KS, Li A, et al. Digital mobile technology facilitates HIPAA-sensitive perioperative messaging, improves physician-patient communication, and streamlines patient care. Patient Saf Surg 2015;9:21.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1186/s13037-015-0070-9&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=26136830&link_type=MED&atom=%2Fjabfp%2F29%2F5%2F581.atom) 

33. 33.Armfield NR, Gray LC, Smith AC. Clinical use of Skype: a review of the evidence base. J Telemed Telecare 2012;18:125–7.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BqdHQiO3M6NToicmVzaWQiO3M6ODoiMTgvMy8xMjUiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzI5LzUvNTgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==)