Abstract
Purpose: To report using electronic medical record (EMR) data to identify patients eligible for a clinical trial and the impact of providing an honorarium and deadline on accrual.
Methods: Six practices using a common EMR participated in a cluster-randomized trial testing a self-administered, web-based familial risk assessment tool. EMR-derived lists of eligible patients were made available for provider review. An honorarium and deadline for responding in the patient recruitment letter were implemented in the last half of the recruitment process.
Results: We identified 22,376 potentially eligible patients. Lists not returned by providers accounted for 9840 (44%) patients. We mailed invitations to 11,956 patients; 2398 (20%) requested more information and a consent document, 1489 (12.5%) consented to participate, and 1305 (11%) completed the baseline data collection. Patients receiving the additional $2 and a deadline compared with those receiving the personal invitation alone had significantly higher interest in participating (25% vs. 17%, P = .0001) but were less likely to complete baseline data collection (57% vs. 65% P = .01). Once consented, 85% completed the study with no significant difference by recruitment approach.
Conclusions: Using EMR data reduces the burden to identify potentially eligible patients. However, some providers still did not review and return the lists. Adding a $2 incentive and deadline for responding did not improve the rate of eligible patients consenting and completing the study. Other patient recruitment methods to get better response by providers and population from primary care offices must be explored.
Despite increasing trends toward conducting clinical trials in primary care practice,1 there is little information in the literature that describes specific strategies and logistics involved in enrolling primary care patients into practice-based clinical trials.2–4 Although recruitment of patients from primary care settings has been compared with alternative methods of community-oriented recruitment such as public databases, mass media advertising, community health screening, and patient referrals,5 to our knowledge, no published studies have examined the impact of using electronic medical records (EMRs) data to identify potentially eligible primary care patients for research trials.
To date, identifying eligible patients for primary care practices has been extremely time-consuming and fraught with many problems. Hand-searching paper medical records of varying quality and legibility is inefficient. In addition, the organization of paper medical records varies between practices and even between providers within the same practice. Often the information in the paper medical records is not accurate or legible, with, for example, often conflicting information such as contact information, age, and chronic medical problems. As a result, study participants are primarily volunteers not representative of the population served by the practice. Using an EMR could improve the quality of clinical trials by enabling less biased samples.
As more primary care providers move to an EMR, the identification of potentially eligible patients could be very efficient and accurate. In pediatric clinics from one tertiary center, 11 studies used clinician alerts and most referred an adequate volume of potential subjects (range, 17 to 1162; median, 324). Only a small portion of these potential subjects consented to participate (range, 3% to 25%; median, 11%). Two of the three studies that used EMR-derived patient lists and on-site research assistants reached their enrollment goals.6 In discussions with primary care clinicians in our institution, the EMR-derived list was the overwhelming preferred option to identify eligible patients for research. We report the experience of using EMR data to identify patients for recruitment into a clinical trial of a new web-based health assessment tool. In addition, we examine the impact of providing an honorarium and deadline for responding in the patient recruitment letter on accrual rates.
Methods
The aim of the primary study was to test the clinical utility of Family HealthwareTM. Family HealthwareTM is an interactive on-line tool that provides personalized risk assessments based on an individual's family history of six common chronic diseases and prevention plans with recommendations for lifestyle changes and screening tests. Details about the development of Family HealthwareTM and its features have been described elsewhere.7 The study used a practice-based, cluster-randomized design as described elsewhere in detail.8 Primary care practices were randomized to the intervention or control arm. Approximately 6 months later, both the intervention and control groups completed a follow-up survey.
Six primary care clinics affiliated with the University of Michigan Health Care System agreed to participate in the study. Five of the practices were family medicine and one was general internal medicine. All six practices used the same EMR system for 15 years or more. Three of the six practices had never used any other form of medical records. The five family medicine clinics also used a supplemental software package to create patient registries along with providing prompts and reminders for preventive and chronic care management.9 This system also identified and had the clinician verify the primary care provider of record annually.
The authors were colleagues to all the participating clinicians. Each participating clinic met with the research team to explain the study and review the clinician's role in the study. The primary focus of the meeting was on the process to identify potentially eligible patients and get each clinician's approval to contact the patient about the study per local human subject review requirement.
Electronic problem list, billing, and diagnosis data for each clinic were searched to generate a list of potentially eligible patients for the study by primary care provider. The search parameters were age (35 to 65 years) and sex, with no medical record of coronary heart disease, stroke, diabetes, breast, ovarian, or colorectal cancer. Lists of potentially eligible patients were sent to the primary provider of record. The delivery of the lists was customized to the provider's desires in terms of format (paper copy or electronic copy), number of patients per list, and how often lists were sent. The goal of the clinician's review of patient lists was to identify patients who stood out as not being eligible, who were not interested in getting letters about research, no longer a patient, or any other reason not to contact them.
The research team then mailed approved patients a personal letter from their primary provider introducing the study, requirements of the study, and contacts for the study team (telephone, e-mail addresses, fax numbers, and mailing address). A self-addressed postcard was included for patients to express interest in the study. About halfway through the study accrual, patients assigned to the intervention study arm received a recruitment letter with a $2 bill and noted a deadline for enrolling in the study. The incentive amount was based on findings recruiting patients to on-line health programs.10 We were not able to randomly assign patients to receive or not receive the $2 incentive per our human subject committee review. Therefore, we staged the incentive process.
Patients subsequently contacting the study team and expressing interest in the study were sent study details and written informed consent documents. Once informed consent documents were returned, the participant was provided a study identification number and log-on password to complete baseline data collection and begin the study. Once baseline data collection was completed, participants were mailed a $10 honorarium along with surveys for them to give to any health care provider they saw in the next 6 months. For patients not responding to the first invitation letter, a second letter was sent within 2 to 4 weeks of the first letter without any additional incentive.
About 6 months after completion of the baseline data collection, participants were notified by e-mail or letter to log on to the web site to complete the study data collection. If participants did not complete the month-6 data collection within a week of the first letter, they received up to two additional reminders by e-mail, telephone, or mail. After completion of month-6 data collection, the participants were mailed an honorarium of $10. This report is limited to the baseline data collection.
For each of the six practices, the sex and race of patients between the age of 35 and 65 years without any of the six diseases seen in the last year of this study were determined from billing data. The data were used to determine if the study participants are similar to the usual patients seen in these six practices.
Statistical Analysis
Descriptive statistics were used to characterize the respondents. Comparisons between the two recruitment methods (letter alone versus $2/deadline) were done using χ2 analysis. The analysis was done using SPSS for Windows version 15.0.
Results
As summarized in Figure 1, 22,376 potentially eligible patients were identified from the EMR data in these six primary care practices. The lists were not returned by several providers, which accounted for 9840 patients. This group of 9840 patients was not significantly different from the patients mailed the invitation by age, sex, or race (P > .15). No reasons were given by the providers for not returning the list. Another 581 patients were removed by the primary provider for the reasons listed in Figure 1.
Diagram of recruitment, accrual, and study completion.
The demographic characteristics of the study population for both responders and nonresponders are summarized in Table 1. There were no significant differences between the responders to either recruitment strategy. The nonresponders were significantly (P < .001) more likely to be African American. There were no significant differences (P = .42) in response rate by practice site or between the family medicine sites and the internal medicine site (P = .49). There were no significant differences by age, sex or race for patients on the list not returned by clinicians, removed by clinicians, and ineligible (P > .10).
Demographic Characteristics of the Study Population That Completed Baseline Data Collection
Significantly (P = .002) more women in control practices (15.7%) than in the intervention practices (12.9%) consented to participate in the study. The response was not significantly different (P = .16) among men in the control (10.1%) and intervention (9.2%) practices. Significantly (P < .001) more women (13.9%) consented than men (9.6%). There were no significant differences by race for requesting a consent document or returning a completed consent document (P > .45). Among the consenting participants, the preferred contact method regarding the study was 62% e-mail, 22% mail, and 16% telephone. There was no significant (P = .43) difference on preferred contact method by sex. Completion of baseline data was not different by preferred contact method (P = .21).
As highlighted in Figure 1, 11,956 patients were mailed an invitation to the study. A consent document was requested from 2398 (20%). A returned signed consent document was received from 1489 (12.5%), and 1305 (11%) completed the baseline data collection. Significantly more (P = .0001) of the patients receiving the personal invitation with $2 and a deadline requested a consent (25%) compared with a patient getting the personal invitation only (17%). There were no significant differences in other responses to the invitations by study arm or message.
There were significantly (P = .002) more patients returning a completed consent document after receiving the personal invitation only (66%) than patients getting the $2 and deadline (57%). Significantly (P = .01) more patients completed the baseline data collection who received only the personal invitation (65%) than patients getting the $2 and deadline (57%). The response to the personal invitation versus the invitation plus $2 and deadline was not different between men and women (P = .36). Of the consented study participants, 1111 (75%) completed the entire study. There were no significant differences for completing the study by recruitment method or study arm.
Of the participants completing baseline data collection, 17 men and 14 women, for a total of 31 of 1305, declined the honorarium. Of those declining the payment, 23 participants were from the control arm and 8 participants were from the intervention arm, which is significantly different (P = .001).
Among the patients who consented, 85% completed the study. The completion rate was not significantly different by study arm, sex, or recruitment process (P > .50).
Discussion
As EMRs become more commonplace in primary care offices, more investigators will be able to take advantage of using these databases to identify possible participants for clinical studies. In this study, we highlight that less than 5% of patients expressing an interest in the study were found to be ineligible. This highlights the importance of the content of the EMR and study eligibility criteria. The key eligibility criteria were sex, age, and disease status. All EMRs contain sex and age information. Chronic problem or disease lists are frequently a component of an EMR. Even if a problem list is present, then the accuracy of the data is critical to the utility of such a list. The EMR used by the participating practices contained a problem list populated by provider-entered terms selected from a controlled, clinical set of terms, which avoids colloquial diagnostic labels. Problem lists in the system are routinely verified and updated at each patient visit. As a result, the patients found to be ineligible were rare. Studies that have more complicated eligibility criteria might have more difficulties finding potentially eligible patients. EMRs may not routinely capture race and ethnicity in all patients. Therefore, if this variable is important in the recruitment process, then more missing data will be present.
Our approach was designed to enable our group of busy clinicians in six academic medical practices to efficiently review lists of their patients to approve the recruitment effort, thereby meeting human subject and HIPAA requirements. However, 44% of the patients were not reviewed and returned to the investigators after numerous prompts and reminders and customized requests to each provider. The providers did not give us any feedback about why the lists were not returned. We have explored other options such as printing study brochures at the time of appointment for eligible patients, giving the EMR-derived list to clinicians so they can bring up the study at the time of the appointment, and telephone calls to eligible patients. There are barriers to each of these approaches in our institution. First, our primary care clinicians almost universally do not want to take the limited time of an appointment to explain a study to a patient at the time of an appointment. Therefore, any process that requires time during the appointment is not going to be acceptable. Second, we have found that caller identification process keeps the majority of our patients from taking cold calls from our clinical sites. However, all our clinical sites appear as the University of Michigan on caller identification and not as the unique clinical site. The unique clinic site name appearing on caller identification might improve such a process. Third, our human subjects committee requires that the physician of record from the site approve any contact with patients about research recruitment. Therefore, EMR-derived lists for telephone contact, electronic mail, or regular mail require clinician review and approval. Currently, a clinician cannot give universal approval for all his or her patients.
Paying the clinicians for their time spent reviewing lists of potentially eligible patients is a possible method for increasing clinician response. If the review is a critical, institutional review board–requested activity for conducting research, then yes, we should pay for their time and expertise. It remains to be determined if this would change accruals rates. If clinicians are paid for their assistance with recruitment, would it be more effective to pay for them to introduce the study at appointments with eligible patients? There are not published data to guide us. There are a number of ethical and logistic issues that must be considered. During an appointment, a patient and his or her health insurance is paying for the clinician's time. Is it ethical to take away from this precious, limited time to bring up research? How do we value the time and create an equitable payment? How do we determine if the clinician is actually giving the information to the patient? Paying clinicians for patients that enroll in a study has been deemed unethical.11
Our hypothesis that a $2 honorarium in the recruitment letter and a deadline would get more patients to respond to the invitation was based on other research.10 During the study, our hypothesis seemed to be true because more patients did express an interest in the study and requested a consent document after getting the invitation with $2 and a deadline. However, fewer of this group returned a signed consent document and completed the baseline data entry than patients getting the invitation letter only. The $2 and deadline appears to have enticed patients who were not really interested in completing the study to respond. In addition, the honorarium was returned by a small percentage of the study participants. Therefore, we have concluded that $2 and a deadline to enter the study are not useful strategies to increase accruals to this type of study.
The recruitment process required patients to sign a written informed consent document and return it before getting access to the on-line study. The consent document was eight pages, with a reading level of ninth grade, using the approved template. Given the nature of the study and minimal risk from participation in the study, one could easily question the need for signed written consent. We originally proposed to send patients a letter about the study with secure unique log on identification for each patient. Patients accessing the study site would be taken through an on-line consent process. However, our institution's human subject review board deemed this process unacceptable. We believe it is an acceptable process. We hypothesize that such an approach would have improved the response rate.
Nonresponders to the invitation were more likely to be African American; however, race did not appear to affect request and completion of consent documents. Significantly more women than men responded to the invitation to participate in the study. The invitation letters were identical in content for both study arms; patients did not know which study arm they were assigned. These statistically significant differences are only an absolute difference of 3% to 4% and may not be clinically meaningful. As numerous studies have demonstrated, women are more health-conscious than men, they tend to visit their doctors more often, especially for prevention, and they are also more likely to follow through on protocols related to health care. Any of these factors or their combinations can result in increased participation. Several comparable technology implementation studies described similar trends in participation: white, educated females use web-based health information technology resources more than any other group.12
Even in the current era of e-mail communication, a little over a third of patients preferred to be contacted about the study after consenting via mail or telephone. The response rate to complete study activities was not different based on preferred contact method. However, mail and telephone contact required more resources for the research team.
Even though this clinical trial was very low risk and effort for participants, only 12.5% consented to participate. The retention rate of 85% for the consented participants was very good. The study participants were primarily Caucasian, well-educated, middle class and higher adults recruited from six primary care clinics affiliated with an academic medical center. One could hypothesize that many of the patients not responding to the invitation had less education or income and represented more racial and ethnic minority groups. Another hypothesis could be that more of the nonresponders were likely to have less comfort using computers or access to computers, because the invitation letter described the study as using the Internet to collect health information. The invitation letter did specify that the participant did not need to own a computer or have access to the Internet to participate. A few study participants did use alternative methods other than their own computer and Internet access to complete the study. Other methods to obtain more diverse study populations are being examined.
The study was limited to only six clinical sites using a unique EMR; thus, the findings may not generalize to other practices or EMRs. We do not have any data on the clinicians' experience with any aspect of the study. The accrual process was not the primary outcome of the study; therefore the study design was limited. A significantly more informative study would be a head-to-head comparison of the accuracy, effectiveness, and feasibility of various methods (eg, paper, billing, EMR, large public health databases) to identify study participants. We did not have the resources or flexibility to alter our study recruitment to this degree.
Many questions remain unanswered with respect to the most efficient and feasible method to identify patients eligible for research studies in primary care in the evolving era of EMR. There remain human subject concerns, clinician issues, and patient acceptance. Unfortunately, this critical topic is not easily funded even as a secondary outcome. This paper represents our effort to address this question within a large, funded clinical trial. We encourage other investigators to publish the results of their efforts in this area.
Notes
-
This article was externally peer reviewed.
-
Funding: The Family HealthwareTM Impact Trial (FHITr) was supported through cooperative agreements between the Centers for Disease Control and Prevention and the American Association of Medical Colleges (UM#U36/CCU319276 MM-0789). The trial is registered at clinicaltrials.gov as NCT00164658.
-
Conflict of interest: none declared.
- Received for publication July 20, 2010.
- Revision received February 11, 2011.
- Accepted for publication February 14, 2011.
