Abstract
Background: Practice-based research network (PBRN) study investigators must interface with multiple Institutional Review Boards (IRBs), many of which are unfamiliar with PBRN research.
Objective: To present 2 IRB-related issues that have not appeared in the literature but occurred during the course of a large 5-year PBRN study involving 32 sites dispersed around the United States.
Results: Our study required IRB approval from a total of 19 local, hospital, academic center, and professional organization-based IRBs that reviewed a protocol of postpartum depression screening and follow-up completed in English or Spanish. Initial approval of the protocol and consent forms proceeded with only the usual barriers of submitting 19 different forms, and no protocol amendments were required. However, 2 unanticipated IRB issues provided significant additional work for the study team and the local practice sites. First, several IRBs required staff to repeat human subjects training every 1 to 2 years, resulting in 92 practicing physicians, residents, and members of the nursing staff spending time completing the exact same human subjects’ training at least twice during the course of this study. Second, 17 of the 19 IRBs required that the patient be given consent forms that were newly stamped and dated each year, requiring the central site to reprint and replace consent forms yearly. Because not all IRBs returned the newly stamped and dated forms in a timely fashion, study enrollment with valid consent forms was interrupted in 4 sites for periods of 2 to 13 weeks.
Conclusions: IRB requirements not directly responsive to federal regulations can add significant costs, frustrations, and burdens to PBRN studies. Non–federally mandated IRB requirements should be based on an identified need with evidence to support the solution.
Practice-based research network (PBRN) studies have several unique issues when dealing with Institutional Review Boards (IRBs), which have been discussed in previous publications.1–11 These issues include obtaining approval for sites that are not affiliated with a local or regional IRB,1 the need to obtain approvals from multiple site-specific IRBs,1,3,4,6 the variation in requirements for submission and local rules for study assessments among multiple IRBs, and requirements for varying types of human subject training.11 Solutions have been recommended and adopted for many of these issues.1,2,10,12 For example, central IRBs that review and approve PBRN protocols on behalf of multiple sites and other IRBs have been proposed and tested with at least limited success.10,13,14 The use of Individual Investigator Agreements to allow an existing IRB to function at the behest of unaffiliated practices rather than turning to review by a commercial IRB has been effective for PBRNs such as the American Academy of Family Physicians National Research Network.2,5,10,13,15
As PBRN studies have evolved from simple cross-sectional designs to longitudinal interventions and randomized controlled trials, other IRB-related issues have arisen.16 Here we discuss 2 issues that were encountered during the conduct of a 5-year study of postpartum depression screening and management related to the retraining of local practice staff about working with human subjects yearly or every other year and to a yearly “date stamp” on patient consent forms. These issues became significant barriers to study completion and have led at least 2 local sites to vow to never do another practice-based research study. In this article we review the implications of these IRB requirements for the practices and for the overall study and suggest alternative approaches to deal with the issues.
Methods
The issues discussed here were identified during the second and third year of a large national PBRN study started in 31 practices in 20 different states with 3 investigator sites, including the Olmsted Medical Center (Rochester, MN); the American Academy of Family Physicians (AAFP); and the Department of Family Medicine at Dartmouth College (Hanover, NH). Approval of the study was obtained from 19 IRBs: 15 existing community, hospital, or health system IRBs; 3 university-based IRBs; and the AAFP IRB, which also served as the central IRB for 13 unaffiliated practices.5
The Translating Research Into Practice for Postpartum Depression study is a randomized controlled trial requiring enrollment and informed consent of women between 5 and 12 weeks postpartum. At each site, eligible women are enrolled during a period of 3 years, with a 1-year follow-up period for each woman. Within each of the PBRN practice sites, informed consent is also obtained from each physician, other clinicians, and all members of the nursing staff to allow study of their practice processes related to postpartum depression. The involvement of each practice lasts for approximately 36 months.11 Human subjects training was required for all practice personnel who would be involved in the patient consent process, as well as for the lead study physician and nurse coordinator from each site. The required training varied by site and in many instances was specified by the site's IRB. The largest group of practices, those covered by the AAFP IRB, were all required to do the Collaborative Institutional Training Initiative (CITI) online training. Other sites required the training developed by the medical campus of their state university or by the large health plan with which they were affiliated. Three IRBs accepted the training originally developed by the National Cancer Institute.17 All training covered the history of human subjects training requirements, the basic principles of the Federal Registry requirements, and specific examples of requirements in cases of emergency drug or device use, the need for consent from a proxy, and role of the IRB. None of the programs were tailored for practice-based research or effectiveness studies. Material in the initial training and retraining processes were identical.
Initial approval was obtained from the 19 IRBs, including approval of the English and Spanish versions of the consent form. A yearly review and reaffirmation of approval was required at all sites per federal regulations. The initial approval presented no new or unanticipated barriers and was obtained within weeks to 3 months for all sites. The results presented here are based on events that occurred during the first and second annual reviews at the 19 IRBs.
Results
Expiring Consent Forms
At the initial review, 13 of the 19 IRBs required us to use consent forms that included a site-specific approval stamp. These 13 IRBs covered approval for 25 practices and the AAFP investigators. In most of the cases the stamp included the date of approval and the name of the IRB. The other sites did not require any specific stamps for the consent forms.
At the time of the first annual review we were told that those 25 sites with stamped and dated consent forms would need to have all consent forms replaced each year with newly stamped and dated consent forms and that all expired unused consent forms had to be collected. Women could only be enrolled on a form for which the enrollment date was included between the approval and expiration date. At the second annual IRB review, 5 additional sites added an approval and expiration date so that 30 of the 31 practice sites required new consent forms after the second year.
This requirement had several implications. First, new English and Spanish consent forms had to be reprinted each year and sent to each site before the expiration date on the previous year's consent forms. All unused consent forms had to be returned to the central site so that we could confirm their destruction. Some IRBs were prompt in sending us the stamped and dated consent forms immediately after the yearly approval, but none sent them more than 1 week before the date on the new consent forms. Other sites did not mail the newly stamped and dated copies to us until after expiration of the old consent forms. Urgent requests to the IRB secretaries were addressed with quick responses (delay of only a few days) by several of the IRBs. However, at 4 sites the delays were 2 weeks, 5 weeks, 8 weeks, and 13 weeks, respectively, meaning that study enrollment had to stop for prolonged periods while we waited for new consent forms. This type of disruption in enrollment is particularly damaging when the study is designed to integrate a practice change into PBRN practices.
To account for the delays encountered at the first annual review related to slow return of the new dated and stamped forms from local IRBs, the second annual reviews were submitted 2 to 3 months before the expiration date on the consent form. This resulted in timely approval of the study but continued slow receipt of the stamped consent forms, with 2 sites having no properly dated consent forms for periods of 1 to 9 weeks. In addition, the second 12-month approval cycle was effective as of the new approval date, which was generally several months before the 2-year expiration date. The implication was that the approval period was not 12 months from the end of the previous year approval but now only 9 to 10 months. During the course of a 5-year study this can result in an extra review cycle, which is time consuming and expensive.
At the time of the study and budget development, we were unaware of the need to print and mail new English and Spanish consent forms to each site each year. We had not had such a requirement for previous PBRN studies. The importance of not interrupting or further delaying patient enrollment led us to use overnight express delivery mail to send the new materials to 29 of the sites (Table 1). The new materials were not only 50 to 100 new consent forms but rather 50 to 100 new full enrollment packets. To facilitate the linking of all study data, Translating Research Into Practice for Postpartum Depression consent forms and study packets include bar code stickers that link the consent forms to the study surveys. This assures that we can link data across survey and consent forms even if the woman's name is not legible on any of the forms. Although this was critical for good data management, it meant that we needed to send each site not only new consent forms with appropriate identifying bar codes but also new survey packets with corresponding bar code numbers. If this study were an academic center-based study in which each site had a paid study coordinator, this might not be necessary because the study coordinator could replace the old consent forms in existing packets. However, for PBRN studies, which work to integrate research into practice, sites do not have the personnel to do this tedious paper work. The need to replace the consent forms, often in a short period of time, required significant central site staff time. By the time we explained the process to each site, copied and collated new consent forms and enrollment survey packets specifically for each site in English and Spanish, mailed them, and followed up on the change of consent form, the central staff spent approximately 6 hours per site per year, or approximately 180 hours per year or 360 hours for all 3 years of patient enrollment.
For the practices that seldom participate in research studies, the need to change consent forms was confusing. The practices were often slow in responding to our request to change forms because they knew the consent form had not changed and did not see any logic of replacing “perfectly good forms.” Changing the color of the consent form from one year to the next might have simplified this activity but several IRBs and practices required that consent forms be printed on white paper. The practices needed white paper to improve scanning quality for placement in an electronic medical record or for filing in paper medical records, which required all visit-related information to be on white paper. To simplify the process within the practices, we attached a colored cover page using a different color for each yearly update of the consent form and survey packets. This page was attached to remind the person providing consent to use the correct forms (for example, red cover sheet for 2006, yellow for 2007, and blue for 2008).
The date on the consent form lead to another area of confusion for some of the IRBs. Three IRB chairs asked that a woman sign a second consent form if her participation extended beyond the date stamped on the consent form. Larger IRBs were clear that the consent was valid for the life of the study but at least these 3 smaller IRBs did not understand the meaning and implication of a dated stamp. We were able to work with those IRBs to eventually achieve agreement that any signed consent was valid for the duration of the study as described within that consent form. The fact that some IRBs did not understand the intent of a time-limited stamp on a consent form suggests that they also did not understand the stamp's possible role in protecting human subjects or perhaps even the need for the stamped date.
When contacted, most IRB secretaries and chairs did not understand our concerns regarding the yearly date stamp. University IRBs’ primary experience with “community” patients was bringing patients from community primary care practices into the university setting or sending full-time research personnel into a community practice to do a study. In either situation, the switching of consent forms would be done by full-time research staff, not the practice staff. In addition, it would not be necessary to return old forms and deliver new forms by mail. Our explanation of how we were conducting our study was considered an unusual case and not worthy of modifying IRB practices. When we asked how the processes improved human subject protection, IRB staff expressed concerns about missing consent modifications and desired assurance that the most recent consent form was being used. Although discussions were polite, no IRB was willing to consider alternative strategies.
Repeating Human Subjects Training
The second issue affecting our PBRN study was the requirement by 8 IRBs, including the one that covered the 13 unaffiliated practices, for all key study personnel to repeat human subjects training. The time for retraining was every 12 months for 2 of the IRBs and every 24 months for the other 6 IRBs. Each IRB also specified which human subjects training materials had to be used; the online CITI training program was the most commonly required. Other sties required programs developed by their institution or one of the medical schools within their state. Retraining was to be done by key personnel, but the definition of “key personnel” also varied between IRBs. The lead study physician and study coordinator were always considered “key personnel,” but other personnel sometimes included all staff members who provided the informed consent or all personnel who implemented the study intervention, which basically included the entire practice staff.
The requirement led to human subjects retraining for a total of 92 people. The burden of this retraining fell on the practices, requiring an average of 90 minutes for each person doing the retraining. In a few cases the CITI program required as much as 4 hours to complete. We estimate that retraining to date has required approximately 200 hours of unreimbursed physician and nursing staff time. For some of the sites a second episode of retraining will be required, adding further unreimbursed costs.
For all IRBs, the required retraining materials were exactly the same as the initial training materials and, in several instances, the test questions at the end of training had not changed from the previous post-training test. The requirement of repeated human subjects training resulted in one site being told they would have to stop enrollment while a staff member who had been on sick leave completed her human subjects retraining.
Discussion
The primary role of an IRB is to protect human subjects from harm and exploitation. This is a crucial oversight role that benefits researchers as well as study participants.18 Although IRBs must adhere to certain federal regulations, local IRBs are allowed significant flexibility in interpreting those regulations and designing specific implementation standards. Because of the significant implications and impact of local IRBs’ requirements it seems that is it time to ask IRBs to provide evidence to support their specific requirements; in particular, those requirements that do not seem to directly lower the risk for any participating human subject should be supported with evidence of need and added value.
All IRBs with multiwide assurance need to meet some specific federal regulations to be granted this status.19 However, in addition to federal regulations, IRB personnel often attend regional or national meetings20 where they discuss nonmandatory “recommendations” that might improve IRB functions. Many of these recommendations seem to relate to “discussions or verbal suggestions” rather than written regulations of the Office of Human Research Protections (OHRP) or to National Institutes of Health policy. Therefore, it is very difficult for any investigator to know what new recommendations may become local IRB requirements or how to plan ahead for the additional work new nonmandated local requirements may require. The burden is amplified in a national study dealing with multiple IRBs when the requirements vary by IRB and from year to year and are designed to provide protection primarily to participants in phase 2 or 3 drug and device trials.
Most clinical researchers, including those doing PBRN research, are convinced that evidence-based practice is a large step forward and are eager to expand the concept of evidence-based versus eminence-based guidance for all health care activities. There appears to be no literature providing evidence to support the 2 IRB requirements we present here. To date the OHRP has not provided specific guidelines concerning human subjects retraining. On July 1, 2008, OHRP announced in the Federal Register that it is seeking comments about its requirements for human subject training for investigators and IRB members.21 The only information found that might pertain to the required frequency of re-education states that “[t]he intent of the education requirement is for investigators to keep abreast of development in human subjects’ protection.”22 The requirements of the IRBs discussed here for retraining yearly or every other year using exactly the same materials does not seem to meet the stated objective of “keeping abreast of developments.”
No publication could be found that showed a decline in awareness of or correct application of human subject principles after a period of 2 or 3 years.23 The requirement for practicing physicians and nursing staff to complete exactly the same human subjects training course every 2 years does not seem to be based on any evidence. New versions of human subjects training should be developed that use examples more appropriate for translational research, which might increase the PBRN practice staff's perceived value of this training. A shortened update or reassessment, as developed by some IRBs, may also be more appropriate than requiring complete retraining every 1 to 3 years. As part of the programs’ goals to encourage PBRN research and phase 3 translational research,24 the Office of the Director of the National Institutes of Health and the Agency for Health Care Research and Quality should facilitate studies about the appropriate interval and content of human subjects training for staff engaged in community practice-based research, including the Clinical and Translational Science Awards research programs.
No information about the importance of dated consent forms that expire after 12 months was found in the literature or on the National Institutes of Health or OHRP Web sites. Dating consent forms does not seem to be the only or, perhaps, the best solution to ensure all changes are included in the form used. If a consent form changes in the middle of an approval period the dating process is not helpful. Using a different method of identifying the exact form being used by labeling it as “version 1” or “version 2” would assure that the latest version is used and not penalize those studies that do not change the consent form during their study period and in their study sites (in our case, 5 years and 31 sites). An alternative approach might include searching the updated protocol and protocol modifications at each annual IRB review, making sure any protocol changes are reflected in the consent form when warranted. The new consent form could then be stamped as the next version. The requirement of simply adding an annual time stamp does nothing to protect against unreported intrayear protocol changes and therefore does not seem to improve the protection of human subjects.25
IRBs appropriately require that each potential research subject be presented with full information regarding the risks and the benefits of participating in the study. It would be appropriate for IRBs to use this as a framework to evaluate the risks and benefits of the requirements they place on practices and investigators. This is of special importance in PBRN and translational community-based research, where many of the investigators are practicing physicians who add this activity on to full-time practice. We believe that the benefits of the 2 requirements discussed in this article may not be worth the risks. Such requirement could decrease access to community-based practices as sites for translational clinical research, thereby defeating the goals of the National Institutes of Health Roadmap project.26
Conclusions
IRBs perform a very important service to the public, human subjects, and researchers. However, the implementation and translation of the protection into specific requirements for investigators should be based on benefits, risks, and burdens found in studies that provide high-quality evidence. When higher levels of evidence do not exist, the expert opinions that are the basis of the IRB decisions must consider the cost of their requirements for investigators, practices, and subjects against the potential for improving human subject protection. For PBRN studies, the impact of some current requirements may be sufficiently onerous as to stop or decrease practices’ willingness to participate in the community-based research that will provide the generalizable information we need to care for the majority of Americans.
Notes
This article was externally peer reviewed.
Funding: Funding provided by R01 HS014744-01 from the Agency for Healthcare Research and Quality.
Conflict of interest: The authors have no conflicts of interest, including specific financial interests and relationships and affiliations relevant to the subject matter or materials discussed in the manuscript.
See Related Commentary on page 348.
- Received for publication August 8, 2008.
- Revision received October 13, 2008.
- Accepted for publication October 21, 2008.