Abstract
Objective: To present the survey methods and instruments used to characterize a geographically and professionally diverse rural practice-based research network (PBRN).
Methods: A cross-sectional study of Oregon PBRN (ORPRN) member practices and clinicians using a 3-part survey including a survey of the practices, of clinician members, and an anonymous survey of clinician satisfaction.
Results: A total of 31 of 32 (97%) participating ORPRN practices completed the practice survey, 96 of 129 (74%) clinicians within these practices completed the clinician member survey, and 81 of 129 (63%) clinicians completed an anonymous survey of clinician satisfaction.
The survey provided a detailed description of the structure of member practices, patient and clinician demographics, services provided by the practices, and access to specialty and ancillary services.
Conclusions: Survey tools that describe the network practices and individual clinician characteristics contribute to an understanding of the research capacity of an individual PBRN.
A primary care PBRN represents a group of primary care clinicians in multiple practice sites collaborating with each other and with research investigators to study health care issues of mutual interest. The history and role of PBRNs in primary care research are well described.1,2 Over the past 12 years, the growth of PBRNs has been substantial.
In 1994, there were 28 active primary care PBRNs in North America.3 The growth in the number and type of primary care networks began when, in 1999, US government legislation (Public Law 106-129) directed the Agency for Healthcare Research and Quality (AHRQ) to link research to clinical practice by including the use of PBRNs in primary care. The ORPRN was 1 of 36 PBRNs awarded a developmental grant in 2002 (available at www.ahrq.gov/research/pbrnproj.htm). The growth of PBRNs has continued, and as of July 2006, 94 primary care research networks are registered with AHRQ’s PBRN Resource Center located at Indiana University. The Federation of Practice-Based Research Networks (FPBRN) was established in 1997 by the American Academy of Family Physicians to stimulate the growth and development of research at the primary care practice level and to encourage communication among networks. The FPBRN July 2006 Network Inventory lists 49 member networks. Of these 49 networks, 5 are national, 21 are statewide, and 23 are regional.4
PBRNs provide a community-based setting for research studies and ideas. Primary care PBRNs have an important role in providing information about the burden of illness in the community, and the burden of chronic care management and disease prevention on practices in the community. The National Institute of Health’s Roadmap Initiative recognizes PBRNs as a tool to overcome the roadblock of translating efficacy and effectiveness studies into action at the community practice level.5 The Roadmap Initiative recently funded a program of Clinical and Translational Science Awards (CTSAs) to transform clinical and translational research. The 12 academic health centers receiving first round CTSA implementation awards describe strong collaborations with community partners—clinicians, practices, and organizations with PBRNs as a key element of infrastructure.
When considering a PBRN as a setting for research, it is important to understand the specific settings in which the research will take place. For clinical research, it is important to have information about the patient populations served by the practices. For health services research, it is important to know about the practices and clinicians in addition to the patient population. Unfortunately, it has not been the standard in PBRN research to gather comprehensive data about participating practices. For those that have gathered such data, there has been no effort to standardize the approach across networks, nor has the quality of collected data been scrutinized.
It was the goal of ORPRN to conduct a survey of its members to develop a comprehensive descriptive database of participating practices, clinicians, and patient populations. This study reports on the initial development of the database; subsequent waves of the survey will allow investigators to track trends over time among the practices, clinicians, and patients who collectively make up ORPRN. Further, we sought to understand the process by which practices obtained the requested data, and how they would use their own data systems to identify hypothetical patient cohorts. This information is critical for helping ORPRN members move toward increasingly valid and reliable methods of working as a part of the network over time. The objectives of this study were to determine member practices’ willingness and ability to gather and share descriptive data and to evaluate our methods of data collection in an effort to begin to move toward a more standardized approach to describing PBRNs.
Methods
Completion of the practice and clinician surveys is a requirement for membership as defined by the ORPRN bylaws. We developed a survey for clinicians and practices to be implemented every 2 years through interview and self-report.
Survey Development
ORPRN′s survey was designed to meet 5 objectives:
To describe the characteristics of clinicians and practices participating in the network.
To understand the structure, business operations, financial support, and stability of the practices.
To identify the information technology use and research capacity of the practices and clinicians.
To identify gaps in the network’s composition of practices and clinicians to direct future recruitment.
To connect in a meaningful way with the practices and clinicians, providing them perspective on how they compare with the overall network.
In constructing the survey instrument, key factors included the time and effort that the practice staff and clinicians were willing to commit to the study and identifying information valuable to clinician members and the research network. The 10 rural clinician members of the ORPRN Steering Committee were consulted to incorporate these considerations into the survey tool. A literature review of the methods used to describe other networks (clinicians, practices, and patients) found considerable variation ranging from the initial age-sex patient registry used by the Ambulatory Sentinel Practices Network (ASPN),6 to the Primary Care Network Survey conducted by 20 AHRQ-supported PBRNs in 2004,7 to the work done by Stange and colleagues8 to describe the context and content of family practice in their work on how practices provide primary health care in the family practice office.
AHRQ developed the Primary Care Network Survey (PRINS) tools including PRINS-1, a 22-item self-reported questionnaire to assess demographics of clinicians, patient volume, business relationships, practice setting, and laboratory testing; and PRINS-2 a 19-item tool designed to characterize the clinician-patient visit.7 The PRINS tools and the National Ambulatory Medical Care Survey (NAMCS) describe individual clinicians rather than characterizing the practice. In addition, NAMCS does not include nonphysician clinicians in their surveys.9
With input from clinician network members, ORPRN investigators developed 3 instruments: a practice survey, a clinician survey, and an anonymous clinician satisfaction survey (see appendices for the surveys). To allow for national comparison, a subset of questions from PRINS-1 and the Community Tracking Study (CTS) Physician Survey, from the Center for Studying Health System Change, were included. The CTS physician survey is a longitudinal project designed to provide a perspective on the health care delivery system. The questions address sources of revenue, quality of care, information technology, practice ownership, access to services, and career satisfaction.10 In addition, we used items from the Oregon Medical Association physician survey11 and the Oregon Health Workforce Project survey12 to allow network comparisons with statewide data.
The survey instrument designed to assess the practices (Appendix 1) consisted of 21 items separated into 5 domains: practice ownership and history, use of information technology, demographics of the practice panel, staffing patterns, and several research scenarios designed to illustrate practice patient data management familiarity and capacity. The data scenarios addressed the practice’s ability to identify unique patients and specified cohorts of patients.
A second survey of clinicians (Appendix 2) included 32 items separated into 6 domains: clinician demographics, practice description, scope of practice, availability of services in the community, technology utilization and attitudes, and current quality improvement and research activities and interests.
A final survey (Appendix 3) was designed to collect data anonymously from clinicians, to allow unbiased assessments of clinician opinions and attitudes. This last survey consisted of 5 items, including overall satisfaction with medical practice, retention plans, control of practice income and operations, practice attributes influencing quality of care, and net income.
Eligibility
The criteria for membership in ORPRN include practice location in rural Oregon, participation in a prior ORPRN research study, or an expressed an interest to participate in a future study. ORPRN uses the Oregon Office of Rural Health definition of rural, which includes areas at least 10 miles away from a population center of 30,000 or greater. Only practices and clinicians that fit the ORPRN criteria for membership were invited to participate in these surveys.
Clinician members of ORPRN must be primary care practitioners, including family physicians, general internists, or pediatricians; family, adult, or pediatric nurse practitioners; or physician assistants. Clinicians who do not actively see patients and physicians-in-training (residents) were excluded.
A total of 32 practices and 129 clinicians fit the definitions of ORPRN membership; Figure 1 displays the locations of ORPRN practices.
Map of the locations of member practices of the Oregon Rural Practice-based Research Network (ORPRN).
Data Collection
All 3 surveys were administered on paper and were distributed in August 2005 to ORPRN′s Practice Enhancement and Research Coordinators (PERCs). PERCs are regional research assistants who live in rural Oregon, each covering a geographic portion of the statewide network. Between August and December 2005, the PERCs traveled to each practice to meet with a practice representative, usually the practice manager or administrator, to discuss the survey project and interview this representative to gather information for the practice survey. The PERC mailed the practice and clinician surveys to the practice representative before the visit and encouraged them to look at the questions before the interview. The practice survey was completed during the interview. In approximately half of the practices, the site representative had collected required survey information before the interview.
In addition, the PERCs asked the practice representative to distribute and collect completed surveys for each ORPRN clinician in the practice. This representative also distributed the anonymous survey of clinicians along with a stamped-addressed envelope; these surveys were sent directly by the clinician to the ORPRN office at Oregon Health and Science University when complete.
A follow-up electronic query of the site representatives was conducted regarding the source of data for the number of active patients, payer mix, and patient characteristics and responding to the data scenarios.
Data Management and Statistical Analysis
All surveys were reviewed for completeness and data were entered into an Access database for analysis; data were cross-checked by a second person for accuracy. Descriptive statistics were generated for all data.
Results
Thirty-one of 32 (97%) practices completed the practice member survey. In almost all the practices this survey was completed by a person identified as the practice manager or administrator. Ninety-six of 129 primary care clinicians (74%) completed the survey of clinician members. Fifteen clinicians refused, and 18 did not respond despite 2 requests to complete the survey. Eighty-one of 129 clinicians (63%) returned the anonymous survey of clinicians.
The survey data provided a detailed description of the network, characterized at the practice, clinician, and patient level (see Tables ⇓⇓⇓⇓⇓in Appendix 4).
Characteristics of the ORPRN Practices (N = 31 practices)
Characteristics of Patients in ORPRN Practices (N = 31 practices; 166,003 patients)
Characteristics of ORPRN Clinicians (N = 96)
Services Provided by ORPRN Clinicians
ORPRN Clinicians’ Report of Availability of Services to Their Patients (n = 96)
The data scenario responses were free-text. All 31 practices responded with 30 of the practices listing billing software as their source of information for the scenarios. One practice listed the electronic health record as their sole data source. Twenty-eight of the 31 practices responded to the question “Does your practice have any disease registries?” Of these respondents, 32% (9 of 28) reported having at least 1 disease registry.
The follow-up e-mail inquiry of the practice managers revealed that multiple methods of data collection were used. A best-estimate method was used by approximately three fourths of the practices to generate the responses regarding the number of active patients; whereas, billing software data provided responses regarding patient ages and payer mix in approximately two thirds of the practices. The electronic inquiry asked for the specific billing software used by practices. Thirteen different software products were named with 3 practices not specifying a brand of software [OCHIN (3), Medical Manager (5), Medisoft (5), Next Gen (5), Healthco (2), McKesson, Cerner, MARS, Meditech, Clinic Pro, Misys Tiger, Practice Partner, and Centricity].
Discussion
A robust description of the practices and clinicians in a PBRN provides a number of important benefits. These benefits include: (1) describing the network for grant proposals, academic and government partners, communities, and patients; (2) understanding the structure and capacity of member practices for research; and (3) connecting with member practices and clinicians.
The results of the survey of ORPRN clinicians and practices have been included in the ORPRN newsletter and several recent grant submissions, including the institution’s successful National Institutes of Health Clinical and Translational Science Award proposal. Organizational partners, such as the statewide quality improvement organization and the state immunization program office have used ORPRN survey data in seeking funding for collaborative projects. Network staff, including the PERCs and network director, have presented the data to rural Oregon practices, hospitals, and community organizations.
The survey tools and methods that we used were acceptable to the practice administrators and clinicians, resulting in a high completion rate. Rather than being perceived as burdensome, the ORPRN survey has largely been viewed positively, having meaningfully engaged practice administrators and clinicians in the network, and providing data back to them about other rural practices in our geographically diverse state. Often PBRN studies involve a small subset of clinicians or practices and rarely does the entire membership of the network participate in a given study. Clinicians are interested in how they and their practices compare with others in the network. Timely feedback is important in maintaining clinician interest in the network. Practice and clinician-specific reports are delivered to the practices by the responsible PERC. Local meetings of the primary care clinicians are scheduled to examine the context of the ORPRN survey data combined with community health data from the Oregon Office of Rural Health. It is anticipated that these meetings will create a better understanding of important health areas for research and quality improvement.
A limitation of using self-reported survey tools such as PRINS, NAMCS, and the ORPRN member survey is that practices and clinicians used a variety of methods to report data on patient visits, insurance coverage, and patient age-breakdown, and a validation of these numbers has not occurred. Validating the data in a subset of practices would be helpful.
The utility of the data collected need not be limited to cross sectional assessments. For example, ORPRN will conduct the practice and clinician surveys every 2 years, allowing for a longitudinal measure of trends in the rural health care market, as well as an assessment of the network’s success in adding communities, practices, and clinicians so as to be maximally generalizable to the rural health care environment.
The ORPRN survey method provides a practice context as well as the clinician context in describing the settings for primary care practice-based research. We offer our survey instruments for use by other networks to facilitate cross-network comparisons (see appendices).
Appendix 1. Clinical Practice Member Characterization Survey
Appendix 2. Clinician Characterization Survey
Appendix 4. Practice and member characteristics of the Oregon Rural Practice-based Research Network (ORPRN)
Acknowledgments
We acknowledge the contributions of ORPRN PERCs: Ann Ford, Monica Goubaud, and Julie Reynolds. We also thank the 31 ORPRN Practices and their clinicians: Baker Clinic, Eastern Oregon Medical Associates, Elgin Family Health Center, Union Family Health Center, Dunes Family Health Center, Lincoln City Medical Center, Strawberry Wilderness Family Clinic, Robert E. Holland, Russel J. Nichols, Winding Waters Clinic, Pine Eagle Clinic, La Clinica del Carino Family Health Center, Klamath Open Door Family Medicine, Cascades East Family Practice Residency, OHSU Family Medicine-Scappoose, Yachats Community Health Clinic, Gilliam County Medical Clinic, Columbia Hills Family Medicine, High Desert Medical Center, Rinehart Clinic, Bayshore Family Medicine, Treasure Valley Pediatric Clinic, North Bend Medical Center, Columbia River Community Health Services, Asher Community Health Center, Thomas H. Hodge, Maria Bolanos-McClain and Ken McClain, Madras Medical Group, Lake County Medical Clinic, C. Scott Graham, North Lake Clinic.
Notes
This article was externally peer reviewed.
Funding: Oregon Health and Science University through the Oregon Opportunity.
Prior presentation: This article is based on a presentation made at the 2006 Agency for Healthcare Research and Quality National Practice-based Research Network Conference, Bethesda, MD, May 15–17, 2006.
Conflict of interest: none declared.
- Received for publication August 16, 2006.
- Revision received December 21, 2006.
- Accepted for publication January 2, 2007.
Jump to section
- Article
- Abstract
- Methods
- Results
- Discussion
- Appendix 1. Clinical Practice Member Characterization Survey
- Appendix 2. Clinician Characterization Survey
- Appendix 3. Anonymous Satisfaction Survey of Clinician Members
- Appendix 4. Practice and member characteristics of the Oregon Rural Practice-based Research Network (ORPRN)
- Acknowledgments
- Notes
- References
- Figures & Data
- Info & Metrics
- References
Related Articles
Cited By...
- Project ECHO Integrated Within the Oregon Rural Practice-based Research Network (ORPRN)
- The Role of Health Extension in Practice Transformation and Community Health Improvement: Lessons From 5 Case Studies
- A Randomized Trial of High-Value Change Using Practice Facilitation
- Practice Benefit from Participating in a Practice-based Research Network Study of Postpartum Depression: A National Research Network (NRN) Report
- Unmet Dental Needs in Rural Primary Care: A Clinic-, Community-, and Practice-based Research Network Collaborative
- Increasing the Role of Practice Networks in Medical Research