Research ArticleOriginal Research

Patient-Centered Research Priorities: A Mixed-Methods Approach from the Colorado Children's Outcomes Network (COCONet)

Sarah E. Brewer, Natalie M. Crump and Sean T. O'Leary
The Journal of the American Board of Family Medicine September 2019, 32 (5) 674-684; DOI: https://doi.org/10.3122/jabfm.2019.05.190028

Abstract

Background: Practice-based research networks (PBRNs) perform clinically relevant research designed for immediate translation to patient care. Research questions developed with patients and parents are more likely to be relevant to stakeholders. This case study developed priority areas for patient-centered outcomes research in pediatric health within the context of the Colorado Children's Outcomes Network, a statewide pediatric PBRN, and in collaboration with stakeholders.

Methods: We undertook a mixed-methods, community-engaged process to understand and develop a research agenda for our pediatric PBRN. With a stakeholder-engaged research team, we conducted 52 stakeholder interviews and 1 focus group (n = 9) addressing topics including child health issues, personal health care experiences, community health, and the health care system. Transcripts were coded and analyzed by researchers and parent advisors. We conducted a Web-based survey of PBRN stakeholders (n = 75) to rate priority issues identified in qualitative findings. Finally, we facilitated a community-engaged mixed-methods interpretation and issue selection process with our Network Advisory Board to identify issues of highest importance for Colorado Children's Outcomes Network.

Results: Six topic areas of importance to pediatric health stakeholders emerged from qualitative work. Participating stakeholders used rating surveys to rank these 6 topics. Pediatric mental health coordination and communication/integration with primary care was ranked as the most important and highest research priority. The Network Advisory Board additionally identified immunization uptake and transition from pediatric to adult health care as research priority areas.

Conclusions: This pediatric PBRN identified numerous research priorities in pediatric health and selected 3 for immediate research action. PBRNs can use community-engaged, mixed-methods research approaches to set research priorities and develop patient-centered pediatric research agendas.

Participatory approaches to conducting research are crucial to ensuring relevance to stakeholders, addressing issues of equity, and increasing patient-centeredness.1⇓–3 Community-based participatory research (CBPR) is one such approach that equitably involves community members, organization representatives, and researchers in all aspects of the research process.1,4,5 CBPR starts with a topic of mutual importance to the community and researchers with the aim of ultimately improving community health.6 CBPR has been shown to be more relevant, culturally proficient, and effective than research-driven research.4,7⇓–9 The time is ripe for the principles of CBPR and community engagement to be more broadly applied to child health research, especially in practice settings.

Practice-based research networks (PBRNs) perform clinically relevant research designed to be immediately translatable to patient care. While PBRNs have great capacity for clinician engagement, few effectively engage patients (and parents in the case of pediatric patients) in the development of research questions. Many PBRNs have established patient advisory boards; however, few PBRNs use patient advisory boards specifically in the development of research questions and aims, and we are aware of none focused specifically on issues related to children. This poses a particular challenge for pediatric and family medicine practices where both patients and parents should ideally be engaged in meaningful conversations with providers and researchers about their most pressing health care questions.

PBRNs have been instrumental in developing and translating evidence-based care, particularly in Colorado.10 The Ambulatory Sentinel Practice Network broke ground in Colorado and nationally in the 1980s with many important studies addressing primary care topics.11⇓–13 PBRNs also play an essential role in generating evidence and understanding of how to disseminate and implement findings in real-world settings.14 While Colorado has been on the forefront of using CBPR principles within its PBRNs,15 these efforts have focused primarily on adults. Child health research differs from research on adults in many important ways, including the focus on wellness and development, and the involvement of parents and families in the health care of children.

We sought to develop a model for engaging patients, parents, clinicians, and other stakeholders in the collaborative development of research questions and patient-centered research relevant to child health in the PBRN setting. We build on existing PBRN patient engagement models15⇓⇓⇓–19 by creating a patient-driven model for the development of research agendas and research questions specific to pediatric health. In this article we seek to document and describe a process of applying CBPR principles for the purposes of informing a PBRN′s research priorities.

Patients and Methods

The objective of this case study was to develop priority areas for patient-centered outcomes research (PCOR) related to child health issues within the context of a statewide pediatric PBRN in collaboration with parents, clinicians, and other stakeholders. Our intention was to select 2 or 3 topics around which to develop working groups tasked with developing research questions and pursuing research funding. Using a sequential mixed-methods approach (Figure 1),20,21 this project sought to build a stronger understanding of parent and patient concerns in pediatric health, and to prioritize important topics and questions with input from all stakeholders. Our stakeholder-engaged research team consisted of the Director and Associate Director of the Colorado Children's Outcome Network (COCONet) and 3 parent advisors whose children receive health care at member practices. This inclusion of parent advisors from the beginning of the prioritization process ensured stakeholder engagement and guidance throughout all phases of the project, including research question development, interview guide and survey instrument development, and analysis at all phases of the mixed-methods study. The team discussed all aspects of the project design, implementation, and analysis at regular meetings.

Figure 1.
Figure 1.

Overview of study design and prioritization process.

Setting

COCONet is a statewide network of primary care pediatric practices in Colorado. The network was loosely established in the early 2000s and formalized into a PBRN in 2011. Infrastructure for the network is housed at the University of Colorado–Anschutz Medical Campus. All study activities were approved the Colorado Multiple Institutional Review Board.

Participants

Interview participants included parents, adolescent patients, clinicians, academicians, policy makers, and community stakeholders. Participants were identified through a combination of purposive sampling among existing COCONet stakeholders and a snowball recruitment strategy in which stakeholders referred the study team to other potential participants.22,23 Recruitment efforts were conducted in partnership private pediatric practices, a family engagement training program, and a local Family Voices chapter. In addition, we asked stakeholders at these organizations and in each interview: “Who else should we talk to who will tell us something different?” to intentionally seek out stakeholders with differing views, experiences, and perspectives. The sample for the rating survey included a broader group of COCONet stakeholders. Rating survey participants included all participants who agreed to further contact from COCONet staff (42 of 61), and other parents, providers, and stakeholders referred by interviewees, parent advisors, and colleagues.

Data Collection

To identify a broad range of issues, we first conducted interviews. Interviews were semistructured conversations about a set of pediatric health topics. We primarily used a 1-on-1 semistructured interview format with parents and other stakeholders. Parent interviews were conducted in English or Spanish. The interview guide was developed in English and translated independently. Spanish interviews were conducted by a trained bilingual interviewer. All interviews were conducted in the location of the participant's choosing, including professional offices, clinics, home, or by phone, to accommodate geography and varied schedules.

We initially conducted 2 interviews with adolescents but elected for a focus group format instead after initial interviews showed that these adolescents tended to focus on a single health topic they found interesting. The focus group was conducted with a statewide youth health advisory group (n = 9). One researcher and 1 trained parent advisor facilitated the focus group.

We created 2 conversation guides that ensured similarity of topics covered across interviews. Guides were structured to elicit information from parents and adolescents in 4 categories: 1) their or their child's experiences in the pediatric health care system, 2) their or their child's experiences with health in their community, 3) the questions they have had about their/their child's health and their satisfaction with the answers to those questions, and 4) their perspectives on fifteen health issues raised in our PBRN by clinicians or researchers. Questions in categories 1 and 2 were structured to specifically elicit topics of interest and importance to them and probing questions were used to understand whether these topics were also areas of research interest. Category 3 questions focused on other areas of research interest and unanswered questions about health or health care systems. A second guide for providers and stakeholders queried professional perspectives on these same issues with a similar layout. Many providers and stakeholders were also parents and we probed their perspectives in both roles.

We decided to both keep questions open ended and have a list of pediatric health topics to which we would ask people to react. We queried on the structured list of issues to collect feedback from all participants about health issues commonly raised in our PBRN and assess the relevance to parents of the issues on which our PBRN had previously applied focus. We ordered the conversation from the most exploratory to most structured to limit the influence of the interviewer raising specific topics on parents' responses and ensure our findings were not influenced by our existing PBRN topical foci. Detailed notes and interviewer debrief memos were collected for each interview and focus group.

Following analysis of qualitative data, we conducted a stakeholder rating survey to ascertain the importance of each of 6 topics identified from the stakeholder perspective. The survey was 8 pages: 1 for each issue, 1 of limited demographic information, and 1 asking the stakeholder to report their interest in working with our PBRN to address these topics. The parent advisors heavily influenced the survey design to be in a format that was easy for stakeholders to understand, respond to, and identify the limited criteria needed to rate the issues chosen from our narrowed scope. The survey was introduced with a summary of the purpose of this project and some language distinguishing issues of interest (problems we have solutions for but need funding or will to implement, common health problems like asthma) from issues in need of research (unanswered questions about treatment, prevention or health care delivery systems). We also took an inclusive view that research such that it could include implementation research and quality improvement to address service improvement issues raised by stakeholders. Then, each issue page included 3 components: 1) a summary of the issue as it was described by stakeholders in the interviews; 2) a summary of the gaps in research about that issue, or research questions as defined by the stakeholders in interviews; and 3) 2 questions asking the stakeholder to rate the importance of the topic on a 0 to 10 scale for a) its overall importance, and b) the importance of researching that issue. The survey was reviewed by 2 survey methods experts, pretested in 3 waves by naïve users, colleagues and the research team (including the parent advisors), and iteratively revised. Surveys were distributed by email using an online platform (Qualtrics) and stakeholders received up to 4 reminders to complete the survey in line with online survey best practices.24 Based on stakeholder interviews and parent advisor input, the surveys were conducted only in English.

Interview and Focus Group Analysis

This study took a phenomenological approach to data analysis.25 Interview and focus group recordings were transcribed verbatim. Spanish-language interviews were then translated to English for analysis. Coding was conducted by a trained qualitative researcher (SEB) and 1 of the parent advisors (NC). Coding and analysis were completed using ATLAS.ti 7.0 (Berlin, Germany).

An initial codebook included the predetermined issues we asked about in the interviews, including aspects of the health care system, the relationship between patients and physicians, and specific health issues and diagnoses. Second, we reviewed the debrief forms to identify themes that emerged as the interviews were conducted. These were added to the codebook when there was agreement on the research team. We also added codes to our codebook as they emerged from the transcripts. When the coders identified concepts they deemed important to the process, but that were not reflected in the codebook, they marked them and reviewed them with the team to discuss whether a new code was warranted. Consensus from all team members was required to add emergent codes to the codebook.

Finally, we identified a need to distinguish between issues that were raised with passion and enthusiasm from those mentioned without much emotion. This was important because we saw a pattern of parents raising issues they heard about from the local media, but with little energy or direct experience with those issues. Often later in the conversation a parent would discuss another issue about which they were clearly passionate. We were particularly interested in the issues parents wanted to see more research on that were otherwise not being addressed. Thus, we created a set of codes to indicate the level of enthusiasm or emotion that accompanied the issues raised by our stakeholders.

The analytic team coded data following reflexive team analysis,26,27 emphasizing inclusion of emergent and a priori themes. The team met regularly during coding to ensure consensus and triangulate data. The coding team reported back to the research team regularly. Following coding, we utilized 2 ATLAS.ti tools: 1) the co-occurring codes matrix (which codes overlapped on the same text), and 2) the code-transcript matrix (how many times did each stakeholder raise each issue/code). From the first, we were able to identify which codes might group together. We ultimately decided not to combine codes but rather created “buckets” of codes that might go together. The second assisted us in understanding which codes were raised only by a few individuals and were not issues of concern to the majority of participants. No issues were excluded based only on these reports.

Rating Survey Analysis

Rating survey findings were summarized simply to ensure understandability for all stakeholders. Mean rating scores were reported along with a range of rating for each of the 6 topics. A total count of interested stakeholders willing to participate in future work groups for each topic was created.

Issue Selection

We adapted the James Lind Alliance approach28 to issues prioritization with our Network Advisory Board (NAB),29 consisting of 16 members including parents (n = 14), providers (n = 3), researchers (n = 3), child health advocates (n = 4), and policy makers (n = 1), with many holding multiple roles. The James Lind Alliance has developed extensive methodology for “Priority Setting Partnerships.” We modeled several aspects of our process on those described in the James Lind Alliance Guidebook.30

For our NAB priority setting meeting, we purposely created a loose agenda. While we wanted to keep our group on task, we wanted plenty of time for open-ended discussion. We first laid out some general ground rules for discussion and presented a consensus decision-making model as a framework for the process. PBRN staff provided article copies of the results of the interviews and ranking survey. We then stated the goal of the meeting: to come to a consensus on the topic areas for 3 Working Groups. Staff facilitated a discussion of the criteria by which issues should be evaluated for selection. The first was feasibility: Would we be able to address the research question within our PBRN? Second was the potential impact of research questions and topic areas under consideration: What is the potential, if successful, to improve the lives of children in Colorado? The third criterion was interest level: how much interest was there across the different stakeholder groups that we interviewed, and did parents and patients show interest? For example, if a priority area emerged as moderately important across most stakeholders, that topic may take precedence over a priority area identified as highly important among providers but not parents or patients. Within topics, potential research questions were evaluated under the following framework: 1) Is this a question answerable by research? 2) Is this question unique? 3) Is the answer to this question known? and 4) Is this question relevant to the care of children in Colorado and elsewhere? Research questions were not further refined at this point. The intention was for the NAB to create general research questions for the Working Groups to then fine-tune using the PICO format (patient/problem, intervention, comparator, and outcome).31⇓–33 Finally, the study team committed to speak as little as possible and let the NAB do its work.

Results

Interviews and Focus Groups

Sixty-one stakeholders participated in 52 interviews and 1 focus group (n = 9), including 20 English-speaking parents, 9 Spanish-speaking parents, 12 adolescents, and 12 other stakeholders (eg, policy makers, child health advocates; Table 1). Across groups, 2 types of pediatric health research topics emerged as important: pediatric health issues and health care system issues. In addition, a few broad research questions resonated across stakeholder groups. Themes were similar across languages spoken, age-group, and child health stakeholder group (parents, patients and professionals). Illustrative quotes are displayed in Table 2.

View this table:
Table 1.

Demographic Characteristics of Stakeholders Interviewed about Pediatric Health Research Priorities in the COCONet, 2015 to 2016 (N = 61)

View this table:
Table 2.

Illustrative Quotes from COCONet Stakeholder Interviews on Priority Issues in Pediatric Health

Pediatric Health Issues

Participants described interest in research topics related to discrete health issues including immunizations, mental health, obesity and food and nutrition, sexual health, and drug and substance use/abuse. One issue participants reported in pediatric health was parents' refusal of immunizations for their children. Parents expressed concerns about the ripple effects on their own children. Similarly, providers described frustration with how to communicate the importance of vaccination to parents while maintaining relationships with families in their care. Another issue commonly raised was mental health in the primary care setting. Many stakeholders reported that access and integration was not sufficient to meet the needs of children in the community. Obesity, food, and nutrition were interconnected issues raised by all stakeholder groups. There was particular interest in prevention and physical activity as an intervention modality for combating obesity.

We also heard the importance of potentially sensitive topics in pediatric settings; particularly topics of sexual health and drug/alcohol use. About sexual health, many parents described how they wanted to provide evidence-based information to their children, but were unsure how to do so or were uncomfortable initiating the discussions. Providers, similarly, wanted to ensure this information was available to both parents and children. Some providers described the delicate balance of starting those conversations with adolescent patients in the clinical setting. Adolescents reported a need for more education and access to information about birth control. In addition, drug/alcohol use was raised as an important and timely issue in our PBRN context. Given recent recreational legalization of marijuana in Colorado, addressing children's marijuana access, alcohol use, and access and use of other substances was important to all the stakeholder groups. Adolescents, specifically, raised concerns about a lack of educational materials about less common drugs and wanted more educational programming to focus on the impacts of drugs on the body, rather than simple messages of deterrence.

Health Care System Issues

Across stakeholder groups, issues related to the structure and function of the health care system emerged in terms of the system's impact on pediatric health. We heard about care coordination, patient-provider relationships, provider-to-provider communication, access and availability of care (especially in rural communities), early childhood education and its impact on healthy development, and parent education on child health. These issues raised by all stakeholders spoke to how the systems children interact with were structured to support their health, or not.

Emerging Research Questions

Finally, some of these health and system issues started to coalesce into research questions that could be addressed in a pediatric PBRN setting. Two of the questions we heard parents, adolescents, providers, and other stakeholders circling around across our interviews were: 1) How can parents be better advocates for their child's health in the health care setting? (And teens for their own health?), and 2) How are parents supposed to be effective advocates for their kids?

Among the themes that emerged from this qualitative work, researchers and parent advisors identified 6 research issues raised by parents, adolescents, providers and stakeholders that would be included in the rating survey. These are shown in Table 3.

View this table:
Table 3.

“Top 6” Stakeholder-Identified Research Issues Chosen for Rating Survey

Rating Survey

Of 98 stakeholders invited to complete the rating survey, 75 (76.5%) responded. Respondents self reported that they held multiple roles in child health including 47 identifying as parents, 32 as providers, 22 as advocacy staff/volunteers, 18 as health researchers, 11 adolescents, and 32 in other roles (roles in child health were not mutually exclusive). Among respondents, 80% (N = 60) reported that their children (or themselves, for teens) had private insurance coverage.

Findings from the rating survey, as presented to the NAB, are shown in Table 4. The highest ranked issue on overall importance was pediatric mental health care—coordination, communication, and/or integration with primary care providers (M = 9.24; range, 6.0 to 10.0). This issue was also rated highest for research importance (M = 8.59; range, 0.1 to 10.0).

View this table:
Table 4.

Sample Characteristics and Issue Rankings

NAB Issue Selection

Based on the ranking survey and the knowledge of the people in the room, it was clear that something to do with access to mental health was a top priority. Therefore, it was quickly unanimously agreed that 1 of the Working Groups would focus on pediatric mental health access. Discussion was then devoted to determining the other 2 Working Group topics. At this point, the ranking survey was not as helpful, since pediatric mental health was the only issue that stood out from the other topic areas. Some of the members brought up the concept of taking elements from more than 1 of the identified priority areas to create a single Working Group. An open-ended discussion followed, which led to more focus for the already created Mental Health Working Group (potentially including care coordination), as well as our 2 other Working Groups. It was decided that 1 of these would focus on training parents (or teens) as advocates for care, potentially with an emphasis on transitions of care, and possibly incorporating models to improve the doctor-patient relationship. This could include transitions for children with special health care needs and/or healthy adolescents.

For the final Working Group, addressing vaccine hesitancy was selected based on feasibility: NAB members were aware that vaccine delivery research was a particular strength of COCONet-affiliated researchers. Thus, the NAB decided that immunization delivery was a logical choice for the third priority topic as it could increase the patient-centered perspective in existing PBRN infrastructure and research projects. Across all 3 topics, the decided to leave to Working Groups the decisions to turn these topics into answerable research questions. The final 3 priority areas selected were:

  1. Access to and availability of care including mental health

  2. Immunization refusal and Colorado's low rates

  3. Training Parents (or Teens) as Advocates for Care with Goal of Improved Health Outcomes.

Discussion

Parents, adolescent patients, providers, and other stakeholders in a pediatric PBRN identified a wide variety of pediatric health issues in need of further research in Colorado. The topics areas raised by parents and patients aligned with those identified in national priorities such as Healthy People 202034 and the American Academy of Pediatrics agenda.35 However, while the priorities align with these national agendas, our stakeholders brought personal experiences and perspectives to the table regarding the impact of these issues on their health and the potential solutions to address the issues they raised. Parents and adolescents are also concerned with the patient experience and their role in the health care system including their relationships with their providers and their ability to act as self advocates within complex health care systems.

Findings from this study are being utilized to collaboratively develop research priorities, and specific and fundable research questions. Specifically, COCONet developed Working Groups focused on the 3 topic areas selected by the NAB. Each Working Group includes parents, providers and other stakeholders and is tasked to develop and refine research questions, aims, and methods, and partner with researchers to submit funding proposals addressing these priority areas.

This case study describes a model for developing a pediatric PBRN research agenda in collaboration with a diverse group of stakeholders in pediatric health. This mixed-methods, community-engaged approach to priority setting resulted in a set of research priority areas to guide the work of a statewide pediatric PBRN and a network of engaged stakeholders ready to collaborate to address those issues. We leveraged the existing community engagement expertise in our local PBRN community14,15,18,36 to build partnerships with parents, clinicians, and stakeholders to inform future patient-centered child health research. This approach addresses the need for PCOR in pediatric settings through the existing infrastructure of PBRNs, which can leverage their present patient-centered characteristics for research. Prioritizing in this way builds on the tradition of engagement in PBRNs and bolsters the ability of pediatric PBRNs to conduct PCOR and inform continued patient engagement locally and nationally.

This report should be interpreted in light of its limitations. Our interview sample was majority female across stakeholder groups. Our recruitment efforts uncovered much more interest from women, and specifically from mothers. Since mothers are more often primary decision makers for children's health care,37 we believe our results results reflect children's experience in the health care setting. In addition, our qualitative sample included more stakeholders from urban areas of the state. These differences are in line with the state's population distribution. Finally, our survey was conducted among a convenience sample of stakeholders and is not representative of all pediatric health stakeholders in Colorado. However, these stakeholders were highly engaged and passionate about the PBRN research agenda.

Conclusion

This research identified and prioritized pediatric health research issues, revealing that parents and other stakeholders in children's health have myriad concerns and priorities for pediatric health. Future PCOR in pediatric health should focus on a variety of topics including diagnosis-specific research, health care systems issues, the parent and patient experience, and parent and patient roles in the pediatric health care setting.

This mixed-methods process for engaging stakeholders in health research agenda setting in a PBRN setting is feasible and effective. This approach resulted in an engaged stakeholder network and a clear agenda for future community-engaged research in pediatric health. This replicable process could increase the patient-centeredness of PBRN research across specialties, and especially in pediatric health.

Acknowledgments

We extend our gratitude to Cody Belzley, Sean Crump, Family Voices Colorado, the Colorado Children's Campaign It is About Kids Committee, and our numerous other COCONet stakeholders and pediatric health partners for their assistance in identifying and recruiting parent leaders across the state to engage with our research agenda setting process. We could not have completed this important work to understand research priorities in pediatric settings without their support and assistance throughout this process.

Notes

  • This article was externally peer reviewed.

  • Funding: This project was funded through a Patient-Centered Outcomes Research Institute (PCORI) Eugene Washington Engagement Award (EA-1300-UCD).

  • Conflict of interest: none declared.

  • To see this article online, please go to: http://jabfm.org/content/32/5/674.full.

  • Received for publication January 24, 2019.
  • Revision received June 7, 2019.
  • Accepted for publication June 14, 2019.

References

  1. 1.↵
    1. Israel BA,
    2. Coombe CM,
    3. Cheezum RR,
    4. et al
    . Community-based participatory research: a capacity-building approach for policy advocacy aimed at eliminating health disparities. Am J Public Health 2010;100:2094–2102.
    OpenUrlCrossRefPubMed
  2. 2.↵
    1. Woolf SH,
    2. Zimmerman E,
    3. Haley A,
    4. Krist AH
    . Authentic engagement of patients and communities can transform research, practice, and policy. Health affairs (Project Hope) 2016;35:590–594.
    OpenUrlAbstract/FREE Full Text
  3. 3.↵
    Institute of Medicine: Committee on Educating Public Health Professionals for the 21st Century. Who will keep the public healthy? Educating public health professionals for the 21st century. Washington, DC: National Academies Press; 2003.
  4. 4.↵
    1. Israel BA,
    2. Parker EA,
    3. Rowe Z,
    4. et al
    . Community-based participatory research: lessons learned from the Centers for Children's Environmental Health and Disease Prevention Research. Environ Health Perspect 2005;113:1463–1471.
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Blumenthal DS,
    2. DiClemente RJ
    . Community-based participatory health research: Issues, methods, and translation to practice. Springer Publishing Company; 2013.
  6. 6.↵
    1. Minkler M
    . Community-based research partnerships: challenges and opportunities. J Urban Health 2005;82(2 Suppl 2):ii3–12.
    OpenUrlCrossRefPubMed
  7. 7.↵
    1. Metzler MM,
    2. Higgins DL,
    3. Beeker CG,
    4. et al
    . Addressing urban health in Detroit, New York City, and Seattle through community-based participatory research partnerships. Am J Public Health 2003;93:803–811.
    OpenUrlCrossRefPubMed
  8. 8.↵
    1. Israel BA,
    2. Lichtenstein R,
    3. Lantz P,
    4. et al
    . The Detroit Community-Academic Urban Research Center: development, implementation, and evaluation. J Public Health Manag Pract 2001;7:1–19.
    OpenUrlPubMed
  9. 9.↵
    1. Wallerstein NB,
    2. Duran B
    . Using community-based participatory research to address health disparities. Health Promot Pract 2006;7:312–323.
    OpenUrlCrossRefPubMed
  10. 10.↵
    1. Green LA,
    2. Hickner J
    . A short history of primary care practice-based research networks: from concept to essential research laboratories. J Am Board Fam Med 2006;19:1–10.
    OpenUrlFREE Full Text
  11. 11.↵
    1. Gardner W,
    2. Kelleher KJ,
    3. Wasserman R,
    4. et al
    . Primary care treatment of pediatric psychosocial problems: A study from pediatric research in office settings and ambulatory sentinel practice network. Pediatrics 2000;106(4):E44.
    OpenUrlCrossRefPubMed
  12. 12.↵
    A study of headache in North American primary care. Report from the Ambulatory Sentinel Practice Network. J R Coll Gen Pract 1987;37:400–403.
    OpenUrlPubMed
  13. 13.↵
    1. Froom J,
    2. Culpepper L,
    3. Green LA,
    4. et al
    . A cross-national study of acute otitis media: risk factors, severity, and treatment at initial visit. Report from the International Primary Care Network (IPCN) and the Ambulatory Sentinel Practice Network (ASPN). J Am Board Fam Pract 2001;14:406–417.
    OpenUrlAbstract
  14. 14.↵
    1. Westfall J,
    2. Mold J,
    3. Fagnan L
    . Practice-based research—“Blue Highways” on the NIH roadmap. JAMA 2007;297:403–406.
    OpenUrlCrossRefPubMed
  15. 15.↵
    1. Westfall JM,
    2. Fagnan LJ,
    3. Handley M,
    4. et al
    . Practice-based research is community engagement. J Am Board Fam Med 2009;22:423–427.
    OpenUrlFREE Full Text
  16. 16.↵
    1. Fagnan LJ,
    2. Davis M,
    3. Deyo RA,
    4. Werner JJ,
    5. Stange KC
    . Linking practice-based research networks and Clinical and Translational Science Awards: new opportunities for community engagement by academic health centers. Acad Med 2010;85:476–483.
    OpenUrlPubMed
  17. 17.↵
    1. Macaulay AC,
    2. Nutting PA
    . Moving the frontiers forward: incorporating community-based participatory research into practice-based research networks. Ann Fam Med 2006;4:4–7.
    OpenUrlFREE Full Text
  18. 18.↵
    1. Felzien M,
    2. Zittleman L,
    3. Westfall JM
    . Farming, ranching, research: patient engagement on the high plains. J Gen Intern Med 2016;31:3–4.
    OpenUrl
  19. 19.↵
    1. Tapp H,
    2. Dulin M
    . The science of primary health-care improvement: potential and use of community-based participatory research by practice-based research networks for translation of research into practice. Exp Biol Med (Maywood) 2010;235:290–299.
    OpenUrlCrossRefPubMed
  20. 20.↵
    1. Creswell JW,
    2. Plano Clark VL,
    3. Gutmann ML,
    4. Hanson WE
    . Advanced mixed methods research designs. Handbook of Mixed Methods in Social and Behavioral Research 2003;209–240.
  21. 21.↵
    1. Fetters MD,
    2. Curry LA,
    3. Creswell JW
    . Achieving integration in mixed methods designs—Principles and practices. Health Serv Res 2013;48(6pt2):2134–2156.
    OpenUrlCrossRefPubMed
  22. 22.↵
    1. Gentles SJ,
    2. Charles C,
    3. Ploeg J,
    4. McKibbon K
    . Sampling in qualitative research: insights from an overview of the methods literature. Qual Rep 2015;20:1772–1789.
    OpenUrl
  23. 23.↵
    1. Atkinson R,
    2. Flint J
    . Accessing hidden and hard-to-reach populations: snowball research strategies. Social Res Update 2001;33:1–4.
    OpenUrl
  24. 24.↵
    1. Dillman DA,
    2. Smyth JD,
    3. Christian LM
    . Internet, mail, and mixed-mode surveys: The tailored design method. 3rd ed. Hoboken, NJ: John Wiley & Sons, Inc.; 2009.
  25. 25.↵
    1. Creswell JW,
    2. Poth CN
    . Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage Publications; 2017.
  26. 26.↵
    1. Siltanen J,
    2. Willis A,
    3. Scobie W
    . Separately together: Working reflexively as a team. Int J Soc Res Methodol 2008;11:45–61.
    OpenUrl
  27. 27.↵
    1. Finlay L,
    2. Gough B
    . Reflexivity: A practical guide for researchers in health and social sciences. Oxford, UK: John Wiley & Sons; 2008.
  28. 28.↵
    The James Lind Alliance. James Lind Alliance. 2019. Available from: http://www.jla.nihr.ac.uk/. Accessed May 1, 2019.
  29. 29.↵
    1. Brewer SE,
    2. O'Leary ST
    . Engaging parents and stakeholders in patient-centered pediatric research and research agenda setting: a guide for practice-based research networks. 2017. Available from: http://www.ucdenver.edu/academics/colleges/medicalschool/programs/ACCORDS/sharedresources/education/Documents/COCONet%20link%201_4.18.18.pdf. Accessed May 1, 2019.
  30. 30.↵
    JLA Guide Book. James Lind Alliance. 2018. Available from: http://www.jla.nihr.ac.uk/jla-guidebook/. Accessed May 1, 2019.
  31. 31.↵
    1. Riva JJ,
    2. Malik KM,
    3. Burnie SJ,
    4. Endicott AR,
    5. Busse JW
    . What is your research question? An introduction to the PICOT format for clinicians. J Can Chiropr Assoc. J Can Chiropr Assoc 2012;56:167–171.
    OpenUrl
  32. 32.↵
    1. Campbell C,
    2. Jovchelovitch S
    . Health, community and development: Towards a social psychology of participation. J Commun Appl Soc Psychol 2000;255–270.
  33. 33.↵
    1. Viswanathan M,
    2. Ammerman A,
    3. Eng E,
    4. et al
    . Community-based participatory research: assessing the evidence. Evid Rep Technol Assess (Summ) 2004;(99):1–8.
  34. 34.↵
    Healthy People 2020. Centers for Disease Control and Prevention. 2019. Available from: https://www.cdc.gov/nchs/healthy_people/hp2020.htm. Accessed May 1, 2019.
  35. 35.↵
    American Academy of Pediatrics. American Academy of Pediatrics Agenda for Children 2017–2018. Available from: https://www.aap.org/en-us/about-the-aap/aap-facts/AAP-Agenda-for-Children-Strategic-Plan/Pages/AAP-Agenda-for-Children-Strategic-Plan.aspx. Accessed January 17, 2019.
  36. 36.↵
    1. Westfall JM,
    2. VanVorst RF,
    3. Main DS,
    4. Herbert C
    . Community-based participatory research in practice-based research networks. Ann Fam Med 2006;4:8–14.
    OpenUrlAbstract/FREE Full Text
  37. 37.↵
    1. Matoff-Stepp S,
    2. Applebaum B,
    3. Pooler J,
    4. Kavanagh E
    . Women as health care decision-makers: implications for health care coverage in the United States. J Health Care Poor Underserved 2014;25:1507–1513.
    OpenUrl
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools

Jump to section

Keywords