Skip to main content

Main menu

  • HOME
  • ARTICLES
    • Current Issue
    • Ahead of Print
    • Archives
    • Abstracts In Press
    • Special Issue Archive
    • Subject Collections
  • INFO FOR
    • Authors
    • Reviewers
    • Call For Papers
    • Subscribers
    • Advertisers
  • SUBMIT
    • Manuscript
    • Peer Review
  • ABOUT
    • The JABFM
    • The Editing Fellowship
    • Editorial Board
    • Indexing
    • Editors' Blog
  • CLASSIFIEDS
  • Other Publications
    • abfm

User menu

Search

  • Advanced search
American Board of Family Medicine
  • Other Publications
    • abfm
American Board of Family Medicine

American Board of Family Medicine

Advanced Search

  • HOME
  • ARTICLES
    • Current Issue
    • Ahead of Print
    • Archives
    • Abstracts In Press
    • Special Issue Archive
    • Subject Collections
  • INFO FOR
    • Authors
    • Reviewers
    • Call For Papers
    • Subscribers
    • Advertisers
  • SUBMIT
    • Manuscript
    • Peer Review
  • ABOUT
    • The JABFM
    • The Editing Fellowship
    • Editorial Board
    • Indexing
    • Editors' Blog
  • CLASSIFIEDS
  • JABFM on Bluesky
  • JABFM On Facebook
  • JABFM On Twitter
  • JABFM On YouTube
Research ArticleOriginal Research

Stakeholder Engagement in a Patient-Reported Outcomes (PRO) Measure Implementation: A Report from the SAFTINet Practice-based Research Network (PBRN)

Bethany M. Kwan, Marion R. Sills, Deborah Graham, Mika K. Hamer, Diane L. Fairclough, K. E. Hammermeister, Alicyn Kaiser, Maria de Jesus Diaz-Perez and Lisa M. Schilling
The Journal of the American Board of Family Medicine January 2016, 29 (1) 102-115; DOI: https://doi.org/10.3122/jabfm.2016.01.150141
Bethany M. Kwan
From the Departments of Family Medicine (BMK), Pediatrics (MRS), and Emergency Medicine (MRS), and Divisions of Cardiology (KEH) and General Internal Medicine (LMS), Department of Medicine, and Adult and Child Consortium for Health Outcomes Research and Delivery Science (ACCORDS) (BMK, MRS, MKH, DLF, KEH, LMS), University of Colorado School of Medicine, Aurora, CO; Children's Hospital Colorado, Aurora, CO (MRS); The DARTNet Institute, Aurora, CO (DG); Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO (DLF); Metro Community Provider Network, Englewood, CO (AK); Salud Family Health Centers, Fort Lupton, CO (MdJD-P).
PhD, MSPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Marion R. Sills
From the Departments of Family Medicine (BMK), Pediatrics (MRS), and Emergency Medicine (MRS), and Divisions of Cardiology (KEH) and General Internal Medicine (LMS), Department of Medicine, and Adult and Child Consortium for Health Outcomes Research and Delivery Science (ACCORDS) (BMK, MRS, MKH, DLF, KEH, LMS), University of Colorado School of Medicine, Aurora, CO; Children's Hospital Colorado, Aurora, CO (MRS); The DARTNet Institute, Aurora, CO (DG); Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO (DLF); Metro Community Provider Network, Englewood, CO (AK); Salud Family Health Centers, Fort Lupton, CO (MdJD-P).
MD, MPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Deborah Graham
From the Departments of Family Medicine (BMK), Pediatrics (MRS), and Emergency Medicine (MRS), and Divisions of Cardiology (KEH) and General Internal Medicine (LMS), Department of Medicine, and Adult and Child Consortium for Health Outcomes Research and Delivery Science (ACCORDS) (BMK, MRS, MKH, DLF, KEH, LMS), University of Colorado School of Medicine, Aurora, CO; Children's Hospital Colorado, Aurora, CO (MRS); The DARTNet Institute, Aurora, CO (DG); Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO (DLF); Metro Community Provider Network, Englewood, CO (AK); Salud Family Health Centers, Fort Lupton, CO (MdJD-P).
MSPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Mika K. Hamer
From the Departments of Family Medicine (BMK), Pediatrics (MRS), and Emergency Medicine (MRS), and Divisions of Cardiology (KEH) and General Internal Medicine (LMS), Department of Medicine, and Adult and Child Consortium for Health Outcomes Research and Delivery Science (ACCORDS) (BMK, MRS, MKH, DLF, KEH, LMS), University of Colorado School of Medicine, Aurora, CO; Children's Hospital Colorado, Aurora, CO (MRS); The DARTNet Institute, Aurora, CO (DG); Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO (DLF); Metro Community Provider Network, Englewood, CO (AK); Salud Family Health Centers, Fort Lupton, CO (MdJD-P).
MPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Diane L. Fairclough
From the Departments of Family Medicine (BMK), Pediatrics (MRS), and Emergency Medicine (MRS), and Divisions of Cardiology (KEH) and General Internal Medicine (LMS), Department of Medicine, and Adult and Child Consortium for Health Outcomes Research and Delivery Science (ACCORDS) (BMK, MRS, MKH, DLF, KEH, LMS), University of Colorado School of Medicine, Aurora, CO; Children's Hospital Colorado, Aurora, CO (MRS); The DARTNet Institute, Aurora, CO (DG); Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO (DLF); Metro Community Provider Network, Englewood, CO (AK); Salud Family Health Centers, Fort Lupton, CO (MdJD-P).
DrPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
K. E. Hammermeister
From the Departments of Family Medicine (BMK), Pediatrics (MRS), and Emergency Medicine (MRS), and Divisions of Cardiology (KEH) and General Internal Medicine (LMS), Department of Medicine, and Adult and Child Consortium for Health Outcomes Research and Delivery Science (ACCORDS) (BMK, MRS, MKH, DLF, KEH, LMS), University of Colorado School of Medicine, Aurora, CO; Children's Hospital Colorado, Aurora, CO (MRS); The DARTNet Institute, Aurora, CO (DG); Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO (DLF); Metro Community Provider Network, Englewood, CO (AK); Salud Family Health Centers, Fort Lupton, CO (MdJD-P).
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alicyn Kaiser
From the Departments of Family Medicine (BMK), Pediatrics (MRS), and Emergency Medicine (MRS), and Divisions of Cardiology (KEH) and General Internal Medicine (LMS), Department of Medicine, and Adult and Child Consortium for Health Outcomes Research and Delivery Science (ACCORDS) (BMK, MRS, MKH, DLF, KEH, LMS), University of Colorado School of Medicine, Aurora, CO; Children's Hospital Colorado, Aurora, CO (MRS); The DARTNet Institute, Aurora, CO (DG); Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO (DLF); Metro Community Provider Network, Englewood, CO (AK); Salud Family Health Centers, Fort Lupton, CO (MdJD-P).
PA-C, MMSc
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Maria de Jesus Diaz-Perez
From the Departments of Family Medicine (BMK), Pediatrics (MRS), and Emergency Medicine (MRS), and Divisions of Cardiology (KEH) and General Internal Medicine (LMS), Department of Medicine, and Adult and Child Consortium for Health Outcomes Research and Delivery Science (ACCORDS) (BMK, MRS, MKH, DLF, KEH, LMS), University of Colorado School of Medicine, Aurora, CO; Children's Hospital Colorado, Aurora, CO (MRS); The DARTNet Institute, Aurora, CO (DG); Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO (DLF); Metro Community Provider Network, Englewood, CO (AK); Salud Family Health Centers, Fort Lupton, CO (MdJD-P).
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lisa M. Schilling
From the Departments of Family Medicine (BMK), Pediatrics (MRS), and Emergency Medicine (MRS), and Divisions of Cardiology (KEH) and General Internal Medicine (LMS), Department of Medicine, and Adult and Child Consortium for Health Outcomes Research and Delivery Science (ACCORDS) (BMK, MRS, MKH, DLF, KEH, LMS), University of Colorado School of Medicine, Aurora, CO; Children's Hospital Colorado, Aurora, CO (MRS); The DARTNet Institute, Aurora, CO (DG); Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO (DLF); Metro Community Provider Network, Englewood, CO (AK); Salud Family Health Centers, Fort Lupton, CO (MdJD-P).
MD, MSPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • References
  • Info & Metrics
  • PDF
Loading

Article Figures & Data

Figures

  • Tables
  • Figure 1.
    • Download figure
    • Open in new tab
    Figure 1.

    Cascading hub-and-spokes model of implementation. IT, information technology; PCOR, patient-centered outcomes research; QI, quality improvement.

    • Download figure
    • Open in new tab
    • Download figure
    • Open in new tab
    • Download figure
    • Open in new tab
    • Download figure
    • Open in new tab

Tables

  • Figures
    • View popup
    Table 1. Implementation Steps and Decisions for Patient-Reported Outcome Measures
    Implementation StepDecisions
    1: Determine PRO content and select a measurement tool1a: Identify stakeholder goals, available resources and barriers for collecting PRO measures
    1b: Determine which questionnaire(s) to use
    2: Establish local implementation plans2a: Select the patients to be assessed
    2b: Determine the setting and timing of assessments
    2c: Choose a mode for administering and scoring the questionnaire
    2d: Design processes for reporting results to providers and/or patients
    2e: Identify aids to facilitate score interpretation
    2f: Develop strategies for responding to issues identified by the questionnaires
    2g*: Identify local champions and key personnel, and prepare tools and materials for training and implementation across practices
    2h*: Establish systems for electronic data capture of the PRO results in structured, discrete fields
    3: Implementation and evaluation of the PRO measure3a*: Facilitate and track implementation across practices to ensure adequate data quality, and assess and address barriers
    3b: Evaluate the impact of the PRO implementation on the practice
    • ↵* SAFTINet project additions to the International Society for Quality of Life Research patient-reported outcome guidance.39

    • PRO, patient-reported outcome.

    • View popup
    Table 2. Patient-Reported Outcome Measure Data Collection Goals
    Research Partner GoalsClinical Partner Goals
    • Relevant to patients with condition of interest (eg, hypertension and hyperlipidemia)• Align with the clinical partners' current reporting requirements and quality improvement initiatives, such as PCMH and MU
    • Potentially influenced by the care in a PCMH• Brief (≤4 items), valid measure exists in both English and Spanish and at an appropriate reading level
    • Valid and reliable measure exists• Is of utility to the providers (ie, related to a common patient issue, amenable to intervention)
    • Content should not be redundant with existing data sources (eg, the EHR)• Informs day-to-day interactions with patients
    • EHR, electronic health record; MU, meaningful use; PCMH, patient-centered medical home.

    • View popup
    Table 3. Patient-Reported Outcome Measure Implementation Plan Details By Organization
    PRO Planning Worksheet SectionsResearch Team RequestDecisions
    Organization 1Organization 2Organization 3Organization 4
    Patients targetedAll adult patients with hypertension or hyperlipidemiaAll adult patients with hypertension or hyperlipidemiaAll patients with hypertension or hyperlipidemiaAll adult patientsAll patients with hypertension
    Identification of patientsNo request madeData query and list generation by IT staffMedical assistant checks medication list, gives the tool to anyone taking blood pressure or cholesterol medicationsAll adult patients asked to complete the tool upon check-in for visitRooming medical assistant reviews problem list and reason for visit
    Frequency of administrationAt least annually; all visits preferredAnnuallyAt all visitsAt all visitsAt hypertension visits (2 times/year)
    Mode of administration (by whom, in what setting, with what materials)No request madeCare coordinators administered by telephone as part of larger patient surveyAdministered by medical assistant; responses recorded on paper upon check-in for visitAdministered as part of written survey before the appointment upon check-in for visitElectronic survey; provider asked the questions verbally and recorded the answers in the EHR template
    Data captureResponses recorded in structured fields in an EHR or other databaseCare coordinators entered responses directly into discrete fields in the EHR templateResponses manually entered into ancillary database; paper forms scanned into the EHRMedical assistant entered results in the EHR templateResponses entered into the EHR template for hypertension visits
    Presentation and interpretation of results (to whom, when, in what form?)Recent/timely patient responses are available to the provider at the time of the visitCare coordinators scheduled an appointment with a provider for patients with barriers; patient responses are available to the provider in the EHR summaryPaper form is available to the provider at the time of the visitAvailable to the provider at the point of care through the EHRPart of the provider interview with patients during hypertension visits
    TrainingPowerPoint presentation made availableMedical providers, nurses, and care coordinators trained in administration, utilization, and interpretation; led by clinician championsClinician champions trained other providers; practice manager trained the medical assistantsTraining at provider meeting on purpose of tool, interpretation, process/workflow; Web-based training for medical assistantsBusiness analyst trains medical assistants on workflow for hypertension template; medical director trained providers on use of the template; training done remotely with ancillary in-person support for troubleshooting
    Required resourcesTime for local site coordinationCare coordinator time; EHR team time to build template; IT staff time to generate a contact list of eligible patientsTime for training; time for manual data entryTime for modifications to the EHR; time for trainingTime for EHR template building; personnel time for training and monitoring progress
    • EHR, electronic health record; IT, information technology; PRO, patient-reported outcome.

    • View popup
    Table 4. Participating Organizations and Clinical Practices and Data Collection Results over the 9-Month Study Period
    Organization 1Organization 2Organization 3Organization 4
    Clinical practice representativeDirector of behavioral healthInternal medicine physicianAssociate medical directorResearch and data manager
    Participating practices, n149139
    Practice specialties, n
        IMH0050
        FM6479
        IM1210
        FM + IM0300
        FM + IM + pediatrics7000
    Practice location (urban/rural), n*6/89/05/87/2
    Medical providers per practice, median (range)4 (1–9)8 (6–15)2 (1–5)7 (2–15)
    Patients per practice, median (range)2,925 (554–10,359)6,644 (5,539–10,199)2,765 (1,869–16,237)8,689 (2,375–15,058)
    Practices implementing MAS, n (%)10 (71)8 (89)13 (100)9 (100)
    Surveys per practice
        Median (range)24 (0–136)23 (0–149)69 (3–416)164 (16–311)
    • ↵* Based on 2010 census designation; some practices designated “rural” by the census are better characterized as suburban.

    • FM, family medicine; IM, internal medicine; IMH, integrated mental health; MAS, Medication Adherence Survey.

PreviousNext
Back to top

In this issue

The Journal of the American Board of Family     Medicine: 29 (1)
The Journal of the American Board of Family Medicine
Vol. 29, Issue 1
January-February 2016
  • Table of Contents
  • Table of Contents (PDF)
  • Cover (PDF)
  • Index by author
  • Back Matter (PDF)
  • Front Matter (PDF)
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on American Board of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Stakeholder Engagement in a Patient-Reported Outcomes (PRO) Measure Implementation: A Report from the SAFTINet Practice-based Research Network (PBRN)
(Your Name) has sent you a message from American Board of Family Medicine
(Your Name) thought you would like to see the American Board of Family Medicine web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
6 + 3 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Stakeholder Engagement in a Patient-Reported Outcomes (PRO) Measure Implementation: A Report from the SAFTINet Practice-based Research Network (PBRN)
Bethany M. Kwan, Marion R. Sills, Deborah Graham, Mika K. Hamer, Diane L. Fairclough, K. E. Hammermeister, Alicyn Kaiser, Maria de Jesus Diaz-Perez, Lisa M. Schilling
The Journal of the American Board of Family Medicine Jan 2016, 29 (1) 102-115; DOI: 10.3122/jabfm.2016.01.150141

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Stakeholder Engagement in a Patient-Reported Outcomes (PRO) Measure Implementation: A Report from the SAFTINet Practice-based Research Network (PBRN)
Bethany M. Kwan, Marion R. Sills, Deborah Graham, Mika K. Hamer, Diane L. Fairclough, K. E. Hammermeister, Alicyn Kaiser, Maria de Jesus Diaz-Perez, Lisa M. Schilling
The Journal of the American Board of Family Medicine Jan 2016, 29 (1) 102-115; DOI: 10.3122/jabfm.2016.01.150141
Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • Methods
    • Results
    • Discussion
    • Acknowledgments
    • Appendix 1
    • Notes
    • References
  • Figures & Data
  • References
  • Info & Metrics
  • PDF

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • A Review of 50 Years of International Literature on the External Environment of Building Practice-Based Research Networks (PBRNs)
  • A Review of 50-Years of International Literature on the Internal Environment of Building Practice-Based Research Networks (PBRNs)
  • Adapting Diabetes Shared Medical Appointments to Fit Context for Practice-Based Research (PBR)
  • A Practical Approach to Establishing a Practice-Based Research Network Stakeholder Engagement Infrastructure
  • Dissemination and Implementation Science in Primary Care Research and Practice: Contributions and Opportunities
  • Are We Learning More about Patient-centered Medical Homes (PCMHs), or Learning More about Primary Care?
  • Family Medicine Research That Provides Compelling, Urgent Data to Improve Patient Care
  • Google Scholar

More in this TOC Section

  • Successful Implementation of Integrated Behavioral Health
  • Identifying and Addressing Social Determinants of Health with an Electronic Health Record
  • Integrating Adverse Childhood Experiences and Social Risks Screening in Adult Primary Care
Show more Original Research

Similar Articles

Keywords

  • Comparative Effectiveness Research
  • Hypertension
  • Medication Adherence
  • Patient Outcome Assessment
  • Patient-centered Outcomes Research
  • Practice-based Research

Navigate

  • Home
  • Current Issue
  • Past Issues

Authors & Reviewers

  • Info For Authors
  • Info For Reviewers
  • Submit A Manuscript/Review

Other Services

  • Get Email Alerts
  • Classifieds
  • Reprints and Permissions

Other Resources

  • Forms
  • Contact Us
  • ABFM News

© 2025 American Board of Family Medicine

Powered by HighWire