Table 2.

COVID-19 Connected Care Center Evaluation Research Questions, Measures, and Data Collection Approach

RE-AIM Element and Research Question (RQ)	Measures/Data Collection Approach
Reach—the absolute number, proportion, and representativeness of individuals who are willing to participate in a given initiativeRQ: To what extent does the hotline reach the intended target population?	Numerators: number of target population for telephone hotline that called and spoke to someone; includes describing the following: trend in use over time, description of user (OHSU patient or not, sex, age), user patterns (time of the day), user location.Denominator: number of of OHSU patients and number of Oregonians without a primary care provider.Source: data from phone system, demographics for OHSU patients, statewide estimate of patients without a primary care provider.
Effectiveness—the impact of an intervention on important outcomes, including potential negative effects.RQ: Does the telephone hotline impact key outcomes?Primary outcomes:Patient and provider experienceWere questions answered?Did patients feel reassured?Would patients/providers recommend this service?How much do patients value the service?Clinical outcomes and care use • Was testing offered when indicated. • Follow-up rate. • In-person visit/video visit—follow-up rate. • Appropriate alignment of care level with severity—accuracy based on peer review. • Infection prevention—adherence to quarantine, isolation, return to work recommendations. • Emergency department—admission rates, total care cost.	Survey data: post-telephone surveys will be conducted with patients. Post-e-mail and post-telephone surveys will be conducted with providers.Qualitative interviews: these can be used to explore patient experiences more deeply and to try to understand variations in outcomes. Specifically, we will examine other information seeking and support seeking, access to services, alignment of the center with patients’ and providers’ needs, the community value of the call center, and access from both patients’ and providers’ perspectives.Source: electronic health records, Medicaid claims, statewide data.
Adoption—willingness and experience of staff implementing the hotline.	Source: semistructured interviews with staff to understand enjoyment of role, perceived readiness for role after training, responsibilities consistent with top of licensure, enjoyment of other nonclinical peers, perceived value of the hotline, confidence in the quality, and impact on well-being.
Implementation—at the setting level, implementation refers to the intervention agents' fidelity to the various elements of an intervention's protocol. This includes consistency of delivery as intended and the time and cost of the intervention.RQ: How was the hotline implemented? What were the key elements of the “intervention” and what processes were put in place to ensure fidelity of the hotline?	Source: qualitative data collection; interviews with key stakeholders: this will include understanding the staffing and functionality of the hotline, the staff full-time equivalent (FTE) and change in staff FTE over time, the type of staff involved, barrier/facilitators to implementation (eg, opportunities, technical issues, drop calls, hang up, hold times, and how these were resolved), partnership with the state and others that made this work possible.
Maintenance—the extent to which a program or policy becomes institutionalized or part of the routine organizational practices and policies.RQ: Among key stakeholders, how would the hotline (if it is needed) be maintained, and what is needed to maintain it?	Source: semistructured interviews with key stakeholders, after the active implementation phase of the pilot has ended, to assess factors related to maintenance.

RE-AIM, Reach, Effectiveness, Adoption, Implementation, and Maintenance; OHSU, Oregon Health & Science University.