Article Figures & Data
Tables
Clinic 1 (partially implemented) is a primary care Federally Qualified Health Center (FQHC). This practice is partnering with a local nonprofit, community-based prevention, mental health, and addiction agency to colocate one licensed clinical psychologist in the practice 5 days a week. The psychologist provided brief, targeted behavioral health care to patients, with a particular focus on providing behavioral health support to patients with diabetes. Primary care physicians in the practice referred patients to the psychologist, with the front desk scheduling these appointments, and when needed, they engage the psychologist during a patient's visit for an introduction or warm-handoff, or to seek this professional's advice regarding the best treatment path for the patient (eg, see psychologist, see mental health provider, seek specialty treatment outside the practice). The psychologist had a private office in the practice that is located in close proximity to some of the primary care pods and farther from others, as this is a large practice. In addition, this practice also contracted with the same organization to colocate a mental health professional. This professional was an unlicensed social worker and was supervised by a licensed professional located at the mental health organization. This person provided traditional mental health services for patients with more serious and persistent mental illness, typically meeting with patients for 50-minute visits and for 12 weeks or more, as needed. This practice had written protocols in place for introducing the psychologist and mental health provider to patients and for making care transitions/referrals to these professionals. Not long after implementation of the BH e-Suite, the practice lost a critical team member who knew the tool well and was a consistent user. When this individual left, so did the institutional knowledge on tool usage. Although new behavioral health clinicians (BHCs) used the tool, use was superficial (eg, for documentation purposes only). Clinic 2 (fully implemented) is a primary care FQHC that employs two licensed clinical social workers (LCSW) to provide brief behavioral health care to their patients. The LCSWs shared an office that is adjacent to the primary care team's location in the practice. They did not have a private office to see patients, but generally did so in a medical examination room. Typically, the LCSWs were engaged by a primary care physician when the patient was in for a medical visit; the physician or medical assistant would go find the LCSW, who conducted a brief assessment and intervention and established a plan for following up with the visit. LCSWs would see patients, as needed, for brief visits. These visits are scheduled by the LCSW with the patient. The LCSWs played a central role in screening and brief intervention for substance use, as this had been a practice focus, and had developed protocols and a committee that reviews prescriptions for controlled substances. The practice had clear protocols for these workflows. Prior to expansion of the LCSW workforce in this practice, there was a colocated mental health professional, but this role has been phased out, and LCSWs assisted in transitioning patients to care at a local community mental health center (CMHC) for patients with more serious and persistent mental illness and substance use care need. Tool uptake was initially slow, due in part to internal billing negotiations. Once implemented, additional BHCs joined the staff and all used the tool extensively and fully. Clinic 3 (did not implement) is a FQHC-CMHC hybrid. It is a county health department health center that includes primary care, mental health care, developmental disabilities services, environmental health, and other health-related community services. The units in this health center functioned autonomously, with the primary care practice located on the 1st floor and the mental health practice located on the 2nd floor. The CMHC served the county and took referrals from other organizations in the county, and primary care physicians in the building referred patients with mental/behavioral needs to the mental health unit in the same way other organizations in the community do. This health center does not currently employ BHCs to work closely with the primary care team. This practice had prior experience embedding BHCs with primary care, but just prior to implementation of the BH e-Suite these clinicians left the clinic and were not replaced. In addition, this health center does not have written protocols or other documentation in place to describe how medical, behavioral, and mental health care might be integrated for patients. The clinic experienced a great deal of turnover during the study. Clinic 4 (partially implemented) is a primary care FQHC. This practice partnered with a local nonprofit, community-based prevention, mental health, and addiction agency to colocate one licensed clinical psychologist in the practice 5 days week to provide patients with brief, targeted behavioral health care, with a particular focus on providing behavioral health support to patients with diabetes. The psychologist had a private office and treated patients there, but s/he also had a workstation that is within close proximity to the primary care team. In addition to accepting referrals from the primary care physicians in the practice, physicians engaged the psychologist via warm-handoff so that s/he could provide brief services to patients in the medical examination rooms, as needed. The physicians also sought this professional's advice regarding the best treatment path for patients (eg, in-clinic psychologist, in-clinic mental health provider, out-of-clinic specialty treatment), as needed. The practice also employed a colocated mental health provider who cares for patients with more serious and persistent mental illness who need longer visits (50-minutes) and a longer treatment course. This role was filled by a psychology practicum student from a local university where s/he received supervision for this work. This practice developed workflow for their integrated approach, specifying the roles for the front desk, medical assistants, physicians, psychologists, and mental health provider. At the time the BH e-Suite was launched, the practice experienced turnover among clinical staff that was quite disruptive. Operations did stabilize and the BH e-Suite was used by BHCs, but these professionals did not use all functionality. Clinic 5 (fully implemented) is a primary care FQHC that employed one unlicensed social worker (MSW) to provide brief behavioral health care to their patients. The MSW shared an office with the practice's Epic specialist. This professional did not see patients in their office but in medical examination rooms. Typically, the MSW was engaged by a primary care physician when the patient was in for a medical visit; the physician or medical assistant will go find the MSW, who will do a brief assessment and intervention and establish a plan for following up. MSW also helped when patients need to be referred to outpatient mental health and substance use care. MSW would see patients, as needed, for brief visits. These visits were scheduled by the MSW with the patient. In addition, the MSW played a central role in screening and brief intervention for substance use, as this has been a practice focus. The practice had clear protocols for these workflows. The BHC role was new to this practice. Tool implementation was limited in part due to access issues (the BHC had difficulty in accessing certain functionality). Documentation was the primary focus of tool usage. Clinic 6 (did not implement) is a FQHC primary care branch of the county health department. This practice hired one LCSW to work in this practice to deliver integrated care. This was the practice's first BHC, and this person was hired a few months prior to the start of the study. The LCSW provided both brief therapy to patients as well as more traditional mental health care to patients who need it. To accommodate both types of patients, the LCSW took referrals and was engaged in warm-handoffs by the primary care physicians when the LCSW was available. The LCSW shared an office with a diabetes counselor and this office was located next to the medical examination rooms and a couple rooms away from the primary care provider office. The LCSW saw patients in his/her office or in a medical examination room, depending on space availability. This practice did not have written protocols or workflows for care integration. This BHC was excited about the BH e-Suite but practice leadership was not. Description Intervention, N = 4,377 Control, N = 3,628 N (%) N (%) Age, mean, SD 49.1 (14.2) 49.1 (14.6) Sex, female 1,308 (30) 1,113 (30) Race/Ethnicity White, Non-Hispanic 2,964 (69) 2,302 (65) Black, Non-Hispanic 36 (1) 101 (3) Hispanic 1,216 (28) 990 (28) Other 86 (2) 170 (5) Language, English 3,280 (75) 2,792 (77) Household income, $ <14,999 2,254 (56) 2,219 (63) 15,000 to 29,999 1,339 (33) 835 (24) 30,000 to 44,999 319 (8) 277 (8) 45,000 to 59,999 77 (2) 113 (3) 60,000+ 36 (1) 71 (2) Number of comorbidities 0 455 (10) 356 (10) 1 1,008 (23) 791 (22) 2 1,107 (25) 910 (25 3+ 1,807 (41) 1,571 (43) Depression diagnosis 1,951 (45) 1,678 (46) Diabetes diagnosis 1,175 (27) 1,122 (31) SD, standard deviation.
- Table 3.
Clinical Care and Patient Experience of Care Outcomes across Degree of Implementation
Overall Fully Implemented Partially Implemented Did Not Implement Process of Care Outcomes* PHQ-9 screens, OR 1.06‡ 1.10‡ 1.01 1.11‡ GAD-7 screens, OR 1.01 1.01 0.85§ 1.03§ Intermediate Clinical Outcomes* PhQ-9 scores, β 0.16 -0.64 0.23 1.98‡ GAD-7 scores, β 1.38 2.59 4.42 −4.55‖ Patient Experiences of Care†, mean (SD) Satisfaction with care 0.06 (0.07) 0.09 (0.11) 0.08 (0.12) −0.03 (0.08) Care communication with PCP 0.15 (0.08) −0.02 (0.13) 0.39§ (0.13) 0.04 (0.13) Care communication with BHC 0.06 (0.13) 0.10 (0.24) 0.12 (0.23) −0.10 (0.20) Integration of care 0.20§ (0.07) 0.44‡ (0.13) 0.09 (0.13) 0.10 (0.13) Care coordination 0.06 (0.04) 0.06 (0.06) 0.11 (0.06) −0.02 (0.07) BHC, behavioral health clinician; GAD-7, Generalized Anxiety Disorder-7; PCP, primary care provider; PHQ-9, Patient Health Questionnaire-9; OR, odds ratio; SD, standard deviation.
↵* Pre/post change among intervention clinics compared to control clinics.
↵† Pre/post change among intervention clinics.
↵‡ P value <.001.
↵§ P value <.01.
↵‖ P value <.05.
- Table 4.
Implementation Degree across Practices and Consolidated Framework for Implementation Research Domains
Domain Definition Fully Implemented Partially Implemented Did Not Implement Inner Setting Baseline Adaptive Reserve (mean, SD) Practice capacity for change 0.63 (0.08) 0.56 (0.07) 0.55 (0.08) Implementation climate Commitment to implement change ∙ BHCs interested in improving BH documentation in EHR ∙ BHC interested in improving BH documentation in EHR ∙ BHCs satisfied with current system and resistant to new tool ∙ Upper management advocated for using a new tool tailored to BHC needs ∙ Interest in tool from upper management ∙ Little buy-in from upper management ∙ Epic specialist available to assist with implementation. ∙ Epic site specialist did not assist with implementation ∙ Integration model did not facilitate use of the tool, as these clinics focused on long mental health appointments, rather than brief interventions with BHCs Readiness for implementation Organizational capacity to implement change ∙ Step-by-step guides produced by BHC leadership on how to use the tool ∙ Unstable BHC workforce/turnover; no written BHC workflows ∙ No time allocated for training and educating BHCs about tool although Epic site specialist available ∙ Stable BHC workforce, written BHC workflows in place Process Characteristics Planning Tasks and methods conducted to prepare for implementation ∙ Had introductory implementation meeting before rolling out the tool ∙ No implementation plan aside from one introductory implementation meeting before rolling out the tool ∙ Had introductory implementation meeting before rolling out tool ∙ Lead BHC acted as opinion leader and champion of tool implementation ∙ No follow-up by clinics to implement ∙ MH manager felt the tool was not useful to them and declined implementing the tool Engaging Involvement of staff in the change process ∙ Familiarized other clinical staff, including front desk staff and PCPs, with the new tool ∙ BHC supervisor left this role; loss of champion derailed implementation ∙ No clinic champion or formally appointed internal implementation leader Reflecting and evaluating Team debriefing about process and experience ∙ Dedicated agenda time at BHC monthly meeting to discuss the tool and provide feedback to tool developer ∙ Feedback given to tool developers, but no follow through by practice ∙ No reflection or evaluation because tool was not implemented Individual Characteristics Knowledge/beliefs about intervention Individuals' attitudes toward and value placed on intervention ∙ Believed tool enhanced EHR use (simplified it) ∙ Newly hired BHCs reported receiving limited (1 hour), and inadequate training in the tool ∙ Did not find the tool to be useful ∙ Consensus view: tool was a step of progress ∙ BHCs did not know how to use tool Self-efficacy Belief in one's ability to execute implementation tasks ∙ BHCs felt they received sufficient training to use tool BH, behavioral health; BHC, behavioral health clinician; MH, mental health; PCP, primary care provider; EHR, electronical health record; SD, standard deviation.
Measure Survey Questions Response Items Care coordination using the Picker Institute Scale 1) Did you know who was in charge of your care for each of your health problems?
2) How often were the doctors, nurses and other health care providers who cared for you familiar with your most recent medical history?
3) How often were your providers aware of changes in your treatment that other providers recommended?
4) Do you think your providers had all the information they needed, such as test results, to make decisions about your treatment?
5) How often did you know who to ask when you had questions about your health problems?
6) How often were you given confusing or differing information about your health or treatments?
7) How often did you know what the next step in your care would be?
8) How well did your health care providers work together?
Questions 1 to 4: [1]= No, [2]= Yes, sometimes, 3]= Yes, most of the time; or
Questions 5 to 8: [1]=Never, [2]=Rarely, [3]=Occasionally, [4]= Often, [5]= Very oftenSatisfaction with care How satisfied have you been with … [1]= Very dissatisfied, [2]=Dissatisfied, [3]= Neutral, [4]= Satisfied, [5]= Very satisfied 1) the overall quality of care at your primary care clinic;
2) being able to get an appointment in your clinic as soon as you think you need it;
3) being able to get a telephone call or email answered from your primary care clinic as soon as you think you need it.
Integration of care using the Primary Care Assessment Tool 1) Did your primary care doctor suggest you go to the behavioral health provider?
2) Did your doctor discuss with you different places you could have gone to get help with that problem?
3) Did your doctor or someone working with your doctor help you make the appointment for that visit?
4) Did your doctor provide any information for the behavioral health provider about the reason for the visit?
5) Did the doctor know you made this visit to the behavioral health provider?
6) Did your doctor know what the results of this visit were?
7) After going to the behavioral health provider, did your doctor talk with you about what happened at the visit?
8) Does your doctor seemed interested in the care you get from the behavioral health provider?
[1]=Probably Not, [2]=Probably, and [3]= Definitely Care communication using the Connecting, Assessing, Responding, and Empowering (CARE Tool) How was the doctor at… [1]=Poor, [2]=Fair, [3]= Good, [4]= Very good, [5]= Excellent 1) making you feel at ease
2) letting you tell your story
3) really listening
4) being interested in you as a whole person
5) fully understanding your concerns
6) showing care and compassion
7) being positive
8) explaining things clearly
9) helping you take control
10) making a plan of action with you
