Abstract
Background: A standing order (SO) authorizes nurses and other staff to carry out medical orders per practice-approved protocol without a clinician's examination. This study implemented electronic SOs into the daily workflow of primary care practices; identified methods and strategies; determined barriers and facilitators; and measured changes in quality indicators resulting from electronic SOs.
Methods: Within 8 practices using the Practice Partner® electronic health record (EHR), a customized health maintenance template provided SOs for screening, immunization, and diabetes measures. EHR data extracts were used to calculate the presence and use of these measures on health maintenance templates and performance over 21 months. Qualitative observation/interviews at practice site visits, network meetings, and correspondence enabled synthesis of implementation issues.
Results: Improvements in template presence, use, and performance were found for 14 measures across all practices. Median improvements in screening ranged 6% to 10%; immunizations, 8% to 17%, and diabetes, 0% to 18%. Two practices achieved significant improvement on 14 of the 15 measures. All practices significantly improved on at least 3 of the measures.
Conclusions: A small sample of primary care practices implemented SOs for screening, immunizations and diabetes measures supported by PPRNet researchers. Technical competence and leadership to adapt EHR reminder tools helped staff adopt new roles and overcome barriers.
A standing order (SO) in primary care practice authorizes nurses and other staff to carry out a medical order according to a practice-approved protocol without a provider's examination or requirement for approval. By establishing SOs and empowering staff to complete them, practices can improve efficiency and quality of care. Success with the implementation of SOs is related to the education and enthusiasm of both clinicians and staff in the practice,1 as well as performance feedback and reminder systems.2,3 Previous research on SO use in primary care resulted in increased immunization rates for children and older adults in practices with paper-based medical records.4⇓–6 To improve the delivery of preventive services, the strongest interventions have focused on organizational change that includes delegation of prevention activities.7 Research on the effectiveness and sustainability of other disease-specific SO systems is limited, however,8 and extending the repertoire of practice-approved protocols may improve overall quality in primary care.
Inherent within different electronic health record (EHR) systems are a variety of features to embed clinical guidelines to improve the quality of care, yet these are often only partially adopted by practices.9⇓⇓–12 Reminder systems offer good potential to prompt the delivery of preventive services,13,14 but ineffective training and technical support15 and competing demands in primary care may result in inadequate delivery of preventive services and chronic care management.16,17 Evidence is needed for the optimal use and impact of multifunctional, commercially available EHR systems.18 The purpose of this pilot demonstration study—Standing Orders-Translation of Research into Practice (SO-TRIP)—was to implement electronic health maintenance (HM) reminders for screening, immunizations, and diabetes care monitoring as SOs in a subset of PPRNet* practices.
Context
PPRNet is a national primary care practice-based research network established in 1995, with administrative and research offices at the Medical University of South Carolina. The network currently is composed of 225 practices (150 practices were active at time of study) located throughout the United States that all use Practice Partner® (PP) patient records (McKesson, Inc, San Francisco, CA). Practices submit quarterly EHR data extracts and receive practice reports for benchmarking and performance improvement on a wide variety of measures relevant to primary care practice. They also have opportunities to participate in network research projects relevant to their interests. PPRNet developed a theoretically based and empirically validated quality improvement (QI) model within previous research, termed the PPRNet-TRIP QI model.19⇓–21 This model is disseminated during site visits and network meetings, including 5 concepts that are used to stimulate practices to focus on improvement: prioritizing performance, involving all staff, redesigning the system, using EHR tools, and activating the patient.22 This provides the basis for PPRNet intervention and a theoretical guide for improvement, enabling dissemination of practice strategies used on the path to improvement.23,24 High-performing practices in PPRNet that are skilled in the use of the EHR have seen the implementation of SOs, among other strategies, lead to improved quality of care.19,23,25
Methods
Design
The SO-TRIP study was designed as a mixed methods demonstration project and conducted over 2 years beginning July 1, 2008, guided by the PPRNet-TRIP QI model. The aims were to implement an electronic SO system into daily workflow in 8 primary care practices, identifying the best methods and strategies utilized; to determine barriers and facilitators to the use of electronic SOs in these practices; and to measure changes in quality of care indicators resulting from use of electronic SOs. The study received institutional review board approval from the Medical University of South Carolina.
Sample
Practices within PPRNet were invited to participate in this study if they were new to PPRNet research and had not already implemented electronic SOs. Invitations to participate were sent to the PPRNet member listserv and additional recruitment took place during the McKesson PP annual meeting in Seattle, Washington, in July 2008. Eight practices were purposely selected from 20 interested practices to get a diverse mix of geographic representation and practice sizes. Practice clinicians agreed to implement electronic SOs within their practice, host 2 site visits by the PPRNet research team, and send 2 practice representatives to 2 network meetings between October 2008 and April 2010.
Measures
SOs were developed for 15 measures (Table 1) in the areas of screening,26 adult immunizations,27 and diabetes care,28 as published by the United States Preventive Services Task Force, Centers for Disease Control and Prevention Advisory Committee on Immunization Practices, and American Diabetes Association as of July 2008. These measures complemented the “recommended starter set of clinical performance measures for ambulatory care,” selected by the investigators on the basis of their level of evidence, inclusion in primary care–relevant guidelines, and expected ease of translation to an SO format.
Intervention
An interdisciplinary team (medical doctors, registered nurse, and doctor of pharmacy) of PPRNet researchers conducted a 1-day introductory network meeting in October 2008 to explain the project to the 8 participating practices. Each practice sent a clinician and a staff member to the meeting, and they served as liaisons to lead the project implementation within their practices. The investigators introduced the clinical measures to the participants and reviewed evidence to guide using these as SOs. The PP HM features were demonstrated as the source of the electronic SOs. The PP HM system was adapted at the practice level to create templates based on age, sex, diagnoses, and medications. Utilities within the EHR allowed application of templates based on diagnoses and an extensive list of synonyms for the diagnoses. The HM system calculates when procedures are due based on rules created with the template, displaying all procedures relevant for the patient and highlighting overdue items in red. Overdue items also were viewable on a chart summary screen and, as desired by the practice, in a pop-up window when a patient's chart was opened.
At the introductory network meeting, all practice participants made plans to launch the project within their practices and to configure their EHR to ensure that the relevant HM templates (HMTs) were present in their patients' charts. Two investigators (SMO and LSN) made initial site visits from October through December 2008 to further reinforce the project goals and assist with electronic SO implementation. Quarterly SO-TRIP performance reports, highlighting baseline and quarterly data for the presence of the template for each measure, use of the template, and performance on the measure, were provided to each of the study practices. The practice's performance reports and plans from the network meeting were reviewed during the site visit, and each practice was encouraged to develop follow-up plans. Each practice developed an approach to implementing electronic SOs that fit within their individual contexts, recognizing that primary care practices are complex systems.29 Second site visits were made at the midpoint of the project (June–September 2009) to learn how the project was implemented at practices and to help practices overcome barriers to routine SO implementation.
A second network meeting was held in September 2009 to share best practices. At this meeting, each practice presented the specific experiences they had when implementing SO-TRIP; this encouraged discussion, further reflection by practices, and reconsideration or revision of their own plans and strategies. A final site visit or evaluation phone conference took place within the last quarter of the data collection period (March 2010) to elicit general perspectives related to the project and perceptions about sustainability.
Quantitative Data Collection and Analyses
Practices electronically submitted EHR data extracts to PPRNet on a quarterly basis. To protect patient confidentiality, the extract program assigned a unique, anonymous numerical identifier for each patient. Data were processed, aggregated, and converted into a longitudinal database as SAS datasets (SAS Inc, Cary, NC).
The research team used the PPRNet database to measure the presence and use of HMTs as process measures and performance of the study indicators as the outcome measure. Each measure was calculated by determining the patients eligible for the measure (denominator) and those who met the criterion for the measure (numerator). The percent of patients who met each measure criterion was calculated for each practice. For example, to determine the presence of the measure on the HMT, the HMT records of each eligible patient were searched for the presence of the specific item. Similarly, to determine the use of the template, HM records of eligible patients with the item on the template were searched. Performance for the study measures was calculated in a similar manner, examining appropriate data to assess the measure (eg, laboratory records were searched for results of cholesterol tests).
Practice-level repeated measures analyses (using a generalized, linear, mixed model approach for longitudinal analyses)30 were used to examine whether there were significant increases (trends) in each of the individual measures across all practices as well as within practices over time. All analyses were conducted using SAS version 9.2 (SAS Inc), and P values < .05 were considered statistically significant.
Qualitative Data Collection and Analyses
Multiple qualitative methods were used to determine the barriers and facilitators to the adoption of and ability to sustain a new electronic SO system within each practice. Site visit discussions were led by a qualitative researcher (LSN) to elicit practice members' perspectives on the implementation of SOs. Many of the site visit discussions were recorded for later transcription and qualitative analyses. A formative evaluation was used31,32 to explore practice context, leadership, workflow, time management, and relationships. Observations of how the staff used the electronic SO system were made during each practice site visit. Field notes were taken during site visits by the project investigators using an internally developed evaluation template to record practice attitudes, activities, plans, and the context for the implementation at each practice as well as observations, impressions, and reflections. Monthly correspondence (by E-mail or telephone conference) was initiated with each practice to follow up on strategies that the practices had used to implement their electronic SO system.
Semistructured interviews were conducted at each network meeting (by LSN) to elicit staff and clinician perspectives about the process of change related to the use of SOs in the practice. A semistructured interview guide provided a starting point for discussion to evaluate the clinician and staff satisfaction with using the electronic SO, perceived barriers and facilitators, and perceptions about the workflow changes related to this process. Site visits and network meetings were designed to encourage maximum participation and discussion about the use of SOs, what was helpful, what was difficult, what new activities were being undertaken, and whether these were working. We sought out diverse perspectives, looking for differences of opinion and differences in the ways each individual personally conducted this new approach. Qualitative analyses used a constant comparative method,33 comparing data from all sources to generate new insights related to the perspectives of the practices and to look for examples of success and difficulty with implementing this electronic SO system. A set of core barriers, facilitators, and best practice strategies for implementation was developed.
Results
Practice Characteristics
The project was conducted within 8 primary care practices from 8 states, with 2 to 25 clinicians per practice. Table 2 provides characteristics of each practice.
Quantitative Findings
Table 3 shows baseline and end of study data for the 2 process measures (presence and use of HMT for each clinical measure) and performance measures. At the beginning of the study, the median percentage among practices in the presence of the HMT was at least 80% for 7 of the study measures; by the end of the study it was at least 80% for all measures other than influenza vaccination in high-risk patients. Statistically significant improvements in HMT use were noted for 6 of the 15 study measures. As shown in Table 3, statistically significant improvement in performance was exhibited across all practices for 6 of the 15 study measures: osteoporosis screening, pneumococcal vaccination in those ≥65 years old and younger adults at high risk, tetanus/diphtheria vaccination, zoster vaccination, and urinary microalbumin. Although not statistically significant, improvement trends were noted for 8 of the 9 other measures, with glycosylated hemoglobin performance unchanged. Statistically significant improvements were found in at least 5 of the 8 practices for all measures other than influenza vaccination. Two practices achieved statistically significant improvement on 14 of the 15 measures, and all practices significantly improved on at least 3 of the 15 measures.
Qualitative Findings
Each practice had a unique experience during the project, based on their individual contexts and relevant strengths and weaknesses. Because there was not a predefined implementation protocol, the process occurred differently in each of the practices. Practices with the greatest improvement had established policies and protocols and educated their staff about their new roles. Reinforcing the system was critical to success, and these practices followed up on the project with staff, soliciting staff input and posting quarterly performance reports to share successful approaches. Regular staff huddles, or “rounding” with staff, took place in some practices to hardwire expectations. Several practices enhanced staff knowledge to enable them to act confidently on an SO using regular staff meetings or dedicated training sessions conducted by practice physicians. Many of the practices took an incremental approach to implementing a set of measures at first and the added others when success was demonstrated. Some practices focused on a more limited set of SOs throughout the project.
All practices applied EHR HMTs and some practices developed additional tools to help the nursing staff systematically assess the patient's status for overdue HM measures. The most effective tools included nursing note templates for staff to address the SO on which they acted, which also recorded the data on the HMT. Several practices had staff assess patients' HM needs during a brief session before the visit, enabling more time for the clinician to discuss other issues with the patient. Some practices activated patients to be more aware of their screening, immunization, and diabetes care schedules through the use of patient information update forms (derived from EHR data). These forms for updates were distributed to patients by front desk staff when the patient checked in for appointments, and the nursing staff addressed the patient's needs directly before the provider visit. Bulletin boards were used frequently by practices to emphasize the importance of these measures to both the staff and the patients.
Two of the 8 practices experienced more difficulty than others in implementing the project. The Texas practice was the largest in the study, with 3 office locations, and variable interest in the project among the numerous staff and physicians. The New York practice, in contrast, merged 2 separate offices during the study, but only one of the original offices participated in the project activities. Additional qualitative findings related to each practice's specific experience can be found in the Appendix.
Barriers and Facilitators /Best Practice Strategies
A set of barriers and facilitators were found to be related to implementing electronic SOs in these practices and are presented in Table 4. The table synthesizes the themes found in the evaluation of all practices as a whole rather than the experience of each practice.
Discussion
This study evaluated the implementation of electronic SOs in a diverse group of small to medium sized primary care practices. Participating practices largely adopted the electronic SO approach, resulting in statistically significant improvement in 6 of the 15 study measures, with trends toward improvement for 8 of the other measures. The measure that did not change, glycosylated hemoglobin, in patients with diabetes was higher at baseline within these practices, indicating that improvement of diabetes care had been prioritized through other initiatives and therefore was not a main focus of practice change.
There are several limitations to our study. First, because there was no concurrent control group, we cannot be certain that changes in the performance measures can be attributed to the SO-TRIP intervention, although the qualitative analyses support this interpretation. Second, given the short duration of the study, both longer-term impacts and sustainability of the intervention could not be evaluated. Third, lessons learned from practices that use a multifunctional EHR and that volunteered to participate based on an interest in implementing SOs may not be generalizable to other primary care practices. Also, assistance in using EHR functions and support for workflow redesign was provided within our study, which may limit generalization to practices without such resources. Finally, because receipt of a procedure (mammogram, bone mineral density testing, or any immunization) was in part assessed from data recorded on the HM table, increased use of the HM features rather than actual delivery of more services may have biased our assessments of improvements of these measures. We could not corroborate the findings through other data sources, such as radiology databases, immunization registries, or billing data.
Although this study is the first to report the impact of electronic SO for a broad set of measures, the practice development and team approach implemented is consistent with the widely promoted patient-centered medical home model.34 Further study is warranted to test the effectiveness of the screening and immunization measures in a larger sample of practices. Additional work also is needed to identify which chronic care management measures are best suited for inclusion in future SO protocols. Through wider adoption of electronic SO protocols, primary care practices may be better able to meet the expected demand for preventive services with the implementation of the Patient Protection and Affordable Care Act.35
Conclusions
This study established the feasibility of implementing an electronic SO protocol in a sample of small to medium sized primary care practices that participated in a QI study and received facilitation and some support from PPRNet. Practices successfully adapted EHR tools, and staff in these practices embraced the protocol with the support of leaders without significant time burdens. Technical competence and leadership are needed to optimally adapt and use EHR reminder tools and to help staff adopt new roles and overcome barriers.
Acknowledgments
The authors appreciate the practices that participated in the SO-TRIP study and thank Loraine Roylance, MA, who served as research coordinator.
Appendix: Brief Case Studies of 8 Practices Implementing Electronic SOs
Each practice had a unique experience in the project, based on their individual contexts and relevant strengths and weaknesses. Brief case studies of each practice are presented here.
Missouri
This family medicine practice comprised 4 providers and made significant improvements on all measures. They made extensive plans and followed through on most actions to accomplish major improvements in performance. The plans included applying HMTs, adapting nursing note documentation of SOs, and template changes; disseminating generic and patient-specific handouts; posting patient information on practice web sites and bulletin boards; providing staff education monthly and happy hours; and improving the practice's stock of vaccines. This practice had a technically savvy physician leader, and nursing staff were competitive and wanted the practice to do well on this project; it helped that there were strong interpersonal relationships within this practice. Some financial barriers were initially perceived related to the reimbursement for certain vaccines, but the practice developed an alternative mechanism for patients to receive Zostavax (Merck, Whitehouse Station, NJ) from community pharmacies for administration in the office. Staff turnover in this practice resulted in the hiring of a new medical assistant (MA) who supported the SO approach to primary care practice.
Connecticut
This 4-provider family medicine practice made significant improvements on most measures. They implemented the SO-TRIP measures in stages, starting with immunizations and mammograms, adding bone density, urinary microalbumin and glycosylated hemoglobin, and later lipid profiles. The front desk staff provided the patient with a “patient information update” form at check-in (using patient data recorded within the patient's HMT; the patient was asked to update form in the waiting room). They had MAs review the form, address, and document the relevant services due and their actions on these, using an MA note template that updated the HMT. All the MAs bought into the process and goals of the SO-TRIP project and the physicians agreed that they were too busy to always pay attention to these HM goals. The physicians changed the way notes were opened and referred to the MA note and update form. In this state MAs were not permitted to administer any vaccines, so the physicians administered the immunizations when needed. They felt the MA note template was the most helpful change they implemented.
North Carolina
This 14-provider family medicine practice with 65 employees improved in all measures of the SO-TRIP project. The practice administrator introduced the project to the providers, gaining their support and requiring sign off on a policy regarding implementation of electronic SOs. The medical director and clinical nursing supervisor educated staff and set organizational expectations by rounding with the staff on a regular basis. Staff understood that this was their responsibility and worked in teams with respective providers to maintain continuity of care. Staff recorded their actions on the SO-TRIP measures directly on the HM table. The practice was involved in several pay for performance initiatives and was focused on quality and evidence-based practice. They met regularly with the staff to work through any barriers and ensure staff were educated and working as a team, and after learning of another practice's efforts to address all overdue items on the HM template, they embraced a “Get the Red Out” (which refers to addressing all the overdue HM items appearing in red on the HMT) campaign after the second network meeting. The practice addressed all the goals of the SO-TRIP project despite having some technical issues with their data extracts, which limited the receipt of performance reports until October 2009, and some concerns regarding cost of some vaccines.
Washington
This 11-provider, multispecialty primary care group (family medicine, internal medicine, and obstetrics and gynecology) had a bit of a slow start in their efforts to implement the project but made rapid improvements during the last half of the project. They had initial technical problems with their EHR and had little faith that the HM data were correct, and they lacked clear leadership to resolve the problems. Efforts to address their technical issues stalled until the second site visit, but a clear plan emerged at the conclusion. The technical issues were addressed by modifying their HMT. A nursing supervisor was appointed as the project liaison with the medical director, who each spearheaded efforts to assist the providers and nursing staff to integrate the SO-TRIP goals into their daily operations, documenting SO-TRIP measures directly on the HM table. Regular team huddles and staff meetings were used to keep the agenda focused on implementing the electronic SOs. Significant improvements in most measures were noted, and the practice won a State Medical Society quality award for this project.
Idaho
This family practice of 2 providers had 2 young and enthusiastic MAs that made improvements on the majority of measures. They were quickly able to see how the use of the electronic SOs would lead to better care. Consistent with the philosophy of the physicians, they simply followed through as requested. Templates were quickly applied. MAs reviewed HMs for all patients at the visit, checked records were complete, and directly updated HM table as needed, ensuring the SOs were conducted during the planned visit. Staff meetings were used to follow their progress with performance reports on the project and revise plans as needed. They felt the project was easy and improved their efficiency. Some reimbursement issues related to some of the vaccines were noted, and a perception that the economic impact of the recession in their area might be related to some patients' refusal of services. Success was related to overcoming inertia through a simple reminder they posted on the computers to check the HM template.
Maryland
This 3-provider family practice had a group of MAs who were more skeptical regarding their ability to actually follow through on SOs but made modest improvements on the measures. They implemented an update form for patients to use at check-in to update any needed services. Nursing staff were inconsistent with the actual follow-through on all the SOs using direct entry on the HM table, although one MA clearly felt empowered to do so in most cases except for mammograms. The providers had multiple competing issues that seemed to limit their focus on this project, but they believed that the project caused them as providers to be more tuned in to the patients' overdue HM.
Texas
This practice was a health care group (mixed group of physicians and nurse practitioners) with a staff of 145 supporting 25 providers in 3 office locations. They made minimal improvements in the measures. Training, turnover, and retraining were constant barriers to this practice, as was the need to establish specific champions to lead the project in each of the 3 clinic locations. They used a combination of nursing note templates and direct entry on the HM table to address the SO-TRIP measures. Some of the staff seemed to be more empowered as a result of the project and more actively involved in patient care. This practice focused on improvement in immunizations, mammograms, and bone density and did not really address any of the lipid or diabetes measures.
New York
This internal medicine practice moved to a new facility in the middle of the project and did not improve on the measures within this project. Four primary care providers within the practice comprised the group, which agreed to participate in the project with a stable nursing staff that had worked for many years within this practice. After merging with another group of internists/specialists during the last 6 months of the project, the number of providers increased to 11. Lack of success was due to a number of interpersonal issues within the practice, no nursing champion, and a probable lack of leadership focus on this project during the design and building of a new facility.
Notes
This article was externally peer reviewed.
Funding: This study was funded by the Agency for Health care Research and Quality (AHRQ) through Contract No. HHSA290200710015, Task Order #2.
Prior presentation: Preliminary findings of this research were presented at the Agency for Healthcare Research and Quality 2010 Annual Conference, September 29, in Bethesda, MD, and as a poster at Academy Health 2010, June 27 to June 29, in Boston, MA.
Conflict of interest: none declared.
↵* PPRNet previously stood for Practice Partner Research Network, which is now known only as PPRNet.
- Received for publication July 7, 2011.
- Revision received December 19, 2011.
- Accepted for publication January 3, 2012.