Abstract
Background:The government is encouraging the adoption of electronic medical records (EMRs). There is little information about using EMRs in the obstetric literature and none about using them in family medicine residencies. Our purpose was to assess if using an EMR was associated with improvement in the ordering and availability of prenatal tests.
Methods:A retrospective chart review comparing the rate at which prenatal laboratory values were present on the chart, ordered on time, and recorded on a prenatal flow sheet.
Results:Comparison of charts before and after implementation of an EMR showed statistically significant improvement in the percent of patients with all first trimester (87.5% vs 96.0%; P = .0025), quadruple screening tests (91.1% vs 98.1%; P = .012), and second trimester screening results (93.5% vs 100%; P = .044) in their charts; first trimester laboratory tests (91.6% vs 99.5%; P = .001) and second trimester ultrasounds (90.9% vs 97.3%; P = .027) being ordered on time; and first trimester results (88.2% vs 95.5%; P = .009), quad screen results (93.1% vs 98.0%; P = .0495), and second trimester ultrasounds (93.5% vs 100%; P = .003) being recorded on the American Congress of Obstetricians and Gynecologists flow sheet.
Conclusion:Adopting an EMR was associated with an improved rate at which prenatal tests were ordered on time, present on the chart, and recorded on a prenatal flow sheet.
The use of electronic medical records (EMRs) is a highly debated topic in the larger debate surrounding health care reform. President Obama has included $19 billion in his stimulus plan for EMRs with the belief that implementation of EMRs across the nation will lower administrative costs, streamline patient care, improve surveillance, and increase efficiency. Through the American Recovery and Reinvestment Act of 2009, the stimulus plan provides a $44, 000 incentive for health care professionals using EMRs that meet certain guidelines.1 In general, there are two categories of EMR systems under which physician practices fall. Basic systems have the following functionalities: patient demographic information, patient problem lists, clinical notes, orders for prescriptions, and viewing of laboratory and imaging reports. Fully functional systems contain basic system capabilities as well as medical history/follow-up, electronic prescribing and test ordering, warnings of drug interactions or contraindications, highlighting of out-of-range test levels, and reminders for guideline-based interventions. Despite their proposed benefits, adoption of EMRs has been slow in the United States; only 4% to 6.3% of physicians have adopted comprehensive EMRs and 13% to 20.5% have opted for basic systems.2,3
There is very little in the medical literature regarding the use of EMRs in obstetric care, and most of it relates to inpatient care in a labor and delivery setting. A 2005 study showed that EMRs improved communication among prenatal and labor and delivery providers, 4 although no obstetrical outcome data were reported. Another study in 2008 showed that EMR use improved completeness of documentation in the labor and delivery unit without compromising patient care.5 In 2007, Haberman et al6 showed that compliance with documentation of estimated fetal weight, pelvic adequacy, and fetal position was low when prompts and reminders in the electronic medical record were deactivated. A follow-up study7 in 2009 showed that automated prompts built into the EMR increased the documentation for estimated fetal weight and indications for labor induction. Other items in the obstetric literature include review articles, 8 personal testimonies, 9 discussions of the theoretical benefit of electronic records over paper records, 10 opinion pieces, 11–13 and descriptions of EMR implementation.14 We found no studies about the use of EMRs for prenatal care in a family medicine residency setting.
When our residency program used paper charts, a frequent, anecdotal complaint from our obstetrical faculty and consultants was that prenatal laboratory results were missing from the chart, were not ordered on time, were not transferred to the American Congress of Obstetricians and Gynecologists (ACOG) flow sheet, or some combination of the three. There are some data that show that 19% of medical errors in family medicine are related to the logistics, timing, and follow-up of laboratory results.15 From July to November 2005 our residency program transitioned to a full-function EMR (Centricity, GE Healthcare, Waukesha, WI). We accessed the EMR on laptop computers via a wireless network and used it for all clinical documentation after implementation was completed in November 2005. Physicians documented in the EMR in real time while seeing patients. Whenever possible, we received test results electronically through the EMR. When this was not possible we scanned paper results into the EMR as document attachments, and the physicians manually entered the results into the EMR database. When we were planning our conversion to the EMR, we decided to perform a chart review study to see if the EMR helped alleviate the perceived problems that occurred with paper charts: missing results, untimely orders, and/or the lack of transfer of results to the ACOG flow sheet. Embedded within the EMR was a prenatal flow sheet modeled after the paper ACOG flow sheet, which was updated automatically from information in the EMR database. Except the implementation of our EMR, we had no significant changes in our office's processes relating to prenatal care during the study period.
Methods
JFK Family Medicine Residency is a suburban community hospital-based program in central New Jersey in the greater Newark-New York City area. It is a 3-year unopposed program with 6 residents per year. All the residents and one of the faculty performed prenatal care during the study period. There were no midlevel providers in the program. One family medicine physician and a team of obstetricians guided the residents' prenatal care. The residency serves an ethnically diverse community consisting primarily of white, Asian-Indian, and Latino patients but also includes a wide variety of African-American, Caribbean, and East Asian ethnic groups. Our office was responsible for 395 deliveries in 2005 and 336 deliveries in 2008.
During a 3-week period in March/April 2005, we retrospectively reviewed the paper charts of all patients who had physician appointments at our office for prenatal care during that period. If a patient had two or more appointments during this period, we reviewed her chart only once for the purposes of this study. We collected each patient's age and gestational age based on her estimated delivery date and then reviewed the patient's chart as described below.
First we checked to see if the results of recommended prenatal studies were in the chart. The results of tests that are part of an initial prenatal profile—complete blood count (CBC), blood type, antibody screen, rubella titers, rapid plasma reagin (RPR), urine culture, hepatitis B surface antigen (HBsAg), human immunodeficiency virus (HIV) status, hemoglobin electrophoresis, and cystic fibrosis screening—were expected to be in the chart by the time of the second visit because all of our prenatal patients have an intake visit with a nurse who obtains samples for these tests. We chose the second visit because some patients required a visit for acute obstetrical issues such as threatened abortion before having their initial nurse intake visit. After these acute visits, the patients were referred for a nurse intake visit if appropriate, after which they had an appointment during which a physician performed an initial complete obstetric evaluation. For other tests done during the pregnancy, we expected the results to be in the chart within a reasonable amount of time after they were performed: 2 days for blood tests, 4 days for cultures, and 1 week for ultrasounds. We recorded whether laboratory results were present or absent and/or if tests were refused or not applicable based on the patient's gestational age.
Second, we determined if the tests had been ordered on time. We expected first trimester laboratory tests—CBC, blood type, antibody screen, rubella titers, RPR, urine culture, HBsAg, HIV, Papanicolaou smear, gonorrhea and chlamydia cultures, hemoglobin electrophoresis, and cystic fibrosis screening—to be ordered by the date of the second visit for the reasons stated above. Quadruple screen testing (for α-fetoprotein, human chorionic gonadotropin, estradiol, and inhibin-A) should be done between 15 and 22 weeks' gestational age. The second trimester ultrasound for anatomy should be done between 18 and 22 weeks' gestational age. This is the local standard of care in our geographic area even though second trimester ultrasound examination is not absolutely recommended for all patients.16 Second trimester screening—1-hour glucose tolerance test, CBC, and antibody screen—should be done between 24 and 28 weeks' gestational age. Group B streptococcal culture should be obtained between 35 and 37 weeks' gestational age. For each test we recorded whether the test was ordered on time, was not ordered on time, was refused, or was not applicable based on the patient's gestational age.
Third, we determined if the test results had been transferred to the ACOG flow sheet. Using a paper chart required a conscious effort to transfer by hand the information from laboratory reports to the ACOG flow sheet, but with our EMR that process occurred automatically for some of the laboratory reports we received electronically. Other laboratory results, although received electronically, did not cross into our EMR database in a meaningful way and thus had to be entered manually to appear as meaningful data in our ACOG flow sheet. This was a limitation of the software the laboratories used to transmit results and not of the EMR. We received ultrasound reports both on paper and electronically as a single large text report, and pertinent data from those reports had to be entered manually as discrete items into the EMR database. Ensuring the flow sheet was kept up to date was encouraged and monitored through precepting and chart review. For each test we recorded whether the test result was on the flow sheet, was not on the flow sheet, was refused, or was not applicable based on the patient's gestational age.
During a 3-week period in January 2008, we retrospectively reviewed the EMR charts of all patients who had physician appointments at our office for prenatal care during that period. If a patient had two or more appointments during this period, we reviewed her chart only once for the purposes of this study. We collected each patient's age and gestational age based on her estimated delivery date and reviewed the patient's chart using the same criteria as the review of the paper charts. The residents that did the chart reviews in 2005 were different from the residents that did the chart reviews in 2008.
We entered data into an Excel spreadsheet (Microsoft, Redmond, WA) and then transferred it to SPSS statistical software (version 17.0, IBM Corp., Somers, NY) for analysis. To simplify data analysis, we grouped results to allow for analysis of tests that were ordered together, that involved a specific method of collection, or that involved obtaining ultrasound studies. The groups were defined as follows:
Group 1: blood group (A, B, or O), Rh type, antibody screen, CBC, rubella, RPR, urine culture, HBsAg, HIV, hemoglobin electrophoresis, and cystic fibrosis screening.
Group 2: quadruple screen (α-fetoprotein, human chorionic gonadotropin, estradiol, and inhibin-A).
Group 3: CBC, antibody screen, and 1-hour glucose tolerance test.
Group 4: Group B streptococcal culture.
Group 5: Papanicolaou smear and gonorrhea/chlamydia testing.
Group 6: Second trimester ultrasound for fetal anatomy.
Patients with no data based on gestational age for any test within any group were excluded from analysis of that particular group. Statistical analysis for completeness proportions was based on the presence or absence of laboratory test results in fields.
All statistical analysis was conducted using a two proportion Z-test to obtain Pearson χ2 values or Fisher's exact tests at α = 0.05. Furthermore, one-tailed P values were calculated for directional determination. Given the 3 categories of interest (on chart, on time, and ACOG flow sheet), we sought to examine if there was any significant increase in the proportion of completeness for groups 1 to 6. The benchmark of 100% completeness was measured in all 6 groups for the 3 categories.
The study received approval from institutional review boards at both JFK Medical Center and University of Medicine and Dentistry of New Jersey-Robert Wood Johnson Hospital.
Results
Based on the volume of visits during the 2 study periods, our original sample population of pregnant women included a total of 138 patients before EMR implementation and 202 patients after EMR implementation. Two patients from the group before the EMR were not included in the analysis because their gestational ages were not recorded (Table 1). We only analyzed patients for tests that should have been done at the time of review based on their gestational age. For example, if the patient's gestational age was 33 weeks at the time of the chart review, we would only include data pertaining to studies that should have been completed by that point, such as groups 1, 2, 3, 5, and 6 but not group 4 (group B streptococcal testing), which was done at 35 to 37 weeks’ gestation. Therefore, the entire population was not included in each analysis (Table 2).
For the on chart category, the patients after implementation of the EMR were more likely than the patients before implementation of the EMR to have 100% of the tests present in group 1, group 2, and group 3 (Figure 1). The tests in these groups were blood or urine tests ordered by physicians but obtained by other office personnel. Groups 4, 5, and 6, although not statistically significant, did show an increase in the percent complete.
For the on time category, the patients after implementation of the EMR were more likely than the patients before implementation of the EMR to have 100% of the tests present in group 1 and group 6. The percent of patients with 100% of tests present actually decreased for groups 3 and 5 among patients after implementation of the EMR, although not significantly (Figure 2).
The ACOG flow sheet category showed a statistically significant increase among the patients after implementation of the EMR for group 1, group 2, and group 6 (Figure 3). Groups 3 and 4 yielded an increase in percent of patients with complete records, but it was not statistically significant. Group 5 showed a slight decrease that was not statistically significant.
There were no significant differences in groups 4 and 5 in any of the three categories (Figures 1–3).
Discussion
An unanticipated finding was that our performance before the EMR was better than we expected, with the lowest completion rate being 87.5% for first trimester laboratory reports being on the chart (Figure 1). Of the 18 separate data sets we looked at, only 2 were below 90%. This was pleasantly surprising and disproportionate to the number of complaints we heard from our obstetrical faculty.
When we compared data before the EMR with data after the EMR, we found that after converting to an EMR a significantly higher percentage of patients had 100% of the results of first trimester blood/urine tests and second trimester screening tests on their charts. The first trimester tests were also ordered on time significantly more often, and the quadruple screen results were recorded on the EMR's ACOG flow sheet significantly more often. We also showed that results of second trimester ultrasounds for anatomy were done on time significantly more often and the results were more consistently on the ACOG flow sheet. Having information available at the right time and in the right place is critical for medical decision making and guiding medical management.
Interestingly, there were no differences in all categories and analyses between groups 4 and 5, which included tests that relied on physicians collecting a specimen as opposed to blood/urine tests or ultrasounds. Group 4 (group B streptococcal culture) and group 5 (Papanicolaou smear, gonorrhea and chlamydia cultures) were kept separate because they are collected at different times during the pregnancy. When we used paper charts, these tests were ordered on time, were on the chart, and were on the ACOG flow sheet at least 95% of the time, with the exception of group B streptococcal results being present on the ACOG flow sheet (92.6%). Achieving statistical significance in these categories would have required a much larger sample size or completeness approaching 100%. The number of patients who had group B streptococcal cultures was also small because of the small number of patients at the appropriate gestational age to have had this test done. This did not allow us to perform a complete statistical analysis of this group.
Another way of looking at the data before and after implementation of the EMR is that we made significant improvements in on-time ordering and availability on the chart and flow sheet of blood/urine tests collected by a nonphysician. During the study period, other than the EMR, our office made no conscious changes to the patient care processes for prenatal patients. Alternatively, there may have been unrecognized and unstudied patient care factors that occurred in the almost 3 years between the data collection times that caused preceptors and physicians to be more diligent in making sure prenatal patients had recommended tests ordered on time and placed on the chart and the ACOG flow sheet. That there may have been observer bias because the residents were not blinded to the fact that the study was occurring may be an additional factor for consideration.
Another important factor in our results, outside of direct physician ordering of laboratory and other tests, is that our EMR allowed us to electronically track the results of blood tests, urine tests, cultures, and Papanicolaou smears that were sent from our office. Virtually all of our prenatal patients have their laboratory specimens obtained in our office. All the test orders are entered electronically in our EMR, which allows our office staff to track the results electronically to ensure that we receive results for any test that was sent from our office. When we used paper charts, this process was done on paper and was considerably more cumbersome and theoretically more prone to error. It is possible that fewer tests results were lost using this electronic tracking system and led to greater availability of test results, although we never specifically studied that issue.
Our study had several limitations. First, we did not collect data regarding the potential impact on obstetric outcomes such as birth weights or rates of Cesarean sections and preterm delivery. We did not have sufficient resources to gather enough data to perform such an analysis, but it is certainly an area for future study given the lack of obstetrical outcome data in prior studies.4–7
Second, our study population was small, which limited the power of our analysis. We hope this study encourages more research on larger populations and possibly at multiple centers in different settings.
Third, we did not use computer-generated prompts (ie, pop-up boxes) for ordering tests. Our EMR did not allow for such prompts based on a patient's gestational age. We did, however, use electronically entered order sets that we constructed to order the appropriate office visit code, follow-up visit times, and laboratory tests at the end of a visit at a given gestational age with a single mouse click. The only other mechanisms for ensuring that laboratory tests were ordered on time were the physicians' knowledge and reminders from preceptors. If automated prompts based on gestational age had been available, we might have seen the numbers after implementation of the EMR improve more relative to the numbers before the EMR. An obstetric practice considering an EMR would be wise to look for this feature.
Fourth, some test results—received electronically or on paper—still required hand entry into the EMR database. Once they were entered into the database, however, they did appear in the ACOG flow sheet embedded within our EMR. This brings up an important issue regarding choosing an EMR and setting up interfaces with laboratories to receive results electronically and in a meaningful fashion. Our study did not look at other EMRs, and therefore our results cannot be generalized to other systems that may or may not have the same issues we encountered with laboratory interfaces, and other systems certainly may have other laboratory interface problems we did not encounter. If electronic laboratory interfaces and cross-reference tables are constructed appropriately, the availability of laboratory results in the chart and on a flow sheet should approach 100%, and the only deficiencies should come from patients not having tests performed, misplaced laboratory specimens, or missing results. Therefore, the percentages of laboratory results being on the chart and in the ACOG flow sheet in our study may be less than they could be in an ideal world.
Finally, our performance before the EMR was better than we expected, which limited the ability to find more statistically significant differences between all groups before and after the EMR given the relatively limited sample size.
Conclusion
After implementing an EMR there was a greater likelihood that prenatal patients had first trimester, quadruple screening, and second trimester screening laboratory results on their charts. In addition, first trimester laboratory tests were more likely to be ordered on time, and first trimester results and quadruple screening results were more likely to be recorded on the ACOG flow sheet. Second trimester ultrasounds also were more likely to be ordered on time and the results were more likely to be recorded on the ACOG flow sheet. Implementation of EMRs has been shown to improve communication between prenatal providers and labor and delivery units.4 This indicates that having information more readily available may lead to better decision making and better obstetrical outcomes. This would be an important area of evidence-based research that hopefully will grow out of broader acceptance and use of EMRs in the coming years.
Acknowledgments
The authors acknowledge Drs. Rachelle Guinto, MD, Anita Jasani, MD, Lara Mor-Zilberstein, MD, Nimesh Patel, MD, and Sonal Patel, MD, for data collection; Rebecca Vanness for assistance with manuscript preparation; and Rizie Kumar, MA, for assistance with statistics.
Notes
This article was externally peer reviewed.
Funding: none.
Conflict of interest: none declared.
- Received for publication December 16, 2010.
- Revision received February 18, 2011.
- Accepted for publication February 28, 2011.