Abstract
Introduction: Patients at risk for suicide often come into contact with primary care providers, many of whom use electronic health records (EHRs) for charting. It is not known, however, how often suicide ideation or attempts are documented in EHRs.
Methods: We used retrospective analyses of de-identified EHR data from a distributed health network of primary care organizations to estimate the frequency of using diagnostic codes to record suicidal ideation and attempts. Data came from 3 sources: a clinician notes field processed using natural language processing; a suicidal ideation item on a patient-reported depression severity instrument (9-item Patient Health Questionnaire [PHQ-9]); and diagnostic codes from the EHR.
Results: Only 3% of patients with an indication of suicidal ideation in the notes field had a corresponding International Classification of Diseases, 9th Revision (ICD-9), code (κ = 0.036). Agreement between an indication of suicidal ideation from item 9 of the PHQ-9 and an ICD-9 code was slightly higher (κ = 0.068). Suicide attempt indicated in the notes field was more likely to be recorded using an ICD-9 code (19%; κ = 0.18).
Conclusions: Few cases of suicidal ideation and attempt were documented in patients' EHRs using diagnostic codes. Increased documentation of suicidal ideation and behaviors in patients' EHRs may improve their monitoring in the health care system.
Despite the fact that most people who attempt suicide are treated in a hospital or emergency department setting,1,2 many patients at risk for suicide have had recent contact with their clinicians in a primary care setting. Nearly one half of patients who died from suicide saw their primary care physician (PCP) during the month before their death.3 Patients with depression disorders have a demonstrated risk factor for suicide and are commonly treated in primary care.4,5 This suicide risk may manifest as suicidal ideation (ie, thinking about, considering, or planning suicide) or suicide attempt. Therefore, monitoring primary care patients at the point of care for risk of suicidal ideation and behaviors may have the potential to decrease suicide attempts.
Monitoring at-risk primary care patients is becoming more mainstream after the passage of the Health Information Technology for Economic and Clinical Health Act (HITECH) and subsequent uptake of electronic health records (EHRs).6 The first stage of HITECH (2011 to 2012) focused on capturing and sharing clinical data and using those data to monitor clinical conditions. EHR “data capture” includes recording diagnostic codes, documenting conditions in clinical notes, and integrating patient-reported outcomes, thus increasing EHR documentation that can increase a primary care practice's ability to monitor and track patients' progression and treatment response.
As a matter of quality and benchmarking (such as the National Committee for Quality Assurance Health care Effectiveness Data and Information Set, used by >90% of US health plans as a tool to measure performance on important dimensions of care and service), many primary care practices implement universal screening for major risk factors such as smoking, alcohol/substance abuse, pain, falls, and depression symptoms. Patients at risk for suicide thus come into contact with PCPs who may collect information on these risk factors. Likewise, many PCPs use EHRs for charting. However, how often suicide ideation or attempts are documented in the EHRs for monitoring or coordination of care purposes is not known. The most visible and easily accessible form of documentation in an EHR is a recorded diagnostic code using the International Classification of Diseases, 9th or 10th Revision (ICD-9 or ICD-10, respectively). However, a prior study estimated that only 25% of suicidal patients seen in the emergency department had any kind of suicidal thinking or behavior documented in their medical chart.7
The objective of this study was to estimate the use of diagnostic codes in EHRs to document suicidal ideation and attempt among patients seen in primary care. We retrospectively identified primary care patients who indicated thoughts of death or hurting themselves on a patient-reported depression severity instrument, the 9-item Patient Health Questionnaire (PHQ-9), as well as patients whose clinician indicated suicidal ideation or suicide attempt in the notes field of their EHR. Among these patients, we calculated the percentage with a corresponding diagnostic code for suicidal ideation or attempt recorded in their EHR.
Methods
Setting and Subjects
This study leveraged eNQUIRENet, a distributed health data network formerly known as DARTNet, that links EHR data across multiple nonintegrated primary care clinical organizations representing >3 million patients and 1700 clinicians.8 The EHR data reside locally at each practice and are accessed through a data repository designed for clinical decision support, the CINA Clinical Data Repository (CDR; http://www.cina-us.com/technology.html). The CINA CDR standardizes EHR data elements from multiple locations and disparate native codes to eNQUIRENet data standards, which have been previously described.9 Variables included in the CDR include patient age and sex, ICD-9 coded diagnoses from the problems list, a clinician-populated notes field, and medications ordered. Patient-reported measures, such as the PHQ-9, were merged with each patient's EHR when available.
A retrospective cohort of 61,464 patients with a new episode of depression was identified from 14 eNQUIRENet practices for a prior study of depression severity and suicide.10 These patients were identified using prescription and diagnostic records in their EHR, along with the following criteria specified by the Health care Effectiveness Data and Information Set11,12: (1) an ICD-9 code indicating depression (major depression [296.2, 296.3], dysthymia [300.4], or depression, not otherwise specified [311]); (2) no depression codes during the 120 days before diagnosis; and (3) no antidepressant prescriptions written during the 90 days before diagnosis.
From the cohort of 61,464 patients, this study included a subcohort of 32,385 patients from 8 of the 14 practices known to use the PHQ-9 and willing to provide data from the History of Present Illness (HPI) notes field, a clinician-populated text field in the EHR. Four subsets based on the availability of patients' data from the HPI notes field and/or PHQ-9 then were identified for the current analyses: (1) 15,761 patients with text data from their HPI notes field(s) that was successfully processed using a natural language processing (NLP) system, which is described below (HPI subset); (2) 1,013 patients with a recorded response from the 9th item of the PHQ-9, which asks about thoughts of death or hurting oneself (PHQ-9 subset); (3) 16,289 patients with HPI data or PHQ-9 item 9 data (HPI or PHQ-9 subset); and (4) 485 patients with both HPI data and PHQ-9 item 9 data (both HPI and PHQ-9 subset) (Figure 1). We obtained data from 1998 to 2011.
The 8 primary care practices that provided data for this study ranged in size from 2,900 patients to >121,000 patients. Among the 4 subsets of patients, the mean age ranged from 53 to 60 years old; 67% were women. These demographic characteristics were comparable with those of the overall cohort (n = 32,385) from which the subsets were created (Table 1). To assess potential bias introduced by excluding patients from subset analyses, we compared age and sex between patients included in each subset and patients not included in each subset. Across all 4 subsets, patients included in that subset were significantly younger than patients not included in that subset (P < .01). For example, patients from the overall cohort with HPI data (n = 15,761) were significantly younger compared with patients from the overall cohort with no HPI data (n = 16,624; P < .01). Approval for this study was obtained by the University of Colorado Multiple Institutional Review Board, the American Academy of Family Physicians Institutional Review Board, and each local organization.
Data Sources and Measures
The key domains of interest are risk factors for suicide, measured as suicidal ideation (including thoughts of death or hurting oneself) or suicide attempt. Suicidal ideation was captured from 3 sources (each is described below in detail): (1) diagnostic codes in the EHR; (2) the HPI notes field in the EHR; and (3) the patient-reported PHQ-9 collected at the point of care. Suicide attempt was captured from ICD-9 codes in the EHR and the HPI notes field in the EHR.
Diagnostic Codes from the EHR
The Problems field of the EHR was searched for occurrences of the following ICD-9 diagnostic codes indicating suicide attempt or ideation: E950–959 (attempt) and V62.84 (ideation). These codes are from the framework of groupings for self-inflicted injury recommended by the Centers for Disease Control and Prevention.13
HPI Field of the EHR
The HPI text field of the EHR is used by clinicians to enter notes regarding a patient's health history and current health status during a visit. A rule-based NLP system was developed to search for current positive mention or negation of suicidal ideation or suicide attempt using a thorough list of key words (eg, suic, self-harm, hang, cut), 4 primary rules (±attempt, ±ideation), and 7 common linguistic patterns. An iterative manual review compared the HPI notes field to NLP results for a random sample of 1,839 records from 4 clinical sites. Sensitivity of the NLP system to detect suicidal ideation or behavior ranged from 0.905 to 0.942; specificity ranged from 0.911 to 0.964. Personal health information in all HPI fields were de-identified and then processed using the NLP system, resulting in a determination of the current presence of suicidal ideation or suicide attempt for each patient.
Patient-Reported PHQ-9
The PHQ-9 is a 9-item instrument commonly used to measure depression severity among primary care patients.14 Item 9 of the PHQ-9 asks the patient whether they have had “thoughts that they would be better off dead or of hurting themselves in some way” in the past 2 weeks, with possible responses being 0 (none), 1 (several days), 2 (more than half the days), or 3 (nearly every day). Prior work supports a strong correlation between endorsement of item 9 and suicide risk.15,16 A recent analysis of 207,265 completed PHQ-9 for 84,418 patients estimated that those who indicated “more than half the days” or “nearly every day” for item 9 were 4.12 to 6.37 times more likely to have a subsequent suicide attempt or death by suicide.15 The same study found that the 13% of patients who indicated thoughts of death or hurting themselves “more than half the days” or more represented >50% of suicide attempts and deaths. Therefore, for the purposes of this study we chose to define suicidal ideation as a response of 2 (more than half the days) or higher.
All 8 practices from which data were available for this study were regularly administering the PHQ-9 to their patients; specifically, practices were electronically prompted to collect a PHQ-9 at specific intervals for specific patients based on depression diagnoses, other diagnoses used for depression in a given practice, and length of time since the last PHQ-9 was completed. Data were collected using printed forms completed by the patient in the waiting room or by nurses helping the patient, as well as online systems used by nurses or clinicians. Total PHQ-9 scores and item 9 scores were abstracted, standardized, and linked to the CDR.
Statistical Analysis
Within each of the 4 subsets, patients with an indication of suicidal ideation according to the data source(s) used to define that subset were identified. Among those patients, the percentage with a corresponding ICD-9 code for suicidal ideation was calculated. Agreement between the data source(s) and ICD-9 code also was estimated within each subset using the κ statistic. All analyses were completed using SAS software version 9.3 (SAS Institute, Cary, NC).
Results
Among the patients with NLP-processed HPI data (n = 15,761), 1,025 had an indication of suicidal ideation recorded in their HPI note(s) by their clinician; of these, a mere 3% (n = 30) had a corresponding ICD-9 code indicating suicidal ideation recorded in their EHR (κ = 0.036; Table 2). Agreement between an indication of suicidal ideation from item 9 of the PHQ-9 and an ICD-9 code was slightly higher (κ = 0.068). Having an indication of suicidal ideation from the HPI notes or PHQ-9 item 9 did not increase agreement with an ICD-9 code for suicidal ideation (κ = 0.04). No patients had an indication of suicidal ideation from both the HPI notes field and item 9 of the PHQ-9.
Suicide attempt was more likely than suicidal ideation to be recorded using an ICD-9 code. Among the 15,761 patients with NLP-processed HPI notes, 86 had an indication of attempt documented by their clinician in the HPI notes, of which 19% (n = 16) had a corresponding diagnostic code for suicide attempt recorded in their EHR (κ = 0.18).
Discussion
This study found that few cases of suicidal ideation and suicide attempt as documented in a primary care setting using a clinician's notes field or a patient-reported PHQ-9 were also documented in the patient's EHR using diagnostic codes. Agreement between the ICD-9 code for suicidal ideation and suicidal ideation documented in the clinician notes field or indicated by the PHQ-9 was low (κ = 0.036 and 0.068, respectively). Suicide attempt documented in the notes field more often had a corresponding diagnostic code (20%; κ = 0.18). Better documentation in the EHR of both suicidal ideation and attempt for primary care patients would increase the ease with which those patients could be monitored for suicide prevention and could improve the data available for observational patient-centered outcomes research. Such documentation would include both diagnostic codes of suicidal ideation or suicide attempt when appropriate, as well as clarification in the notes field that an affirmative response to the 9th item of the PHQ-9 was discussed with the patient and found to not be of immediate concern.
This study used cutting-edge processing tools for text fields in clinic notes, a critical piece of research using EHR data sources. Although technological advances such as NLP can be used to search clinician notes fields for mentions of suicidal ideation or behaviors to monitor primary care patients with these risk factors, it is not the ideal solution because of the imprecision and associated resource burden. In this study over 50% of the HPI data records could not be processed through the NLP algorithm, even after a fair amount of data cleaning and algorithm development. Other measures of suicidal ideation, such as the 9th item of the PHQ-9, are oftentimes not used to their full potential because a given practice may not record the response to the individual item but instead only record the total score for the PHQ-9. This study included 1,013 patients with a recorded score for the 9th item from the PHQ-9; this was a small percentage of all patients, largely because many practices did not record the individual scores of specific items.
Increased use of diagnostic codes to document suicide risk factors in primary care can improve the ability to use EHR data for observational research. The use of EHR data has increased substantially in recent years.17,18 However, observational research that relies on EHR data to study suicide-related outcomes will grossly underestimate both ideation and attempt if it uses only the ICD-9 codes to identify cases. In addition, researchers cannot rely on measures of ideation from instruments such as the PHQ-9 to always be available in the EHR, and they may not always have access to clinician notes fields or the technology to mine those fields for usable data.
There were several limitations to this study. We determined that patients included in the subsets for final analyses were significantly younger than patients not included in the final analyses, indicating the results may be less generalizable to older patient populations. The cohort of patients included is also a convenience sample because data were drawn from the 8 primary care practices with the largest available amounts of valid PHQ-9 data and were not based on any underlying population factors. The availability of more PHQ-9 data in these practices may indicate the clinicians have a higher awareness of depression and suicide, which in turn may indicate they ask patients about suicidality more often. This increased awareness may also result in a higher mention of suicide in the HPI notes, although one cannot know the impetus for a clinician statement about the presence or absence of risk (whether it was based on a response to a direct question or a result of a validated patient-reported scale that was not recorded). It would not, however, explain the low occurrence of corresponding ICD-9 codes for patients with suicidal ideation or attempt.
As mentioned above, over 50% of patients in the original cohort (16,624 of 32,385) had HPI data that were not able to be processed through our NLP system. Reasons for this included empty HPI fields or unclean data. HPI notes fields may be empty or may include unusable information for a variety of reasons: clinicians may not use the field at all or may use the field only when there is something of clinical importance to report, or data may become corrupt during the data transfer process. The exact impact this moderate rate of NLP processing had on our results is unknown. If these unprocessed fields included indications of suicidal ideation or suicide attempt, our estimates of corresponding ICD-9 code documentation may have been underestimated or overestimated. At the practice level, this moderate rate of unprocessed HPI data may support the use of notes fields whenever possible, particularly if there is no other way to capture clinical observations in an EHR.
Further, there is likely to be at least some degree of underreporting or “undercoding” of suicidal ideation or attempt using diagnostic codes because of various reasons that could not be directly measured in this study. The suicidal ideation diagnostic code (ICD-9 V62.84) may be underutilized because it was only introduced to the ICD fairly recently, in 2005. It may also be used less often because it is a supplementary code meant to classify factors influencing health status. Clinicians also have competing demands for their time and may find coding suicidal ideation or attempt in addition to the main diagnosis, which is likely to be depression, to be cumbersome. Finally, there are administrative and legal issues related to coding suicidal ideation and attempt; clinicians must ensure these patients have access to specialty mental health care. While patients may indicate suicidal thoughts on their PHQ-9, it is also feasible that conversation with a patient allows the clinician to determine that the patient is not in fact suicidal, explaining the low occurrence of suicidal ideation diagnostic codes recorded in the EHR. Regardless, we believe that the patient care implications of more routine coding of suicidal ideation and attempt are substantial and very positive and that cultural and pragmatic issues leading to undercoding must be addressed for EHRs to reach their full potential for screening and monitoring.
Last, there is discussion regarding the ability of the 9th item of the PHQ-9 to actually reflect suicidal ideation because the item asks about both passive and active ideation.14 One study found that only 8% of patients who endorsed the 9th item of the PHQ-9 were actually considering harming themselves.19 Another study found that only 6% of patients who endorsed the 9th item of the PHQ-9 indicated they would be somewhat or very likely to attempt suicide.20 Other work found that few patients who endorsed the 9th item of the PHQ-9 actually indicated suicidal ideation (3% to 20%) or had a suicide attempt plan (8%).21,22 There are, however, studies that support an association between an endorsement of the 9th item of the PHQ-9 and suicidal ideation, as measured by a structural clinical interview,16 and its association with subsequent suicide attempt and death.15 Regardless, while not necessarily a direct indication of intent to commit suicide, a patient indicating thoughts of death or hurting themselves should still raise red flags. Therefore the use of an instrument such as the PHQ-9 to identify possibly suicidal patients remains an important part of primary care. While a positive response on the 9th item of the PHQ-9 may not always warrant a diagnostic code for suicidal ideation, it may at least warrant mention in the clinician's notes field.
Conclusions
If suicidal patients are seen frequently by their PCPs, then the right tools and patient-specific information will enhance successful monitoring among at-risk patients. In this way the use of diagnostic codes in the EHR of patients for whom suicidal ideation has been identified as a concern by their clinician has the potential to decrease suicide attempts and deaths. With the increasing use of EHRs in response to HITECH, tracking the acute and chronic conditions of patients should be easier, primarily with the use of automatically generated clinician reports or “red flag” warnings. Clinicians should be encouraged to document patients' suicidal ideation and behaviors to more easily and frequently monitor these patients in the health care system. More frequent monitoring of patients' treatment and well-being has the potential to identify the need for new or different treatment (medications or psychotherapy) and increase patients' adherence to proper treatment(s), which may ultimately decrease their risk of suicidal thoughts and behaviors.
Notes
This article was externally peer reviewed.
Funding: This project was funded by the Agency for Healthcare Research and Quality, US Department of Health and Human Services (grant no. K12HS019464 and Contract No. HHSA290200500371, Task Order no. 4).
Conflict of interest: All authors have received grant funding from federal government, foundation, and industry sources. HDA, RRA, AML, and RJV have also received funding from antidepressant manufacturers (Eli Lilly & Co., Forest Laboratories, Lundbeck A/S).
Disclaimer: The authors of this report are responsible for its content. Statements in this article should not be construed as endorsement by the Agency for Healthcare Research and Quality or the US Department of Health and Human Services.
- Received for publication June 20, 2014.
- Revision received September 8, 2014.
- Accepted for publication September 16, 2014.