Abstract
Background: Although insomnia is a prevalent complaint with significant consequences on quality of life, health, and health care utilization, it often remains undiagnosed and untreated in primary care settings. Brief, reliable, and valid instruments are needed to facilitate screening for insomnia in general practice. This study examined psychometric indices of the Insomnia Severity Index (ISI) to identify individuals with clinically significant insomnia in primary care settings.
Methods: A sample of 410 patients recruited from 6 general medical clinics completed the ISI before their appointment with a primary care physician. A subsample of 101 individuals also completed a semistructured clinical interview by telephone to determine the presence or absence of an insomnia disorder. Reliability and validity indices were computed, as was the discriminative capacity of each individual item. Convergence between ISI total score and the diagnosis derived from the interview was investigated. Receiver operator characteristic analyses were used to determine the optimal ISI cutoff score that correctly identified individuals with an insomnia disorder.
Results: ISI internal consistency was excellent (Cronbach α = 0.92), and each individual item showed adequate discriminative capacity (r = 0.65–0.84). The area under the receiver operator characteristic curve was 0.87 and suggested that a cutoff score of 14 was optimal (82.4% sensitivity, 82.1% specificity, and 82.2% agreement) for detecting clinical insomnia. Agreement between the ISI cut score and the diagnostic interview was moderate (κ = 0.62).
Conclusions: These findings suggest that the ISI is a valid screening instrument for detecting insomnia among patients consulting in primary care settings.
Insomnia is a highly prevalent condition, with about one third of the adult population reporting difficulties initiating or maintaining sleep and 10% also reporting significant daytime impairments.1⇓–3 When diagnostic criteria from the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV),4 are used, insomnia disorder affects 6% of the general population.2 Prevalence estimates in primary care are higher than in the general population, with rates varying between 10% and 40% across studies.5⇓⇓⇓⇓–10 Strong associations have been found between chronic insomnia and functional impairment, psychological and medical conditions, health-related quality of life, and health care utilization.5,11⇓⇓⇓–15
Despite the high prevalence of insomnia and its important morbidity, it still remains under-recognized and under-diagnosed.7,16,17 Accurate identification of insomnia symptoms is crucial to foster timely diagnosis and management and reduce morbidity. However, early screening of insomnia remains a challenge, particularly in primary care settings. Several barriers to assessment interfere with the recognition of insomnia.18 For example, few patients report their sleep difficulties to their physician and many try passive strategies or self-medication before consulting a health professional.1,19 Furthermore, the impact of insomnia as a health problem may be minimized by health professionals: insomnia often is considered as secondary to another condition.20 Because general practitioners are the health care professional most frequently consulted for a sleep problem,1,6 sleep difficulties should be systematically investigated in patients consulting in primary care.21 Thus, brief and psychometrically sound instruments are needed to facilitate the screening of insomnia in primary care.
Given the large volume of patients in primary care, a multilevel, stepwise approach should guide assessment of insomnia. A sensitive screening with 1 or 2 questions about sleep should be followed by a more detailed clinical evaluation of probable cases of insomnia. Brief questionnaires represent a useful and cost-effective method for routine assessment in comparison with more time-consuming methods of sleep evaluation (eg, clinical interview, sleep diaries). A variety of patient-reported measures are currently available for assessing subjective symptoms of sleep difficulties and their effect on diurnal functioning.22,23 An optimal screening tool for insomnia in primary care should be reliable and valid and produce minimal burden for patients (eg, time to administer and score, low cost).
The Insomnia Severity Index (ISI)24 is a brief instrument that assesses insomnia according to the criteria from the DSM-IV4 and the International Classification of Sleep Disorders.25 It is used increasingly in clinical as well as research activities. Its reliability, validity, and sensitivity to treatment response have been documented in the general population and with patients presenting primary insomnia and insomnia in cancer settings.26⇓⇓⇓⇓–31 However, the ISI has not been validated specifically in primary care. Therefore, this article examines psychometric properties of the ISI in a sample of primary care patients and investigates the optimal cutoff score for identifying individuals with clinically significant insomnia.
Methods
Recruitment and Procedure
The study protocol was approved by Laval University's ethics committee. Participants were recruited from March to May 2011 in the waiting rooms of 6 primary health care clinics of the Québec metropolitan area. Participants were eligible if they were 18 years of age or older and were consulting a general practitioner. Patients presenting at the registration counter were invited to participate in this study, which was described as a survey about sleep, health, and health care consultations. The majority (81.52%) of the 579 patients who were approached agreed to participate, signed the informed consent form, and completed the study questionnaire, including the ISI. A research assistant was available on site to answer questions as needed. Participants who did not have enough time to complete the questionnaire before their appointment had the possibility to finish it at home and send it back by mail using a prepaid envelope.
The 410 participants who completed the ISI also were invited to complete a telephone interview within the following month assessing their sleep and psychological health. Of those, 135 participants agreed to complete the interview (32.93%). The average time between completion of the ISI and the interview was 15 days (standard deviation [SD], 8.24). Twelve individuals withdrew their consent when contacted by the interviewer and 22 could not be reached within the time allowed. The clinical interview was completed by 101 participants (Figure 1). The interviewers were blinded to participants' ISI scores. All interviewers (5 clinical psychology graduate students and 1 research assistant with a BA in psychology) received training and supervision in conducting the diagnostic interview by an experienced clinical psychologist (PhD).
Recruitment and final sample. ISI, Insomnia Severity Index.
Participants
The 410 participants were adults (61.6% women; mean age, 47.9 years [SD, 15.2 years]) attending a primary care medical clinic. Table 1 presents the main sociodemographic characteristics of the sample, as well as some clinical and health care usage variables. The subsample that completed the clinical interview included 101 patients (65.3% women; mean age, 49.0 years [SD, 14.1 years]). These participants were similar to those who did not complete the interview on most variables, except they had more severe insomnia and a higher number of annual consultations with their general practitioner.
Measures
Insomnia Severity Index
The ISI24 is a 7-item self-report questionnaire assessing the nature, severity, and impact of insomnia in the past month. The specific items evaluate the severity of difficulties with sleep onset, sleep maintenance, and early morning awakening; sleep dissatisfaction; interference of sleep problems with daytime functioning; noticeability of sleep difficulties by others; and distress caused by sleep difficulties. A 5-point Likert scale (0 = none; 4 = very severe) is used to rate each item, with total scores ranging from 0 to 28. A higher total score indicates more severe sleep difficulties. Adequate psychometric properties for both the English and French versions have been reported in previous studies.27,28,30,31 The ISI was completed by participants while they were in the waiting room of the family physician's office. A corresponding version of the ISI was completed by the clinician during the telephone interview to assess convergent validity between the 2 versions of the ISI.
Insomnia Diagnostic Interview
The Insomnia Diagnostic Interview (IDI)24 is a semistructured clinical interview assessing history of sleep difficulties (type, duration, and frequency), daytime consequences, and precipitating and maintaining factors (eg, medical condition, presence of a sleep disorder other than insomnia, sleep medication). Diagnosis of an insomnia disorder was based on a combination of criteria from the DSM-IV32 the International Classification of Sleep Disorders,33 and the DSM-534: (1) the predominant complaint is dissatisfaction with sleep quantity or quality; (2) difficulty initiating sleep (sleep onset after ≥30 minutes), difficulty maintaining sleep (wake ≥30 minutes after sleep onset), early morning awakening with inability to return to sleep; (3) significant distress or impairment in daytime functioning (eg, fatigue or low energy, daytime sleepiness, cognitive impairments, mood disturbance); (4) sleep difficulties occur at least 3 nights per week; (4) sleep difficulties occur for at least 1 month. The IDI was administered during the telephone interview.
Mini International Neuropsychiatric Interview
The Mini International Neuropsychiatric Interview (MINI)35,36 was used to identify selected psychiatric disorders that commonly occur comorbid with insomnia. It is a structured interview assessing the presence of Axis I disorders based on DSM-IV criteria. The MINI is designed to be administered by mental health clinicians and has adequate sensitivity and specificity for all diagnoses, as well as good interrater and test-retest reliabilities.35,37 The French version has been validated36 and used in a phone interview context in previous studies.38,39 In this study, the following disorders were assessed: mood disorder (major depressive episode, dysthymic disorder, hypomania or mania); anxiety disorder (panic disorder, agoraphobia, social phobia, obsessive-compulsive disorder, posttraumatic disorder, and generalized anxiety disorder); substance abuse or dependence (alcohol, drug); and psychotic disorders. The MINI was administered during the clinical interview.
Interrater Reliability
Interrater reliability for insomnia and psychiatric diagnoses was examined with 25 randomly selected interviews (IDI and MINI). Clinical interviews were audio-recorded by the first interviewer and submitted to a second blind interviewer for independent diagnosis. Discrepancies were resolved by case discussion with a third independent interviewer to arrive at a consensus. Agreement between interviewers for the diagnosis of insomnia disorder was excellent (24 of 25 [96%]; κ = 0.90; P < .001). The Cohen's κ for the presence of any psychiatric disorder was 0.72 (0.71 for mood disorders, 0.88 for anxiety disorders, and 0.65 for substance abuse or dependence).
Data Analyses
Data were analyzed using SPSS version 12.0 (IBM, Chicago, IL), with a 2-tailed test (P < .05). Reliability was investigated using the standard Cronbach α coefficient and item-total correlations for internal consistency. The ISI performance in identifying insomnia cases was examined with receiver operating characteristics (ROC) analyses 40,41 using the subsample that completed the clinical interview (n = 101). The ability of the ISI to discriminate between cases and noncases is indicated by the area under the curve (AUC; score ranging from 0 to 1), where an AUC of 0.50 indicates a discriminative power similar to random assignment. Guidelines proposed by Swets42 were used to interpret the value of the AUC (0.50 to 0.70 = small; 0.70 to 0.90 = moderate; >0.90 = high discriminative power). Convergence between ISI total score and the diagnosis derived from the interview (presence or absence of insomnia disorder) was investigated. An index of diagnostic precision was estimated by computing sensitivity (true positive rate), specificity (true negative rate), and the percentage of correctly classified cases for all possible ISI scores. For each cutoff score, positive predictive value (proportion of participants correctly diagnosed as insomnia cases) and negative predictive value (proportion of participants correctly identified as noncases) was calculated. The optimal ISI cutoff score correctly identifying individuals with an insomnia disorder was determined as the score that maximized sensitivity and specificity. Last, correlations between the ISI completed by the participant and by the clinician were calculated for each item and for the total score to assess convergent validity.
Results
The mean ISI total score for the entire sample was 8.27 (SD, 6.44), indicating mild or subthreshold insomnia overall. The mean ISI score for the 101 participants who completed the clinical interview was 11.15 (SD, 6.76), also indicating subthreshold insomnia. On the basis of the clinical interview, 34 participants (33.66%) met diagnostic criteria for an insomnia disorder. Table 2 presents demographic and clinical characteristics of participants with and without an insomnia disorder. There was no significant difference between those 2 groups in age (t95 = −0.65; P = .521) and sex (χ2(1) = 1.52; P = .218). However, there was a higher proportion of participants with an insomnia disorder who also presented with a psychiatric diagnosis (n = 20; 58.8%) according to the MINI relative to participants without an insomnia disorder (χ2(1) = 20.82; P < .001). The most frequent diagnoses were major depression and generalized anxiety disorder. Participants with insomnia were not more likely to present with a medical condition than those without insomnia (χ2(1) = 1.83; P = .176). Nonetheless, 10 participants with an insomnia disorder reported having a condition that significantly interfered with their sleep (eg, chronic pain, diabetes, menopause).
Reliability and Item Response Pattern
The ISI demonstrated high internal consistency (Cronbach α = 0.92). Item-total correlations ranged from 0.65 to 0.84 (mean, 0.75), indicating that all items contribute to the ISI total score. Items showing the highest item-total correlations (r = 0.83–0.84) were those targeting dissatisfaction, interference, and preoccupation about sleep difficulties, whereas items about insomnia symptoms, such as difficulty falling asleep and early morning awakenings, showed the lowest correlations (r = 0.65–0.66).
Table 3 displays the percentage of individuals endorsing each item according to the different response choices. Choice 4 (severe symptoms or impact) was used infrequently for all items, whereas choices 0 (no symptoms) and 1 (mild severity or impact) were used frequently. In general, all items showed excellent discriminative capacity, and the response choices of each item were positively and strongly related to the total score.
Screening Ability
Convergence between ISI total score and the diagnosis derived from the interview was moderate (κ = 0.62). ROC analysis produced an AUC of 0.87 (95% CI, 0.80–0.94), suggesting excellent screening ability of the ISI. Figure 2 shows the ROC curve, illustrating the ISI sensitivity as a function of the false positive rate (or 1-specificity). Indices of sensitivity, specificity, predictive positive value, and negative predictive value obtained for all possible ISI total scores are summarized in Table 4. Findings revealed that a cutoff score of 14 provided the best overall balance of sensitivity and specificity in this sample of primary care medical patients. The percentage of correctly classified participants was 82.2% (sensitivity, 82.4%; specificity, 82.1% of individuals without insomnia disorder). Furthermore, the positive predictive value was 70.0% (probability of having an insomnia disorder when the total score is greater than or equal to the cutoff score of 14), and the negative predictive value was 90.2% (probability of not having an insomnia disorder when the total score is <14).
Receiver operator characteristics curve showing the probability of predicting a diagnosis of an insomnia disorder using various Insomnia Severity Index total scores.
With a cutoff score of 14, the false-positive rate (ie, overdiagnosis) was 17.9%. Twelve participants had a total score higher than the cutoff score of 14 on the ISI while not meeting diagnosis criteria for an insomnia disorder based on the clinical interview. Their ISI total score ranged between 15 and 22, indicating moderate to severe insomnia. The clinical interview indicated that 8 of these participants (66.67%) were not dissatisfied with their sleep and 4 (33.33%) had no distress or diurnal consequences associated with sleep difficulties. One third of these participants met criteria for ≥1 psychiatric disorder on the MINI, and a quarter had a medical condition that significantly interfered with their sleep. Finally, 3 participants reported another sleep disorder diagnosis (apnea, sleep-related bruxism, restless leg syndrome). The false-negative rate (ie, underdiagnosis) was 17.6%. Of the 34 participants who met diagnosis criteria of an insomnia disorder based on the clinical interview, 6 had a score on the ISI below the cutoff of 14. Their ISI total score ranged between 8 and 13, indicating subthreshold insomnia.
A cutoff of 8 allowed the identification of all individuals with a diagnosis of insomnia disorder (sensitivity, 100%), but it was associated with a low specificity of 52.2%. The percentage of correctly classified participants was lower than with a cutoff of 14 (68.3%). The positive predictive value was 51.5% and the negative predictive value was 90.2%. With a cutoff of 8, the false-positive rate was 47.8%, 32 individuals without insomnia based on the interview were classified as insomnia cases based on their ISI score.
Convergent Validity
Moderate to large correlation coefficients were obtained between scores on the ISI completed by the participants and the corresponding items of the version completed by the clinicians. The strongest correlation was on the item concerning preoccupation about sleep difficulties (r = 0.73), whereas the item about noticeability of sleep difficulties had the lowest correlation between the 2 versions (r = 0.27).
Discussion
The objectives of this study were to examine the psychometric properties of the ISI in primary care and to investigate the optimal cutoff score for identifying individuals with an insomnia disorder. The results provide additional evidence about the psychometric properties of the ISI and suggest that it could be a useful screening instrument for primary care settings.
The internal consistency was excellent and comparable to reliability coefficients obtained with community and clinical samples,27 as well as in cancer patients.31 All ISI items showed high correlations with the total score and excellent discriminative capacity. As suggested previously, sleep-onset insomnia and early morning awakening symptoms did not contribute as much to insomnia severity as items related to difficulty staying asleep, dissatisfaction, and perceived impact and worry about insomnia.27
The present findings suggest that the ISI is a useful instrument for identifying insomnia in primary care. The ROC analysis revealed that the ISI has a moderate to high discriminative power.42 Using a cutoff score of 14 on the ISI provided the overall best balance of sensitivity (82.2%) and specificity (82.4%) in this sample of primary care patients. This cutoff score was similar to the cutoff of 15 (sensitivity, 51.3%; specificity, 90.7%) obtained in cancer patients,31 although there was a significant discrepancy between sensitivity and specificity in that study. This cutoff score was higher that the cut point of 10 in a population-based sample (sensitivity, 86.1%; specificity, 87.7%) and 11 in a clinical sample (sensitivity, 97.2%; specificity, 100%) previously reported.27 The use of stringent criteria and a structured clinical interview to evaluate insomnia in this study may explain this difference with the cutoff score obtained previously in a community sample, which was derived from a single yes/no question assessing the participants' perception of having or not having a sleep problem. Furthermore, the sample enrolled in the present study was more heterogeneous than the clinical trial sample studied by Morin et al.27 Thus, cutoff scores should be interpreted according to the specific population.
The results showed that with a cutoff score of 14 on the ISI, the majority (82.2%) of participants were correctly classified with regard to insomnia status. From a clinical point of view, even if misdiagnosis was not common (17.8%), it can have important implications. Participants who were incorrectly diagnosed with an insomnia disorder based on their ISI score actually presented with sleep disorders other than insomnia, psychiatric disorders, and/or medical conditions. Even if these cases are not insomnia per se, their screening with the ISI could prompt a more detailed evaluation to make an accurate diagnosis and plan the most appropriate therapy. On the other hand, insomnia disorders undetected by the ISI (false-negative rate of 17.6%) raise some concern, given the morbidity associated with untreated chronic insomnia (eg, functional impairment, psychological and medical conditions).11,13⇓–15 These cases would be identified with a lower cutoff score, but the specificity of the instrument would be negatively affected. In the context of primary care, a measure that is both specific and sensitive is warranted, given the burden associated with the overdiagnosis of insomnia (eg, unnecessary diagnostic investigation or initiation of a treatment not warranted) and the underdiagnosis of insomnia (ie, costs of untreated insomnia).
Some limitations affect the interpretation of these findings. Participants completing the clinical interview were not randomly selected from the total sample, suggesting that they may present some characteristics that limit the generalizability of the findings to the entire primary care population. Participants who completed the clinical interview had more severe insomnia compared with those who completed the ISI only, but their mean total ISI score was in the same range (subthreshold insomnia). In addition, some important psychometric properties were not assessed in this study. Convergent and concurrent validities with other sleep measures should be investigated in primary care. Given the number of measures available using patient-report to assess insomnia, it would be particularly interesting to compare the capacity of the ISI to identify insomnia disorder with other instruments, for example the Athens Insomnia Scale43 and the Insomnia Symptom Questionnaire.44 As Moul et al22 pointed out in their review, selection of the best questionnaire for clinical practice involves careful consideration of its screening capacity in the specific population served. Moreover, sensitivity to treatment response was not assessed in the present study, warranting additional research to quantify the minimally important changes on the ISI to detect a clinically important reduction of insomnia symptoms in primary care. This would be useful for enhancing measurement-based patient care in primary care settings.
Conclusion
This study showed that the ISI is a reliable and valid instrument to identify patients with clinically significant insomnia in primary care. This measure could assist investigators in estimating the prevalence of insomnia in primary care and could be used by health professionals as a screening tool for insomnia disorder. The low burden of the ISI and its good balance between sensitivity and specificity are important aspects that could promote its use in primary care, as a first step to identify patients requiring a more detailed evaluation of their sleep difficulties. Further studies examining the ease of implementation of the ISI and its acceptability to health professionals would be interesting.
Acknowledgments
The authors thank the patients and staff at the general practices involved in this study.
Notes
This article was externally peer reviewed.
Funding: none.
Prior presentation: Parts of this study were presented at the 4th International Conference of the World Association of Sleep Medicine, Québec City, Québec, Canada (September 14, 2011) and at the 26th Annual Meeting of the Associated Professional Sleep Societies, Boston, MA (June 13, 2012).
Conflict of interest: CMM has received research contracts from Merck and has served as consultant for Novartis, Eli-Lilly, Purdue, and Valeant Canada.
- Received for publication February 11, 2013.
- Revision received May 13, 2013.
- Accepted for publication May 23, 2013.
References
In this issue
Jump to section
Related Articles
Cited By...
- Internet addiction prevalence and associated factors among secondary-level private school students in urban Nepal: a cross-sectional study
- Evaluation of the effectiveness of cognitive-behavioural therapy for insomnia delivered by nurses and physicians for patients in primary care (the NPD study): protocol for a cluster randomised controlled trial
- Study protocol for a case series: implementation and evaluation of an integrated mental and physical healthcare programme to screen for mental health symptoms in people with epilepsy
- 2023 International Olympic Committees (IOC) consensus statement on Relative Energy Deficiency in Sport (REDs)
- Methodology for studying Relative Energy Deficiency in Sport (REDs): a narrative review by a subgroup of the International Olympic Committee (IOC) consensus on REDs
- Comparative effectiveness of three versions of a stepped care model for insomnia differing in the amount of therapist support in internet-delivered treatment: study protocol for a pragmatic cluster randomised controlled trial (GET Sleep)
- Effectiveness of low-dose amitriptyline and mirtazapine for insomnia disorder: study protocol of a randomised, double-blind, placebo-controlled trial in general practice (the DREAMING study)
- The Most Frequently Read Articles of 2020
- Factors Associated with Mental Health Outcomes in Oman during COVID19: Frontline vs Non-frontline Healthcare Workers
- Promoting early treatment for mild traumatic brain injury in primary care with a guideline implementation tool: a pilot cluster randomised trial
- Factors associated with mental health outcomes across healthcare settings in Oman during COVID-19: frontline versus non-frontline healthcare workers
- Clinical and cost-effectiveness of a guided internet-based Acceptance and Commitment Therapy to improve chronic pain-related disability in green professions (PACT-A): study protocol of a pragmatic randomised controlled trial
- The INternet ThERapy for deprESsion Trial (INTEREST): protocol for a patient-preference, randomised controlled feasibility trial comparing iACT, iCBT and attention control among individuals with comorbid chronic pain and depression
- Cognitive behavioural treatment for insomnia in primary care: a systematic review of sleep outcomes
- Combination of obstructive sleep apnoea and insomnia treated by continuous positive airway pressure with the SensAwake pressure relief technology to assist sleep: a randomised cross-over trial protocol
- Family Physicians Improve Patient Health Care Quality and Outcomes