Skip to main content

Main menu

  • HOME
  • ARTICLES
    • Current Issue
    • Ahead of Print
    • Archives
    • Abstracts In Press
    • Special Issue Archive
    • Subject Collections
  • INFO FOR
    • Authors
    • Reviewers
    • Call For Papers
    • Subscribers
    • Advertisers
  • SUBMIT
    • Manuscript
    • Peer Review
  • ABOUT
    • The JABFM
    • The Editing Fellowship
    • Editorial Board
    • Indexing
    • Editors' Blog
  • CLASSIFIEDS
  • Other Publications
    • abfm

User menu

  • Log out

Search

  • Advanced search
American Board of Family Medicine
  • Other Publications
    • abfm
  • Log out
American Board of Family Medicine

American Board of Family Medicine

Advanced Search

  • HOME
  • ARTICLES
    • Current Issue
    • Ahead of Print
    • Archives
    • Abstracts In Press
    • Special Issue Archive
    • Subject Collections
  • INFO FOR
    • Authors
    • Reviewers
    • Call For Papers
    • Subscribers
    • Advertisers
  • SUBMIT
    • Manuscript
    • Peer Review
  • ABOUT
    • The JABFM
    • The Editing Fellowship
    • Editorial Board
    • Indexing
    • Editors' Blog
  • CLASSIFIEDS
  • JABFM on Bluesky
  • JABFM On Facebook
  • JABFM On Twitter
  • JABFM On YouTube
Research ArticleOriginal Research

Prediction of Primary Care Depression Outcomes at Six Months: Validation of DOC-6 ©

Kurt B. Angstman, Gregory M. Garrison, Cesar A. Gonzalez, Daniel W. Cozine, Elizabeth W. Cozine and David J. Katzelnick
The Journal of the American Board of Family Medicine May 2017, 30 (3) 281-287; DOI: https://doi.org/10.3122/jabfm.2017.03.160313
Kurt B. Angstman
From the Departments of Family Medicine (KBA, GMG, CAG), Information Technology (DWC), and Psychiatry and Psychology, Division of Integrated Behavioral Health (CAG, DJK), Mayo Clinic, Rochester, MN; and the Department of Family Medicine, Mayo Clinic, Zumbrota (EWC).
MS, MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Gregory M. Garrison
From the Departments of Family Medicine (KBA, GMG, CAG), Information Technology (DWC), and Psychiatry and Psychology, Division of Integrated Behavioral Health (CAG, DJK), Mayo Clinic, Rochester, MN; and the Department of Family Medicine, Mayo Clinic, Zumbrota (EWC).
MD, MS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Cesar A. Gonzalez
From the Departments of Family Medicine (KBA, GMG, CAG), Information Technology (DWC), and Psychiatry and Psychology, Division of Integrated Behavioral Health (CAG, DJK), Mayo Clinic, Rochester, MN; and the Department of Family Medicine, Mayo Clinic, Zumbrota (EWC).
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Daniel W. Cozine
From the Departments of Family Medicine (KBA, GMG, CAG), Information Technology (DWC), and Psychiatry and Psychology, Division of Integrated Behavioral Health (CAG, DJK), Mayo Clinic, Rochester, MN; and the Department of Family Medicine, Mayo Clinic, Zumbrota (EWC).
BCE
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Elizabeth W. Cozine
From the Departments of Family Medicine (KBA, GMG, CAG), Information Technology (DWC), and Psychiatry and Psychology, Division of Integrated Behavioral Health (CAG, DJK), Mayo Clinic, Rochester, MN; and the Department of Family Medicine, Mayo Clinic, Zumbrota (EWC).
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
David J. Katzelnick
From the Departments of Family Medicine (KBA, GMG, CAG), Information Technology (DWC), and Psychiatry and Psychology, Division of Integrated Behavioral Health (CAG, DJK), Mayo Clinic, Rochester, MN; and the Department of Family Medicine, Mayo Clinic, Zumbrota (EWC).
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • References
  • Info & Metrics
  • PDF
Loading

Abstract

Background: The goal of this study was to develop and validate an assessment tool for adult primary care patients diagnosed with depression to determine predictive probability of clinical outcomes at 6 months.

Methods: We retrospectively reviewed 3096 adult patients enrolled in collaborative care management (CCM) for depression. Patients enrolled on or before December 31, 2013, served as the training set (n = 2525), whereas those enrolled after that date served as the preliminary validation set (n = 571).

Results: Six variables (2 demographic and 4 clinical) were statistically significant in determining clinical outcomes. Using the validation data set, the remission classifier produced the receiver operating characteristics (ROC) curve with a c-statistic or area under the curve (AUC) of 0.62 with predicted probabilities than ranged from 14.5% to 79.1%, with a median of 50.6%. The persistent depressive symptoms (PDS) classifier produced an ROC curve with a c-statistic or AUC of 0.67 and predicted probabilities that ranged from 5.5% to 73.1%, with a median of 23.5%.

Conclusions: We were able to identify readily available variables and then validated these in the prediction of depression remission and PDS at 6 months. The DOC-6 tool may be used to predict which patients may be at risk for worse outcomes.

  • Depression
  • Primary Health Care
  • Retrospective Studies
  • ROC Curve

Depressive symptomatology is common among patients seeking primary care. The implementation of collaborative care models (CCMs) has contributed to addressing and improving clinical outcomes for depressive disorders in primary care settings.1⇓–3 CCMs provide additional resources that otherwise would not be available, such as care managers and psychiatric oversight to adapt treatment recommendations. Despite improved outcomes for depression, we previously demonstrated in our CCM population that the predicted probability of persistent depressive symptoms (PDSs) 6 months after implementing the CCM ranged from 6.6% to 42.1%.4 The predictive factors involved were clinical diagnosis (first episode vs recurrent major depression), depression severity, anxiety severity, and the result of the Mood Disorder Questionnaire (MDQ5) screen. Furthermore, prior studies at our institution associated a higher baseline severity of depressive symptoms (as measured on the 9-item Patient Health Questionnaire [PHQ-9]6) and anxiety (as measured by the 7-item Generalized Anxiety Disorder Questionnaire [GAD-7]7), and a positive MDQ screen with worse clinical outcomes in a CCM.8⇓⇓–11

In addition to mental health comorbidities serving as risk factors for poor clinical outcomes of depressive symptoms in primary care, psychosocial difficulties are predictive of poor outcomes for the treatment of depression in primary care.12 Moreover, psychosocial difficulties are associated with complications with treatment engagement and negatively affect medical outcomes in primary care.12⇓⇓⇓⇓⇓–18 In Europe, the PREDICT study developed a multifactor risk score for predicting the development of depression among primary care patients who were asymptomatic for depression.19,20 To our knowledge, however, no empirically derived tool is available to predict depression outcomes among patients in primary care who have recently been diagnosed with major depression.

Given the rate of PDSs among patients in primary care and the risk factors associated with poor clinical outcomes for the treatment of depression in primary care, practical and clinical implications exist for predicting depression outcomes. By estimating the probability of remission from depressive symptomatology using readily available information based on patients' demographic and clinical characteristics, clinicians may be able to optimize outcomes by adapting treatment recommendations, specifically for patients identified to be at higher risk for poor outcomes.

The goal of this study was to develop and validate an assessment tool for adult primary care patients diagnosed with depression that used their initial clinical and demographic characteristics by modeling the probabilities of remission and PDSs at 6 months. By estimating the relative risk associated with the clinical and demographic patient variables, clinicians may be able to adapt therapies, specifically for patients at higher risk for PDSs.

Methods

This study was approved by the Mayo Clinic Institutional Review Board. Adult patients in this study were identified from the primary care practices at the Mayo Clinic in southeastern Minnesota (empaneled patients from the Department of Family Medicine, Division of Primary Care Internal Medicine, and Division of Community Pediatrics and Adolescent Medicine). A CCM for depression care was implemented at 2 sites in 2008 and expanded to all 5 clinical sites (∼100,000 adult patients) by March 2010. Our CCM, through assistance from a specifically trained registered nurse care manager, with weekly oversight of the patients by a psychiatrist, included a depression registry, treatment guidelines, and relapse prevention counseling, and has been reviewed previously.21,22 All adult patients diagnosed (by primary or specialty care clinicians) with major depression or dysthymia and with an initial PHQ-9 score ≥10 had the option of enrolling in the program. The only exclusionary criterion for entry into the CCM was a clinical diagnosis of bipolar disorder (an abnormal MDQ screen was acceptable). Patients who were diagnosed with major depressive disorder and an initial PHQ-9 score <10 were treated with usual primary care.

Cohort

We retrospectively reviewed the medical records of 4663 adult patients (aged 18–93 years) enrolled in the CCM from March 1, 2008, through June 30, 2015. Some patients (1567, or 33.6%) were missing 6-month follow-up PHQ-9 scores and were excluded from the analysis. Patients with complete data enrolled on or before December 31, 2013, served as the training set (n = 2525). Patients enrolled on or after January 1, 2014 till the end of the study, served as a validation set (n = 571), for a study cohort total of 3096 patients.

The dependent variable was the PHQ-9 score at the 6-month follow-up after enrollment in the CCM program. Remission was defined as a 6-month PHQ-9 score <5, whereas the presence of PDSs was defined as a PHQ-9 score ≥10. In addition to the initial diagnosis (first episode of major depression or recurrent major depression or dysthymia), the independent demographic variables were age, sex, and marital status (yes or no). Independent clinical variables included the results of the baseline PHQ-9 score, MDQ screen, Alcohol Use Disorders Identification Test (AUDIT) questionnaire,23 and GAD-7 score.

The PHQ-9 is a 9-item tool that scores each item from 0 to 3 based on the patient's assessment over the prior 2-week period. The minimum score is 1 and the maximum is 27. The MDQ screen was scored as “negative” if the total score was <7 points for first question and if both the second and third questions had a negative response; it was scored as “abnormal” if any mix (or all) of the positive criteria were coded. As noted previously, patients with a clinical diagnosis of bipolar disorder were excluded from CCM enrollment, but no one was excluded based on a positive MDQ screen alone. The AUDIT has 10 questions that are scored from 0 to 4. An AUDIT score >8 signifies an increased risk of hazardous drinking.24 The GAD-7 has a range of scores from 0 to 21, and anxiety symptomology was defined as asymptomatic (score of 0–4), mild (5–9), moderate (10–14), or severe (≥15).

Classifiers

Using the training data set, 2 multivariate logistic regression classifiers were created: 1 for probability of remission and the other for the probability of PDSs. In both models, backward elimination of factors with a two sided P value of > .05 was used to arrive at a final simplified model in both cases. Analysis of variance with P < .05 for statistical significance was used to test the null hypothesis that the coefficients of the factors eliminated from the simplified model were not equal to zero.

Classifier Validation

Both classifiers obtained from the training data set using the simplified models were run on the validation data set to produce the predicted probabilities of remission from depression and PDSs. These predicted probabilities were compared with the actual 6-month PHQ-9 score to produce receiver operating characteristic (ROC) curves. The c-statistic, or area under the curve (AUC), was computed.25 Bootstrapping using 1000 random samples with replacement of the validation data set was used to compute the 95% confidence intervals for the c-statistic in both classifiers. The statistical software R (version 3.0.2; R Foundation for Statistical Computing, Vienna, Austria), including the ROCR26 and Boot27 packages, were used to analyze the data.

Results

Of the 3096 patients in this study, 2525 (81.6%) were in the training cohort and the remainder were in the validation cohort (n = 571 [18.4%]). Remission rates at 6 months were 52.3% (n = 1321) for the training group and 49.7% (n = 284) for the validation group (P = .266). PDS was present in 594 patients (23.5%) in the training cohort and 152 patients (26.2%) in the validation cohort (P = .123). Mild depression severity (PHQ-9 score of 5–9) was present in 23.5% versus 26.6%, respectively.

Remission Classifier

The odds ratios and 95% confidence intervals for the factors retained in the final simplified remission of depressive symptoms classifier are illustrated in Figure 1. Sex, dysthymia diagnosis, and the AUDIT score were dropped from the initial model by backward selection because their P value was > .05. Comparing the final simplified model with the initial full model, the null hypothesis that coefficients of the dropped factors were not equal to zero was rejected (P = .057). Thus, the simplified model is not significantly different from the full model.

Figure 1.
  • Download figure
  • Open in new tab
Figure 1.

Odds of a 9-item Patient Health Questionnaire (PHQ-9) score <5 at 6 months after enrollment in collaborative care management for depression, by variable. MDQ, Mood Disorder Questionnaire; GAD-7: 7-item Generalized Anxiety Disorder Questionnaire.

The remission of depressive symptoms classifier produced the ROC curve shown in Figure 2 when run using the validation data. The c-statistic (AUC) was 0.62, with a 95% confidence interval of 0.57 to 0.66.

Figure 2.
  • Download figure
  • Open in new tab
Figure 2.

Comparison of the Receiver Operating Characteristics (ROC) curves for Depression Outcome Validation using DOC-6 tool.

Predicted probabilities for remission at 6 months using the validation data ranged from 14.5% to 79.1%, with a median of 50.6% (first quartile, 41.0%; third quartile, 59.3%).

Persistent Depressive Symptoms Classifier

Figure 3 shows the odds ratios and 95% confidence intervals for the factors retained in the final simplified persistent depressive symptoms classifier. Sex, dysthymia, and the AUDIT score were dropped from the initial model by backward selection because their P value was > .05. Comparing the final simplified model with the initial model, the null hypothesis that coefficients of the dropped factors were not equal to zero was rejected (P = .42). Thus, the simplified model is not significantly different from the full model.

Figure 3.
  • Download figure
  • Open in new tab
Figure 3.

Odds of a 9-item Patient Health Questionnaire (PHQ-9) score ≥10 at 6 months after enrollment in collaborative care management for depression, by variable. MDQ, Mood Disorder Questionnaire; GAD-7: 7-item Generalized Anxiety Disorder Questionnaire.

The persistent depressive symptoms classifier produced the ROC curve shown in Figure 2 when run using the validation data. The c-statistic (AUC) was 0.67, with a 95% confidence interval of 0.61 to 0.72. Predicted probabilities for PDSs on the validation data ranged from 5.5% to 73.1%, with a median of 23.5% (first quartile, 15.9%; third quartile, 32.9%).

Depression Outcomes Calculator

To ease the burden of referencing numerous coefficients and performing logarithmic calculations in a clinical setting, an online calculator was designed using the validated model. This tool was named the Depression Outcomes Calculator-Six Items (DOC-6) (© 2016 Angstman et al. All rights reserved) and can be accessed at http://www.mayoclinic.org/doc-6. The online calculator was built using simple HTML5 form elements and the Knockout JavaScript framework (http://knockoutjs.com) to bind data between the data model and form fields. By prompting the user for each of the variable inputs through a simple form, the tool automatically applies the appropriate formulas and renders the output numerically and graphically, rounded to the nearest percentage. The use of standard web protocols and full client-side execution allows the maximum portability of this tool across browsers and devices. Results are given as the calculated predictive probability, with placement in the appropriate quartile. The graphical output is dynamically created upon execution and is accomplished using HTML styles in order to achieve instantaneous response times. The overall scale of the graphic display represents 0.0–1.0/300 pixels, which was chosen to optimize the ability to view it on mobile devices. The black quartile demarcations are a single pixel wide, making them accurate to the nearest third of a percentage. The yellow result line is 3 pixels wide, making it equivalent to 1% of the overall scale; the yellow line is placed on the scale to illustrate the percentage point.

Discussion

Using baseline characteristics and standardized assessment tools, we developed and preliminarily validated 2 logistic regression classifiers that could predict the probability of remission and PDSs at 6 months for depressed patients in a CCM, with moderate success (c-statistics of 0.62 and 0.67). We created a simple web-based calculation tool, the DOC-6, which uses these classifiers to deliver easily visualized probabilities and quartile comparisons. This prognostic tool allows care managers and physicians to triage patients based on a data-driven and validated risk stratification, alerting them to those patients who may require closer monitoring or additional therapeutic services.

Figures 4 and 5 demonstrate the probable predictive output of the model for 2 patient scenarios. The first is a 20-year-old married patient with an uncomplicated (asymptomatic for anxiety symptoms and a negative MDQ screen) first episode of depression, with an initial PHQ-9 score of 10 (Figure 4). The DOC-6 output demonstrates a 72% predicted probability of remission at 6 months (fourth quartile), with only a 9% probability of PDSs (first quartile). To contrast this example, Figure 5 illustrates the DOC-6 output for a 67-year-old unmarried patient with a complicated (symptomatic for anxiety symptoms, with a GAD-7 score of 18 and a partially positive MDQ screen) recurrent episode of depression and an initial PHQ-9 score of 26, with a 20% predicted probability of remission (first quartile) at 6 months and a 62% chance of PDSs (fourth quartile) at 6 months.

Figure 4.
  • Download figure
  • Open in new tab
Figure 4.

Predictive probability of 6-month outcomes for a 20-year-old married patient with an uncomplicated (asymptomatic for anxiety symptoms and a negative Mood Disorder Questionnaire screen) first episode of depression, with an initial 9-item Patient Health Questionnaire score of 10.

Figure 5.
  • Download figure
  • Open in new tab
Figure 5.

Predictive probability of 6-month outcomes for a 67-year-old unmarried patient with a complicated (symptomatic for anxiety symptoms with a 7-item Generalized Anxiety Disorder Questionnaire score of 18 and a partially positive Mood Disorder Questionnaire screen) recurrent episode of depression and an initial 9-item Patient Health Questionnaire score of 26.

The DOC-6 is practical tool that uses results from standardized, widely available assessment tools (the PHQ-9, GAD-7, and MDQ). In our practice, all the relevant information was obtained from the patient by the care manager. Obtaining the data and inputting it into the tool would not require clinician activity. Thus, few barriers exist to implementation within primary care. We have demonstrated that other comorbid psychiatric conditions, such as posttraumatic stress disorder, can have a negative effect on depression outcomes in a CCM.28 It is not always possible to make time available for a structured interview or even a full psychiatric history, especially in a busy primary care practice with limited psychiatric resources. Input for the DOC-6 can be obtained during a clinical visit or a phone call, or via other electronic methods. While the result does not predict the individual patient's exact outcome (as this is affected by other factors, especially current treatment availability), it will potentially allow the clinician the opportunity to more effectively stratify risk and triage those patients who may need more clinical or therapeutic support. Future research may wish to evaluate whether the use of the DOC-6 contributes to increased clinical outcomes and cost-effectiveness among the primary care patient population.

This study has several limitations. The patients were diagnosed with major depression or dysthymia by their primary care provider and reviewed by a consulting psychiatrist. A structured diagnostic interview such as the Structured Clinical Interview for DSM-5 was not used, but this is consistent with routine clinical care, making the results more generalizable. The DOC-6 was validated at a large academic institution using a cohort of community primary care patients. While we have no reason to believe that our primary care patient population varies significantly from other primary care settings across the United States, cross-validation in other geographic locations and within minority populations will contribute to the validity of the DOC-6. The patients included in this study were all in a care coordination program. Thus, the algorithm may not generalize exactly to depressed patients who are not receiving coordinated care, as CCM patients receive more intensive care than those in usual care. Predictive classification performance of the model was moderate (AUC = 0.62–0.67), but is similar to other complex clinical classification scenarios, such as predicting 30-day readmissions.29,30 Inclusion of variables such as biomarkers or depression subtypes in future research may help refine the model, improving the predictive performance.

Conclusions

Using primary care patient data in a CCM for depression treatment, we were able to identify 6 readily available variables (2 demographic and 4 clinical), then validated them for the prediction of 6-month remission and PDSs. The DOC-6 tool may be used to predict which patients may be at risk for worse outcomes.

Notes

  • This article was externally peer reviewed.

  • Funding: The study was supported by the Mayo Clinic Center for Translational Sciences Activities through grant UL1 TR000135 from the National Center for Advancing Translational Sciences, a component of the NIH. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. Funds from the Department of Family Medicine, Mayo Clinic were also used for this study.

  • Conflict of interest: none declared.

  • To see this article online, please go to: http://jabfm.org/content/30/3/281.full.

  • Received for publication September 26, 2016.
  • Revision received January 9, 2017.
  • Accepted for publication January 9, 2017.

References

  1. 1.↵
    1. Gilbody S,
    2. Bower P,
    3. Fletcher J,
    4. Richards D,
    5. Sutton AJ
    . Collaborative care for depression: a cumulative meta-analysis and review of longer-term outcomes. Arch Intern Med 2006;166:2314–21.
    OpenUrlCrossRefPubMed
  2. 2.↵
    1. Thota AB,
    2. Sipe TA,
    3. Byard GJ,
    4. et al
    . Collaborative care to improve the management of depressive disorders: a community guide systematic review and meta-analysis. Am J Prev Med 2012;42:525–38.
    OpenUrlCrossRefPubMed
  3. 3.↵
    1. Unutzer J,
    2. Katon W,
    3. Callahan CM,
    4. et al
    . Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA 2002;288:2836–45.
    OpenUrlCrossRefPubMed
  4. 4.↵
    1. Angstman KB,
    2. Shippee ND,
    3. Maclaughlin KL,
    4. et al
    . Patient self-assessment factors predictive of persistent depressive symptoms 6 months after enrollment in collaborative care management. Depress Anxiety 2013;30:143–8.
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Hirschfeld RM,
    2. Calabrese JR,
    3. Weissman MM,
    4. et al
    . Screening for bipolar disorder in the community. J Clin Psychiatry 2003;64:53–9.
    OpenUrlCrossRefPubMed
  6. 6.↵
    1. Spitzer RL,
    2. Kroenke K,
    3. Williams JB
    . Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA 1999;282:1737–44.
    OpenUrlCrossRefPubMed
  7. 7.↵
    1. Spitzer RL,
    2. Kroenke K,
    3. Williams JB,
    4. Lowe B
    . A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006;166:1092–7.
    OpenUrlCrossRefPubMed
  8. 8.↵
    1. Angstman KB,
    2. Dejesus RS,
    3. Rohrer JE
    . Correlation between mental health co-morbidity screening scores and clinical response in collaborative care treatment for depression. Ment Health Fam Med 2010;7:129–33.
    OpenUrlPubMed
  9. 9.↵
    1. Angstman KB,
    2. Maclaughlin KL,
    3. Rasmussen NH,
    4. DeJesus RS,
    5. Katzelnick DJ
    . Age of depressed patient does not affect clinical outcome in collaborative care management. Postgrad Med 2011;123:123–45.
    OpenUrl
  10. 10.↵
    1. Angstman KB,
    2. MacLaughlin KL,
    3. Williams MD,
    4. Rasmussen NH,
    5. DeJesus RS
    . Increased anxiety and length of treatment associated with depressed patients who are readmitted to collaborative care. J Prim Care Community Health 2011;2:82–6.
    OpenUrlCrossRefPubMed
  11. 11.↵
    1. Angstman KB,
    2. Pietruszewski P,
    3. Rasmussen NH,
    4. Wilkinson JM,
    5. Katzelnick DJ
    . Depression remission after six months of collaborative care management: role of initial severity of depression in outcome. Ment Health Fam Med 2012;9:99–106.
    OpenUrlPubMed
  12. 12.↵
    1. Gunn J,
    2. Elliott P,
    3. Densley K,
    4. et al
    . A trajectory-based approach to understand the factors associated with persistent depressive symptoms in primary care. J Affect Disord 2013;148:338–46.
    OpenUrlCrossRefPubMed
  13. 13.↵
    1. Bosworth HB,
    2. Hays JC,
    3. George LK,
    4. Steffens DC
    . Psychosocial and clinical predictors of unipolar depression outcome in older adults. Int J Geriatr Psychiatry 2002;17:238–46.
    OpenUrlCrossRefPubMed
  14. 14.↵
    1. Bosworth HB,
    2. McQuoid DR,
    3. George LK,
    4. Steffens DC
    . Time-to-remission from geriatric depression: psychosocial and clinical factors. Am J Geriatr Psychiatry 2002;10:551–9.
    OpenUrlCrossRefPubMed
  15. 15.↵
    1. Rubenstein LV,
    2. Rayburn NR,
    3. Keeler EB,
    4. Ford DE,
    5. Rost KM,
    6. Sherbourne CD
    . Predicting outcomes of primary care patients with major depression: development of a depression prognosis index. Psychiatr Serv 2007;58:1049–56.
    OpenUrlCrossRefPubMed
  16. 16.↵
    1. Rush AJ,
    2. Trivedi MH,
    3. Wisniewski SR,
    4. et al
    . Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR*D report. Am J Psychiatry 2006;163:1905–17.
    OpenUrlCrossRefPubMed
  17. 17.↵
    1. Sherbourne C,
    2. Schoenbaum M,
    3. Wells KB,
    4. Croghan TW
    . Characteristics, treatment patterns, and outcomes of persistent depression despite treatment in primary care. Gen Hosp Psychiatry 2004;26:106–14.
    OpenUrlCrossRefPubMed
  18. 18.↵
    1. Simon GE,
    2. Von Korff M,
    3. Lin E
    . Clinical and functional outcomes of depression treatment in patients with and without chronic medical illness. Psychol Med 2005;35:271–9.
    OpenUrlCrossRefPubMed
  19. 19.↵
    1. King M,
    2. Weich S,
    3. Torres-González F,
    4. et al
    . Prediction of depression in European general practice attendees: the PREDICT study. BMC Public Health 2006;6:6.
    OpenUrlCrossRefPubMed
  20. 20.↵
    1. Rifel J,
    2. Svab I,
    3. Ster MP,
    4. Pavlic DR,
    5. King M,
    6. Nazareth I
    . Impact of demographic factors on recognition of persons with depression and anxiety in primary care in Slovenia. BMC Psychiatry 2008;8:96.
    OpenUrlPubMed
  21. 21.↵
    1. Angstman KB,
    2. Rasmussen NH,
    3. Herman DC,
    4. Sobolik JJ
    . Depression care management: impact of implementation on health system costs. Health Care Manag (Frederick) 2011;30:156–60.
    OpenUrlPubMed
  22. 22.↵
    1. Williams M,
    2. Angstman K,
    3. Johnson I,
    4. Katzelnick D
    . Implementation of a care management model for depression at two primary care clinics. J Ambul Care Manage 2011;34:163–73.
    OpenUrlCrossRefPubMed
  23. 23.↵
    1. Reinert DF,
    2. Allen JP
    . The alcohol use disorders identification test: an update of research findings. Alcohol Clin Exp Res 2007;31:185–99.
    OpenUrlCrossRefPubMed
  24. 24.↵
    1. Saunders JB,
    2. Aasland OG,
    3. Babor TF,
    4. de la Fuente JR,
    5. Grant M
    . Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption–II. Addiction 1993;88:791–804.
    OpenUrlCrossRefPubMed
  25. 25.↵
    1. Austin PC,
    2. Steyerberg EW
    . Interpreting the concordance statistic of a logistic regression model: relation to the variance and odds ratio of a continuous explanatory variable. BMC Med Res Methodol 2012;12:82.
    OpenUrlCrossRefPubMed
  26. 26.↵
    1. Sing T,
    2. Sander O,
    3. Beerenwinkel N,
    4. Lengauer T
    . ROCR: visualizing classifier performance in R. Bioinformatics 2005;21:3940–1.
    OpenUrlAbstract/FREE Full Text
  27. 27.↵
    1. Davison AC,
    2. Hinkley DV
    . Bootstrap methods and their application. New York: Cambridge University Press; 1997.
  28. 28.↵
    1. Angstman KB,
    2. Marcelin A,
    3. Gonzalez CA,
    4. Kaufman TK,
    5. Maxson JA,
    6. Williams MD
    . The impact of posttraumatic stress disorder on the 6-month outcomes in collaborative care management for depression. J Prim Care Community Health 2016;7:159–64.
    OpenUrlCrossRefPubMed
  29. 29.↵
    1. Kansagara D,
    2. Englander H,
    3. Salanitro A,
    4. et al
    . Risk prediction models for hospital readmission: a systematic review. JAMA 2011;306:1688–98.
    OpenUrlCrossRefPubMed
  30. 30.↵
    1. Garrison GM,
    2. Robelia PM,
    3. Pecina JL,
    4. Dawson NL
    . Comparing performance of 30-day readmission risk classifiers among hospitalized primary care patients. J Eval Clin Pract 2016 Oct 3 [Epub ahead of print].
PreviousNext
Back to top

In this issue

The Journal of the American Board of Family     Medicine: 30 (3)
The Journal of the American Board of Family Medicine
Vol. 30, Issue 3
May-June 2017
  • Table of Contents
  • Table of Contents (PDF)
  • Cover (PDF)
  • Index by author
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on American Board of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Prediction of Primary Care Depression Outcomes at Six Months: Validation of DOC-6 ©
(Your Name) has sent you a message from American Board of Family Medicine
(Your Name) thought you would like to see the American Board of Family Medicine web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
4 + 7 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Prediction of Primary Care Depression Outcomes at Six Months: Validation of DOC-6 ©
Kurt B. Angstman, Gregory M. Garrison, Cesar A. Gonzalez, Daniel W. Cozine, Elizabeth W. Cozine, David J. Katzelnick
The Journal of the American Board of Family Medicine May 2017, 30 (3) 281-287; DOI: 10.3122/jabfm.2017.03.160313

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Prediction of Primary Care Depression Outcomes at Six Months: Validation of DOC-6 ©
Kurt B. Angstman, Gregory M. Garrison, Cesar A. Gonzalez, Daniel W. Cozine, Elizabeth W. Cozine, David J. Katzelnick
The Journal of the American Board of Family Medicine May 2017, 30 (3) 281-287; DOI: 10.3122/jabfm.2017.03.160313
Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • Methods
    • Results
    • Discussion
    • Conclusions
    • Notes
    • References
  • Figures & Data
  • References
  • Info & Metrics
  • PDF

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • Research on Clinical Decisions Made Daily in Family Medicine
  • Google Scholar

More in this TOC Section

  • Integrating Adverse Childhood Experiences and Social Risks Screening in Adult Primary Care
  • A Pilot Comparison of Clinical Data Collection Methods Using Paper, Electronic Health Record Prompt, and a Smartphone Application
  • Associations Between Modifiable Preconception Care Indicators and Pregnancy Outcomes
Show more Original Research

Similar Articles

Keywords

  • Depression
  • Primary Health Care
  • Retrospective Studies
  • ROC Curve

Navigate

  • Home
  • Current Issue
  • Past Issues

Authors & Reviewers

  • Info For Authors
  • Info For Reviewers
  • Submit A Manuscript/Review

Other Services

  • Get Email Alerts
  • Classifieds
  • Reprints and Permissions

Other Resources

  • Forms
  • Contact Us
  • ABFM News

© 2025 American Board of Family Medicine

Powered by HighWire