Abstract
Background: Better risk predictions for colorectal cancer (CRC) could improve prevention strategies by allowing clinicians to more accurately identify high-risk individuals. The National Cancer Institute's CRC risk calculator was created by Freedman et al using case control data.
Methods: An online risk calculator was created using data from the Multi-Ethnic Cohort Study, which followed >180,000 patients for the development of CRC for up to 11.5 years through linkage with cancer registries. Forward stepwise regression tuned to the c statistic was used to select the most important variables for use in separate Cox survival models for men and women. Model accuracy was assessed using 10-fold cross-validation.
Results: Patients in the cohort experienced 2762 incident cases of CRC. The final model for men contained age, ethnicity, pack-years of smoking, alcoholic drinks per day, body mass index, years of education, regular use of aspirin, family history of colon cancer, regular use of multivitamins, ounces of red meat intake per day, history of diabetes, and hours of moderate physical activity per day. The final model for women included age, ethnicity, years of education, use of estrogen, history of diabetes, pack-years of smoking, family history of colon cancer, regular use of multivitamins, body mass index, regular use of nonsteroidal anti-inflammatory drugs, and alcoholic drinks per day. The calculator demonstrated good accuracy with a cross-validated c statistic of 0.681 in men and 0.679 in women, and it seems to be well calibrated graphically. An electronic version of the calculator is available at http://rcalc.ccf.org.
Conclusion: This calculator seems to be accurate, is user friendly, and has been internally validated in a diverse population.
Current colorectal cancer (CRC) risk prediction strategies seem to be effective at reducing deaths from colorectal cancer,1 but they have certain limitations, including cost of the procedures, risk of rare but serious complications such as bowel perforation, and poor compliance. National screening guidelines suggest considering earlier screening for some high-risk patients (eg, patients with a family history of CRC) but otherwise promote a one size fits all philosophy.2 The dichotomization of patients into low- and high-risk categories based on a single variable (eg, age ≥50) by the guidelines is unfortunate since this strategy results in a loss of information and does not take into account other factors associated with CRC risk. Additional established risk factors for CRC include family history of CRC, personal history of adenomatous colorectal polyps, black race, obesity, lack of physical exercise, and inflammatory bowel disease (IBD).3 Other risk factors that are less established include history of diabetes, smoking, excessive alcohol intake, a diet high in red meat, and preference for eating well-done meat.3⇓⇓⇓–7 Several possible protective factors for CRC include nonsteroidal anti-inflammatory medications, multivitamins, and hormone replacement therapy.3 Supplementation with vitamin E, calcium, vitamin D, or fiber was thought to decrease risk based on observational studies but was proved ineffective when tested in randomized controlled trials.8⇓–10 Previous research suggests substantial differences between men and women in terms of CRC risk. Women may be exposed to exogenous estrogen in the form of oral contraceptives or hormone replacement therapy. Elevated body mass index seems to confer different effects on CRC risk in men than in women.11,12 In addition, at least one previous study found that cigarette smoking was an important risk factor for men but not for women.13
There are 2 major CRC risk calculators currently described in the literature: the Harvard Cancer Risk Index14 and a model by Freedman et al13 and validated by Park et al15 for the National Cancer Institute (NCI). The Harvard model only predicts the risk of colon cancer, which limits its use clinically since the screening and diagnostic testing for rectal cancer is essentially the same as it is for colon cancer. In addition, both the Harvard and NCI models were created using data from case control studies, which are prone to significant bias. Case control studies cannot definitively determine temporality and they are particularly prone to recall bias when the outcome being studied is cancer.16 This may make the calculators perform poorly when applied to healthy patients in the future.
Accurate risk prediction estimates might improve the efficiency of screening by targeting high-risk patients with earlier or more frequent screening and by either delaying or foregoing screening in very low-risk individuals. There are limited colonoscopy resources, and each additional procedure carries costs and the potential for complications.17 In addition, there may be a role for chemopreventive medications in some patients. Nonsteroidal anti-inflammatory medications have been shown to prevent adenomatous polyps but in general have not been found to be cost-effective for CRC prevention in patients at average risk.18,19 Improved chemoprevention with fewer side effects may be possible with the combination of sulindac and difluoromethylornithine,20 which may be cost-effective if reserved for high-risk individuals.
The Multiethnic Cohort Study (MEC) provides a potential data source for creating a new CRC calculator. The MEC is a large, prospective survey of a diverse population of residents from California and Hawaii who are >45 years old. Patients were followed for up to 11.5 years until death, the development of CRC, or until December 31, 2004. Details of the survey have been described elsewhere.21 The MEC is an excellent source of data because it contains detailed information about exposure and has a large proportion of racial/ethnic minorities. In addition, the ascertainment of CRC was performed by linking with cancer registries in California and Hawaii, states that have state-wide, population-based cancer registries. Mortality was ascertained with linkages to death certificate files in Hawaii and California. Additional state cancer registries and the National Death Index were used to obtain outcomes for patients known to have moved out of state. Patients were actively followed through an annual newsletter that helped to maintain current addresses, and they were periodically contacted directly for follow-up questionnaires. Loss to follow-up was <1%.
The Cancer Risk Prediction Models Workshop held in 2004 by the NCI in Washington, DC, recommended that risk prediction models incorporate more minority patients and encouraged new risk models to be simple and user-friendly.22 The goal of this project was to create an easy to use CRC calculator using the ethnically diverse population of patients available in the MEC. The idea was not to create a model for patients to use in the hope of changing behavior (which would require assumptions regarding cause and effect), but rather to create a model that could quickly provide a clinician with an accurate estimate of the patient's risk. The hope was to perform a head-to-head comparison of this new calculator with the existing NCI calculator. There were, unfortunately, differences in variable definitions between the MEC and the NCI datasets that made it difficult to accurately calculate the NCI predicted risk in the MEC cohort. Differences in variable definitions between the NCI and the MEC are depicted in Table 1.
Methods
The calculator resulting from this study was created using data from the MEC described above. Individuals with a history of CRC or adenomatous polyps were excluded from the analysis, resulting in a final sample size of 180,630 patients, of whom 2762 developed CRC. The following risk factors were measured at baseline in the MEC and were included as candidate variables in our final model: age (continuous); sex (male vs female); personal history of cancer (dichotomous); body mass index (continuous); regular aspirin use (currently, previously, or no); family history of colon cancer (dichotomous); estrogen use (currently, previously, or no); daily alcohol intake (continuous); regular multivitamin usage (currently, previously, or no); hours of moderate or strenuous activity per day (continuous); primary race/ethnicity (black, Hawaiian, Japanese, Latino, or white); diabetes (dichotomous); years of education (continuous); pack-years of smoking (continuous); regular use of nonsteroidal anti-inflammatory drugs (NSAIDS) (currently, previously, or no); intake (in ounces) of red meat per day (continuous); and preference for well-done meat (dichotomous). Continuous variables were modeled using restricted cubic splines, which use calculus to describe the nonlinear relationships between the predictor variables and the outcome.23 Categorization of continuous variables is frequently done but results in a loss of information and decreased prediction accuracy.24,25 Categorization is no longer necessary with computer-based calculator interfaces.
Detailed dietary questions were not included since randomized trials for fiber and vitamin supplementation proved ineffective at preventing adenomatous colorectal polyps and/or CRC. In addition, it was important to keep the calculator simple by avoiding detailed dietary questions. It is unfortunate that the initial MEC survey did not contain information regarding IBD disease, sigmoidoscopy, or colonoscopy; as a result, these risk factors could not be included.
Because of the potential differences in risk between men and women, we decided to create separate risk calculators for men and women to keep the models simple while allowing the greatest flexibility in the selection of variables without increasing the work required by the end user. The decision to create separate models for men and women also obviated the need to include other interactions in the model since there were no clear interactions described in the literature that did not involve sex. As recommended by Harrell et al,26 a posteriori interactions were not explored because of the possibility of spurious findings and unnecessary model complexity.
Missing data were imputed using the “mice” package for R in which all the predictor variables were used in regression equations to impute the missing values without knowledge of the outcome.27 Table 2 displays the missing data for each of the variables. Models for predicting the risk of CRC in men and women were fit using Cox regression from the time of the initial survey until the development of CRC or death.
Variable selection was performed using a modified version of forward stepwise regression that was tuned to Harrell's c-statistic,28 which will be referred to herein as the forward stepwise c-statistic. In a previous study involving 100 random cross-validations of 4 separate datasets, the forward stepwise c-statistic was the most likely to produce the most accurate model (highest c-statistic) when compared with traditional forward or backward stepwise regression and Harrell's model approximation.29 The forward stepwise c-statistic was determined by simply calculating the apparent c-statistic for each possible 1-variable model, selecting the most accurate variable, and then sequentially adding additional variables to the model by repeating this process until the c-statistic no longer increased (or the full model is reached).
The regression equations used to create the models also were used to construct a free online version of the calculator using the Cleveland Clinic Risk Calculator Constructor (http://makercalc.ccf.org). The models were internally validated using 10-fold cross-validation to assess both discrimination (c-statistic) and calibration (assessed graphically). Statistical calculations were performed using R version 2.10 with the Design library (available at http://www.r-project.org).
Results
Participants in the MEC had a mean age of 59.7 years at baseline, and there were 2762 incident cases of CRC. Detailed descriptive statistics according to sex and CRC outcome for the variables included in the final models are shown in Table 2A (men) and B (women). In univariate analyses among both men and women, the following factors were significantly associated with risk of CRC: age, race, smoking, alcohol intake, family history of colon cancer, multivitamin use, diabetes, and regular use of NSAIDs. Years of education and hours of activity per day were associated with CRC in unadjusted analyses among men only, whereas intake (in ounces) of red meat per day, preference for well-done meat, and use of estrogen were associated with CRC in women only.
Table 3A and B shows the absolute contribution that each additional variable made on the c-statistic for men and women, respectively. Age had a substantially stronger effect on the accuracy of the model than any of the other variables for both men and women. Race/ethnicity was the second most important variable for both sexes, and the relative importance of the additional variables were different between the sexes. The regular use of NSAIDs was an important predictor for women but not for men, whereas a personal history of cancer was not an important predictor variable for either sex. The final models contained 12 variables and 11 variables for men and women, respectively. These sized models were chosen because the c-statistic was within 0.001 of the maximal c-statistic that could be achieved with a full model.
Figure 1A and B shows the paper-based nomograms for calculating risk of CRC in men and women, respectively. The nomograms provide a visual method for quickly identifying the relationship between predictor variables and outcome and provide a general idea of the relative impact that different values of the variable of interest might have on risk. As you move to the right along any variable axis, the risk of CRC increases and the length of the axis shows the maximum contribution that any variable may have on overall risk. Age had the highest potential effect on risk of CRC for both sexes. The relationship between variables and risk of CRC seems to be consistent with the existing literature, except that the years of education variable has a j-shaped relationship for both men and women. Table 4 displays the multiple regression models using hazard ratios. Continuous variables were modeled using restricted cubic splines, and therefore a consistent hazard ratio for each incremental increase in a continuous variable cannot be displayed. Therefore, Table 4 shows the hazards associated with the 75th percentile versus the 25th percentile for these variables. Individuals at the 75th percentile for age (67 years in both men and women) were approximately 3 times more likely to be diagnosed with CRC than someone aged 52 years (point estimate of 3.03 in men and 2.81 in women). Coefficients for the final models are displayed in Table 5.
Nomogram for predicting colorectal cancer risk in men. Instructions: Draw a perpendicular line from the patient's age to the “points” axis and record the value. Repeat this process for the remaining variables and tally. The 10-year risk of colorectal cancer (CRC) is identified where a line drawn straight down from the “total points” axis intersects the “10-year risk of CRC (%).” Please note that the “years of education” variable has a U-shaped relationship with the 10-year risk of CRC. That is, the lowest risk of CRC occurs at 8 years and increases as you move along the top of the axis from left to right until reaching the highest risk at 14 years, and then it decreases along the bottom of the axis as you move to the left from 14 to 16 years.
Nomogram for predicting colorectal cancer risk in women. NSAIDS, nonsteroidal anti-inflammatory drugs.
The median predicted 10-year risk of CRC was 1.0% in women and 1.6% in men. The percentage of men with a 10-year risk <1% was 29.5%, whereas >48% of women had a 10-year risk <1%. In contrast, there were 2600 men (3.2%) and 206 women (0.2%) who had a predicted 10-year risk >5%. A preponderance of men with a risk of >5% were Japanese (66.6%), whereas 74.8% of the women with a risk >5% were black.
The 10-fold cross-validation revealed bias-adjusted c-statistics of 0.681 (95% confidence interval [CI], 0.669–0.694) and 0.679 (95% CI, 0.665–0.692) in men and women, respectively. Calibration curves, which suggest that the models are well calibrated, are shown in Figure 2A and B for men and women, respectively. An online version of the CRC-PRO calculator that provides calculations for men and women is available on the Cleveland Clinic calculator site (http://rcalc.ccf.org) under the heading “Colorectal Cancer.”
Calibration curve for the prediction in men (A) and women (B). The calibration curves were created by plotting the mean predicted risk of colorectal cancer in each quintile of risk on the x-axis against the corresponding Kaplan-Meier (K-M) estimated incidence in the same quintile. Error bars reflect the 95% confidence interval around the K-M estimate. The risks are displayed as probabilities. Perfect calibration would fall directly on the 45-degree line.
Discussion
The user-friendly CRC risk calculator created in this project seems to have good prediction accuracy and to be well calibrated in the 10-fold cross-validation. There is always a trade-off between model complexity and prediction accuracy. The models in this study were limited to 11 or 12 variables because of the very meager gains in accuracy beyond this model size. The full models for both men and women had apparent c-statistics that were <0.001 better than the final chosen models, which is unlikely to provide any meaningful benefit in clinical practice. It was surprising to see the dominance of age, in terms of prediction accuracy, for the models of both men and women. A single-variable model containing only age had bias-corrected c-statistics of 0.657 (95% CI, 0.628–0.685) and 0.662 (95% CI, 0.636–0.688) in women and men, respectively. At first glance, the ability of age to predict CRC this well by itself may argue for the current practice of creating screening guidelines based largely on age alone. However, there may still be individual patients of the same age whose risk of CRC could vary greatly. For example, a 50-year-old woman with the lowest risk possible according to the female nomogram in Figure 1B would have a 10-year risk of CRC of approximately 0.2%, whereas a 50-year-old at the highest risk, according to the extremes on the nomogram, would have a risk that is >10 times higher (ie, >2%).
It is hoped that future cost-effectiveness analyses and clinical guidelines involving CRC screening will focus on absolute risks based on sophisticated risk models rather than simple patient categories. The American Diabetes Association recently adopted this type of strategy with its recommendation that aspirin for the prevention of cardiovascular disease among patients with type 2 diabetes should be based on absolute risk of cardiovascular disease.30 The current joint guidelines for CRC screening recommend screening for all average-risk individuals starting at age 50.2 The 10-year probability of CRC for both men and women of all races at age 50 is approximately 0.646%.31 A look at the MEC data shows that 81.9% of men but only 57.0% of women between 50 to 55 years of age without a history of intestinal polyps have a 10-year probability of CRC that is ≥0.646%. A cost-effectiveness analysis that uses individual patient risk probabilities could help to better delineate the exact risk threshold at which screening becomes cost-effective.
Some researchers have suggested that absolute cutoffs for c-statistics can reflect the quality of a model (eg, a c-statistic >0.7 is considered to be good). In reality, any model with a c-statistic >0.5 may have clinical utility. The current model is more accurate than predicting risk with age alone and, as shown above, could result in a significant change in practice. However, the relatively low c-statistic does seem to indicate that important risk factors are conspicuously missing.
The nomograms revealed relationships between predictor variables and CRC that are largely consistent with the existing literature. It is also somewhat artificial to evaluate a single variable in the nomogram. In the real world, a change to one independent variable (ie, a patient characteristic) is almost always associated with changes to other variables. The goal of this calculator was not to evaluate the relationship between individual variables and CRC risk with the aim of changing behaviors, but rather to develop the most accurate overall risk prediction possible for tailoring screening and prevention strategies.
Because of differences in the datasets and variable definitions, it was, unfortunately, not possible to compare the accuracy of the models in this study head-to-head with the NCI calculator. In theory, the predictions obtained from this calculator should be more accurate than the NCI model because of the large sample size, the prospective data analysis, and the avoidance of the categorization of continuous variables. We hope that future analyses can directly compare the calculators using another dataset.
As mentioned previously, the MEC does not contain information about IBD, which is a known CRC risk factor. The estimated prevalence of Crohn's disease and ulcerative colitis in North America ranges from 37 to 246 and 26 to 199 cases per 100,000, respectively.32 In practice, patients with IBD should undergo more aggressive screening and prevention strategies, as reflected in a separate set of guidelines that have been established for these patients.33 It would have been ideal to have excluded patients with IBD from the creation of the risk calculator. Ignorance of IBD status could have artificially reduced the accuracy of the calculated prediction, but it is unlikely that this had a tremendous effect since the expected number of patients with IBD in the MEC should only be a few hundred.
Also mentioned previously, the initial survey of the MEC did not ask about a history of large-bowel endoscopy. Although this question was asked of a significant portion of the original patients in a follow-up survey, we did not feel it was appropriate to include this variable since the information was not known at baseline.
The large proportion of patients in the MEC who had a predicted 10-year risk of CRC <1% suggests that it may become reasonable clinically to delay or forgo colonoscopy in some very low-risk patients. This may be especially true for women since 1475 of the women in the study with a predicted risk <1% were at least 65 years old. Some experts have already suggested that a once-per-lifetime colonoscopic screening may be cost-effective in low risk patients.34
In contrast, there were more than 2806 patients who had a 10 year risk ≥5%. In comparison, a meta-analysis of ulcerative colitis found a 2% cumulative incidence of CRC 10 years after diagnosis.35 This raises the possibility that some of these high-risk patients might benefit from more aggressive preventive measures. NSAIDS have clearly been shown to reduce the risk of adenomatous polyps in randomized trials36⇓–38 and seem to reduce the risk of CRC in at least one post-trial analysis.37 A couple of decision analyses have found that NSAIDS would save lives but would not be cost-effective in average-risk patients.18,19,38 A combination regimen comprising a low-dose NSAID (sulindac) combined with a low-dose chemotherapeutic drug (difluoromethylornithine) has been successful at preventing recurrent adenomatous polyps with few side effects.20,39⇓–41 It is possible that this regimen could prove to be a cost-effective adjunct to current prevention strategies in some patients, especially given the extremely poor compliance with current CRC screening guidelines.
It is hoped that the calculator created in this study can help to improve the assessment of CRC risk in individual patients and encourage the use of absolute risk thresholds for decision making. Cost-effectiveness analyses may be needed to determine the exact effect that this improved prediction may have on clinical care.
Acknowledgments
The authors would like to acknowledge the assistance of Stephanie Kocian in editing the manuscript.
Notes
This article was externally peer reviewed.
Funding: Funding for the Multi-Ethnic Cohort Study was provided by grants 5R37 CA054281 and 5P01 CA033619 from the National Cancer Institute. Dr. Koroukian was supported in part by the Clinical and Translational Science Collaborative of Cleveland, UL1TR000439 from the National Center for Advancing Translational Sciences (NCATS) component of the National Institutes of Health and NIH Roadmap for Medical Research. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.
Conflict of interest: none declared.
- Received for publication January 18, 2013.
- Revision received July 5, 2013.
- Accepted for publication July 10, 2013.
References
In this issue
Jump to section
Related Articles
Cited By...
- Lifestyle predictors of colorectal cancer in European populations: a systematic review
- External Validation of Risk Prediction Models Incorporating Common Genetic Variants for Incident Colorectal Cancer Using UK Biobank
- Risk Prediction Models for Colorectal Cancer Incorporating Common Genetic Variants: A Systematic Review
- Comparison of prognostic models to predict the occurrence of colorectal cancer in asymptomatic individuals: a systematic literature review and external validation in the EPIC and UK Biobank prospective cohort studies
- Inclusion of a Genetic Risk Score into a Validated Risk Prediction Model for Colorectal Cancer in Japanese Men Improves Performance
- Risk assessment in pulmonary arterial hypertension
- Risk Prediction Models for Colorectal Cancer: A Systematic Review
- Content Usage and the Most Frequently Read Articles of 2014
- Serum Vascular Adhesion Protein-1 Level Predicts Risk of Incident Cancers in Subjects with Type II Diabetes
- Family Physicians are Complex Care Physicians and Quality of Care Advancement Experts