Too much ado about two-part models and transformation?: Comparing methods of modeling Medicare expenditures

https://doi.org/10.1016/j.jhealeco.2003.10.005Get rights and content

Abstract

Many methods for modeling skewed health care cost and use data have been suggested in the literature. This paper compares the performance of eight alternative estimators, including OLS and GLM estimators and one- and two-part models, in predicting Medicare costs. It finds that four of the alternatives produce very similar results in practice. It then suggests an efficient method for researchers to use when selecting estimators of health care costs.

Introduction

The health economist or health services researcher modeling health care costs or use faces a daunting literature about alternative estimators. The econometric challenges posed by these data include restricted range (non-negative observations), a “spike” of zero values, and skewness (a heavy right-hand tail). These properties make ordinary least squares estimation biased and inefficient. Popular alternatives to OLS include two-part models (Manning et al., 1981, Duan et al., 1983, Duan et al., 1984), which model the probability of nonzero costs separately from their level conditional on nonzero costs. The dependent variable is commonly log-transformed before OLS estimation to accommodate skewness. Predictions from these models must be retransformed to obtain estimates on the original scale, and these retransformations can be sensitive to model misspecification. In particular, heteroscedasticity can bias estimates drawn from the frequently used two-part logged dependent variable models even when smeared retransformation factors are used (Duan, 1983, Manning, 1998). More recently, generalized linear models (GLMs) have been proposed to facilitate inferences about predictors of expected costs (Mullahy, 1998). Manning and Mullahy (2001) compared the performance one-part transformed models and GLMs using simulated data representing various violations of model assumptions, and suggested some model selection criteria.

With these new approaches, more than a half dozen alternative estimators are available to the conscientious researcher. Depending on the characteristics of the data and the research questions, each of them could be the “best” estimator under certain circumstances.

In this paper, we model Medicare payments for elderly Medicare beneficiaries, comparing eight alternative estimators. Our results suggest that researchers modeling health care costs and use might first fit one-part GLMs, proceeding to two-part GLMs or OLS models with transformed dependent variables if warranted.

Section snippets

Objectives

Our objective was to model expected health care costs incurred by Medicare beneficiaries given their demographic characteristics and responses to survey questions about their health status and conditions. Other objectives might have differently shaped our choices. If interpretation of regression coefficients were of interest, then the interpretability of the scale on which they are modeled would assume greater importance. If it were important to model the full predictive distribution of the

Data and model

Data were drawn from the 1996 Medicare Current Beneficiary Survey (MCBS), which combines survey responses and administrative data for a random sample of Medicare beneficiaries (Adler, 1994). We restricted our analysis to aged, non-institutionalized, non-HMO Medicare beneficiaries who were eligible for both Medicare hospital insurance (Part A) and outpatient insurance (Part B), both of which were included in our definition of Medicare reimbursements. HMO members and those ineligible for Part A

Potential methods: what estimators should be considered?

Given our research objectives, none of the estimators suggested in the health econometrics literature could be rejected out of hand. Each of the estimators is described briefly below; see also the review by Jones (2000).

Model selection procedures

The models described in Section 4 are the alternatives from which health economists customarily choose when modeling health care cost or use data. The first step in evaluating these models is to examine the distribution of the data to be modeled in more detail and conduct the various diagnostic tests proposed in the literature.

As previously described, the distribution of costs in the MCBS sample was skewed (asymmetrically distributed) with many zeros. Would transforming the positive part of the

Fitting models to the entire sample

Nine alternative models were estimated on the MCBS data, including the standard OLS model. The models’ predictions were then examined to see how well they predicted costs. We also investigated how well calibrated they were against sample means for groups of observations in each range of predicted costs and for other analytically relevant subgroups of the sample.

Scheme 1, Scheme 2 illustrate the calibration of the alternative estimators, that is, how well predicted means agree with sample means,

Split sample cross-validation

We used cross-validation to evaluate reliably the predictive accuracy of the models, fitting the models to one part of the sample and assessing predictive accuracy on the remaining part of the sample. Thus, we simulated the accuracy with which a model fit to one dataset might predict the costs for individuals in another sample from a similar population. This approach protects us from misleadingly optimistic assessments due to overfitting of complex models. Furthermore, we cannot assess models

Discussion

Four out of the eight alternative models estimated here performed well in terms of calibration of predictions, mean square error, absolute prediction error, and cross-validated forecast error. One was an OLS model that used two smearing factors to correct for heteroscedasticity. Two were constant variance GLM models, in either a one-part or two-part overall formulation. Of these, the two-part model fit the data slightly better than the one-part model, but the difference was very small. The

Conclusions

This paper demonstrates that finding the “best” estimator for a given problem and data set can require a researcher to perform a large number of specification checks, some of which add little value. After performing the extensive plotting, testing, and cross-validation exercises described above, we found that at least four of the estimators performed well enough to be used in many applications. Future research could shed light on whether the models that performed best for our sample of Medicare

Acknowledgements

We thank Paul D. Cleary, Richard Frank, Emmett Keeler, Joseph Newhouse, and Kathy Swartz for helpful comments, Larry Zaborski and Kanika Kapur for assistance, and the Commonwealth Fund and the Center for Medicare and Medicaid Services (contract 500-95-007) for support. Melinda Beeuwkes Buntin acknowledges support from the Harvard/Sloan Center for the Study of the Managed Care Industry.

References (18)

There are more references available in the full text version of this article.

Cited by (583)

View all citing articles on Scopus
View full text