Comparative efficiency of national health systems: cross national econometric analysis

Theory

Efficiency is defined as the ratio of the observed level of attainment of a goal to the maximum that could have been achieved with the observed resources. Normally, outputs are zero when inputs are zero. In health, however, health levels would not be zero if there were no health expenditures—that is, no health systems. So to measure the contribution of the health system we have to determine what it achieves in excess of what would be achieved in its absence (the minimum). Accordingly, we define performance as the current level of population health, in excess of the estimated minimum, compared with the maximum achievable level of health given the inputs. Because of the similarity between performance and efficiency, we use the terms interchangeably.

Neither the maximum (frontier) nor the minimum levels of health are observable, so they have to be estimated. Two strategies could be used for estimating the maximum. One involves defining feasible interventions, identifying their costs and outcomes, and choosing those that maximise health for the available resources. This approach has not been widely used 11 12 because of data limitations but is currently being pursued by the World Health Organization.13

The second approach, which we have used here, estimates the maximum from a sample of observed inputs and outcomes. This approach requires the relation between outcomes (population health) and inputs to be specified. We estimated this relation with a form of regression analysis that shows how health levels vary with inputs. The country with the highest health level, after controlling for inputs, is the most efficient. The maximum is the level of health the most efficient country would have produced at each observed combination of inputs. Efficiency of other countries is measured with respect to the maximum. Inefficiencies might be from wastage or because the most cost effective set of programmes or interventions are not used. Further details of the method are given on the BMJ's website.

Data

We estimated the efficiency of 191 countries from data for 1993-7. Population health was measured as healthy life expectancy (box). The health system input was health expenditure per capita measured in 1997 US dollars (adjusted for the cost of a generic basket of goods in different settings).

Levels of health are not solely affected by health systems.9 The most widely accepted other determinant is education, which is strongly associated with the health of both children and adults in developed and developing countries. 14 15 Educated people translate information and health services into health more effectively than uneducated people do. We used a summary indicator of educational attainment—average years of schooling in the adult population.10

We did not include income per capita because income is highly correlated with both health expenditure and education and complicates statistical estimation. Moreover, income does not directly contribute to health but acts through factors such as education, housing, and food intake. Inclusion of the part of income acting through mechanisms other than health expenditure and education made little difference to our results—the rank order correlation of efficiency scores was >0.99.

We estimated the minimum achievable health in the absence of a health system from observations on 25 countries before the existence of a modern health system (average year, 1908). Health levels were correlated mainly with literacy. We estimated the minimum health level for 1997 on the basis of current literacy rates as though the 1908 relation still applied.16

We generated an uncertainty interval as well as a point estimate of healthy life expectancy for each country.9 For all countries, we randomly drew observations from the uncertainty distributions and estimated efficiency and rank, repeating the procedure 1000 times with slightly different results. The reported efficiency estimate is the mean score for that country, and the uncertainty interval represents the range in which estimates fell, omitting the bottom and top 10%. Rank was based on mean efficiency, and rank uncertainty intervals were generated in a similar manner.

Validity of findings

Although other non-health variables affect health (housing quality, environmental conditions, etc), relevant indicators are difficult to find or estimate for many countries. In addition, many are highly correlated with educational attainment, which we used because it functions as a broad measure of non-health inputs.

Omission of non-health variables reduces the estimates of efficiency. On the other hand, the measurement strategy biases estimates upward. The fact that five countries have efficiency scores >0.97 does not mean they can improve performance by nearly 3%. It means they could improve by 3% compared with the most efficient country, but we have no way of estimating the potential of the highest performer to become more efficient. Microlevel studies suggest the potential is there nevertheless.

Reasons for inefficiency

We found that efficiency is positively related to health expenditure per capita. Performance increased greatly with expenditure up to about $80 per capita a year, suggesting it is difficult for systems to be efficient at low expenditure. There seems to be a minimum level of health expenditure below which the system simply cannot work well. We estimate it would cost just over $6bn a year (<0.3% of global annual health expenditure) to increase health spending to this threshold in the 41 countries with lowest expenditures.

Despite the need to increase funds in poor countries, there is enough variation in efficiency at all levels of expenditure to suggest that using current resources better could improve health considerably. Reducing wastage is one way, but the studies of Tengs and Murray et al show that allocation of resources is also important. 12 20 They argued that health in the United States and sub-Saharan Africa could be greatly improved by reallocating available resources from interventions that are not cost effective to those that are more cost effective but not fully implemented.

Another possible reason for inefficiency is that goals other than health may be deemed important. The World Health Report 2000 recognised that countries may also wish to reduce inequalities or increase the responsiveness of the system.9 The efficiency of health systems in achieving all defined goals has been explored elsewhere.21 The analysis produced some changes in rank—for example, France was estimated to have had the most efficient system overall—but, in general, countries efficient in producing health are also efficient in producing other goals.21

Future research

Our conclusions are, of course, tentative. The quality of data across countries varies greatly, and only some of this is accounted for in our uncertainty analysis. Our main objective was to show that the attainment and efficiency of health systems can be measured and compared across countries and over time. Much can be done to improve the data and methods, and WHO is currently working on this with member countries and academic experts. We believe this is critical work for health policymakers considering reforms. Without the ability to measure the inputs and outputs of health systems, they cannot know if the reforms achieve their objectives.