In a recent paper, Soneji and Yang revisit a topic we first explored in the April 2012 issue of Health Affairs — namely, whether the U.S. gets value for its cancer care. We found that life expectancy after cancer diagnosis rose more quickly for patients in the U.S. than for patients in Europe. Moreover, while spending per patient also rose more quickly in the U.S., Americans still received good value from the health care system. Compared to the gains seen in Europe, for example, each additional life-year gained in the U.S. cost roughly $20,000 in additional U.S. spending.

Soneji and Yang re-examine trends in cancer deaths in the U.S. and Europe and draw different conclusions. While we welcome the attention paid to this important issue, Soneji and Yang’s conclusions rest on fundamental flaws in their own approach and a misunderstanding of the methods we use in our study.

To understand the value of U.S. cancer care, one must ask whether the health care system performs better for U.S. cancer patients than those of other countries and at what cost. In attempting to answer this question, Soneji and Yang ask whether more people die from cancer in the U.S. or in Europe. This isn’t the right question. The total number of people dying from cancer is a misleading indicator of health system performance. Factors like poverty, pollution, smoking, diet, and exercise all contribute to the number of people acquiring cancer and dying from it, and confound the effects of cancer treatments. The bottom line is that mortality reflects treatment, but it also reflects the number of people who get cancer.

Focus On Improvement After Diagnosis

This is precisely why we focused on improvements in cancer outcomes after diagnosis, because at that point, trends in the causes of cancer no longer matter. In particular, we focused on the question of most interest to patients and doctors: How long can a diagnosed patient expect to live? We then documented that life expectancy after diagnosis was higher in the U.S. in our baseline year of 1982. Even more importantly, life expectancy rose more quickly in the U.S. between 1982 and 2005 than it did in Europe.

Soneji and Yang’s study makes some basic errors cautioned against in the health economics literature. They measure cross-sectional differences in mortality across countries and then attribute these differences to the health care system. Soneji and Yang presume that, where mortality is lower, health care is better. Nearly 40 years ago, the famed health economist Victor Fuchs pointed out that residents of Nevada had a much lower life expectancy than Utah, despite similar availability of physicians, hospitals, and other health care resources.

The missing element in the comparison, of course, was the difference in lifestyle and health behavior between the two states. In our context, ample research demonstrates that the incidence and prevalence of diabetes, heart disease, hypertension, lung disease, stroke, and even cancer, are all higher in the United States than in Europe (for example, see Banks et al. (2006) and Michaud et al. (2011)).

It is untenable to argue that all (or even most) of the cross-sectional differences in cancer mortality are due to health care, and not to rates of smoking, environmental exposures, obesity, sedentary lifestyle, poor diet, and other factors. By examining changes over time in patient outcomes instead of a snapshot of cross-sectional differences, our longitudinal approach removes, as best one can, any baseline differences across countries related to health behaviors and other non-health care factors.

Like any other academic study, ours faced several analytic challenges, all clearly stated in our paper. One such issue is that changes in life expectancy after diagnosis can be confounded with earlier diagnosis and detection of cancer (so-called “lead-time bias”). The possible existence of lead time bias is not, on its own, reason to question our conclusions, because we examine changes in survival over time.

This approach eliminates such bias in most cases. Rather, one must argue the special case where lead-time bias grew more rapidly in the U.S. than in Europe. We addressed this issue in an analysis reported in the online appendix. The results showed that total mortality rates declined faster in the U.S. than in Europe, even for the two cancer types most likely to suffer from lead-time bias: prostate and breast cancer.

Our findings on mortality are significant, because total deaths—while susceptible to confounding by non-health care factors—are insensitive to lead-time bias. Simply diagnosing patients earlier without improving their actual survival outcomes would not affect the number of patients dying. The evidence on stage at diagnosis also supports this interpretation: 80 percent of U.S. cancer survival gains persist, even after adjusting for the number of patients getting diagnosed at earlier stages of disease. Indeed, we are not alone in reporting more rapid mortality improvement in the U.S. than in Europe — a 2011 National Academy of Science study by demographers Samuel Preston and Jessica Ho echoed these findings.

Soneji and Yang also make a second fundamental error by applying U.S. life tables to both E.U. and U.S. cancer survivors. Put another way, they assume that every death avoided from cancer generates the same number of additional life-years in the U.S. and in Europe (see Section E of their online appendix, and page 392 of the published paper). However, the central point of our study is that each cancer patient in the U.S. lives longer than in Europe, and that the number of life-years gained per patient varies meaningfully across countries. By ignoring this point, Soneji and Yang effectively assume what they claim to prove, namely that U.S. cancer patients do not gain more years of life than their European counterparts.

Cost Per Death Averted

If we put aside Soneji and Yang’s flawed life-years gained assumption, their analysis becomes much more consistent with the conclusions in our own paper. Taking the deaths averted from Exhibit 4 in their article and dividing the incremental cost of cancer care in the U.S. by the number of deaths averted, the cost per death averted (for cancers where the number of deaths averted was positive) ranges from $78,000 (stomach cancer) to $7.3 million (prostate). These are within standard ranges for the value of a statistical life, even before we address the confounding effects of non-health care factors in their analysis, and before we correct their unrealistic assumption that every incident cancer case—regardless of tumor type—costs the same.

Finally, several of Soneji and Yang’s assertions are quite puzzling and—in some cases—erroneous. For example, they claim to have replicated our analysis of mortality rates, but then report in their appendix that they used a statistical specification that differs from ours by allowing predicted deaths to scale disproportionately with the size of the population at risk. In statistical terms, we treat population as the “exposure” variable, as is the conventional practice and as we described in our appendix, while they treat it as a covariate like any other. Their approach fails to produce properly age-adjusted mortality rates, but instead posits an ad hoc and idiosyncratic statistical model of cancer deaths. This departure from our more standard method could in itself account for their inability to arrive at our results.

Perhaps most telling, Soneji and Yang claim that our estimate of 224,212 excess stomach cancer deaths in the United States is implausible, because the stomach cancer mortality rates were always below those in Europe during this period. However, our study did not aim to measure cross-sectional differences in mortality; we compare changes in survival outcomes over time. This example also serves to illustrate the fundamental differences in approaches in the two studies. They rely primarily on cross-sectional analysis, whereas we use a difference-in-differences design that can better control for unobservable factors.

We welcome robust scientific debate that moves forward our understanding of the world, particularly as it relates to the important issue of value in cancer care. However, by refocusing on mortality—and using a cross-sectional design for an international comparison—Soneji and Yang have moved the debate backward, not forward. We find that survival after diagnosis rose more quickly in the U.S. than the E.U. This is the quantity of interest to newly diagnosed patients and their physicians.