No one doubts that the US spends more than any other country on health care.  Whether this higher spending produces commensurate health benefits, however, is far from certain.

In the April 2012 issue of Health Affairs, Philipson et. al. make an intuitively persuasive observation, one which they summarize in their recent Health Affairs Blog post (authored by Goldman, Lakdawalla, and Philipson): “We find that survival after diagnosis rose more quickly in the U.S. than the E.U.”

Given this observation, they go on to make an inference, namely that the US gets value from it additional expenditures on cancer care.  It would be nice were the world so simple.  But it’s not.

Early Detection And Overdiagnosis Biases

Philipson et. al. fail to appreciate the powerful early detection biases associated with survival comparisons both across time and place.  While they do acknowledge that advancing the time of diagnosis can spuriously elevate survival without any delay in the time of death (the so-called lead time bias), they assert in their blog post:

The possible existence of lead time bias is not, on its own, reason to question our conclusions, because we examine changes in survival over time.  This approach eliminates such bias in most cases.

But that would only be true if lead time was stable across time and place, or changing in exactly the same manner from country to country.  Given the enthusiasm for opportunistic screening in the United States — not to mention the high rates of cross-sectional imaging that add to incidental cancer detection — such an assumption is simply not credible.

Moreover, as pointed out in a letter by Dr. Keith Marton, Philipson et. al. ignore the yet more powerfully misleading effect of making cancer diagnoses in patients not destined to die (or experience symptoms) from their cancer (the so-called overdiagnosis bias).  Overdiagnosis can have a powerful effect on survival rates, even if no one has their life prolonged.

To understand why, imagine a country in which 1,000 people are found to have Cancer X because of symptoms (they all have progressive cancer).  Five years after diagnosis, 500 are alive — producing a five-year survival rate of 500/1000 or 50 percent — and 500 have died.

Now imagine the same country with lots of cancer screening and/or incidental cancer detection.  Perhaps 2,000 would be given a diagnosis of Cancer X, although 1,000 would actually have indolent forms and not be destined to die from their cancer.  Five-year survival will increase dramatically, to 75 percent, because the 1,000 people with indolent cancer appear in both parts of the fraction: 1,500/2,000.  But what has changed?  Some people have been unnecessarily told they have cancer (and may have experienced the harms of therapy), and the same number of people (500) still died from Cancer X.

Kidney Cancer In The US, Thyroid Cancer In South Korea, And Other Real-World Examples

Too hypothetical, you say?  Well it’s exactly what has happened with kidney cancer in the United States.  Over the last 30 years the rate of diagnosis has doubled, and its 5-year survival has increased from 50 percent to 75 percent.  Yet the mortality rate for kidney cancer is totally stable.

Kidney cancer is not the lone example.  The rates of diagnosis for melanoma and thyroid cancer have more than doubled in the United States, and their survival rates are rising dramatically.  But their mortality is stable.  And South Korea serves as the poster child for the problem — their rate of thyroid cancer diagnosis has increased 15-fold.  Thyroid cancer is now the most common cancer in Korea, more common than lung cancer, colon cancer and breast cancer.  The apparently good news of rising survival obscures the basic truth: Thyroid cancer mortality is stable and overdiagnosis is rampant.

Problems With Survival Statistics That Cannot Be ‘Modeled’ Away

Lead time and overdiagnosis combine to produce powerful biases in survival statistics.  This explains why survival statistics are increasing for all cancers, even those with mortality rates that are stable or rising.  The problem cannot be “adjusted” or “modeled” away.  To do so requires investigators to simulate the natural course of screen-detected and incidentally detected cancers; investigators must make assumptions on how much lead time and overdiagnosis have been introduced, which are the fundamental unknowns.

This is not to say that cancer treatment can’t improve (the 30 percent decline in breast cancer mortality is a tribute to improved oncologic care), but that survival is not a reliable measure of improvement.  The only way to disentangle the real value of improved survival from the spurious effects of changing diagnostic practices is to compare two groups that are treated differently, yet diagnosed in exactly the same manner — in other words, a randomized trial.

Finally, Philipson et. al. criticize a recent study by our colleagues Soneji and Yang as “fundamentally flawed” because it used mortality rates to assess progress:

The bottom line is that mortality reflects treatment, but it also reflects the number of people who get cancer.  This is precisely why we focused on improvements in cancer outcomes after diagnosis, because at that point, trends in the causes of cancer no longer matter.

But words matter here.  Mortality does reflect the number of people who get progressive cancer; survival reflects the number of people told they have cancer (and the time at which they are told).  And the fact is that the underlying risk of developing progressive cancer is much more stable than the rapidly changing landscape of diagnostic practices. (Even if every smoker stopped today it would take decades to see the full impact on the amount of lung cancer detected, while the advent of PSA screening nearly doubled prostate cancer detection in a few years.)

That’s why cancer epidemiologists agree – along with the Extramural Committee to Assess Progress Against Cancer: The way to assess progress in cancer care is with population-based mortality rates.