Health Affairs

Imagine the following. You are the senior White House health policy adviser, and you’ve been told to brief the president and his cabinet officials about the number of Americans who lack health insurance. The president turns to you, and you say: “Mr. President. The government has four different national surveys that count the uninsured. Unfortunately, each survey has different estimates that range from 18.9 million to 45 million. Each one measures things distinctly, and we’re not sure which, if any, of them is correct.”

That was the situation facing Doug Badger, then the White House health policy adviser, in the middle of 2004. Following passage of the Medicare Modernization Act, the Bush administration was contemplating new polices for dealing with the uninsured. Health and Human Services Secretary Tommy Thompson was pushing hard to follow up on the success of the Medicare prescription drug benefit with a new initiative for the uninsured that would go beyond the tax-credit approach that was then administration policy.

This is the chart that Doug put in front of the president.

Estimates that varied from 18.9 million to 45.0 million. Uncertainty about whether people were answering that they were uninsured for the full year at the time of the survey or at any time during the year? HHS data vs. Census data? Not a good situation. Not the way you want to make policy decisions.

Now imagine how President George W. Bush must have felt. Unfortunately, he was neither the first nor the last president to face this problem. President Bill Clinton would have received a similar presentation a decade earlier, and President Barack Obama was probably told the same thing earlier this year.

This is the unstable ground these presidents have found themselves on. It is crucial that lawmakers, administration officials, policymakers, stakeholders, and the public have a timely and accurate estimate of the number of uninsured people. How are the Hill negotiators supposed to design a program to extend coverage to the uninsured when it’s not clear how many uninsured there are? How is the Congressional Budget Office (CBO) supposed to develop rigorous estimates of the low-income subsidy provisions without accurate data on the number of Americans living below the poverty line?

This topic will gain new visibility when the Census Bureau releases an updated set of estimates of the uninsured on September 10, just as Congress resumes consideration of landmark health care reform legislation. Two weeks later, visibility and pressure will intensify when Census releases a different set of estimates and a brand-new way to count the uninsured.

A solution is long overdue.

Two Administrations Attempt To Improve The Accuracy Of Data On Uninsured Americans

Steps have been taken to improve the accuracy of government surveys that estimate the number of uninsured Americans. In the aftermath of President Clinton’s health reform efforts, HHS pushed hard to fix this problem and a number of other parallel data-quality problems. As the principal deputy to the Assistant Secretary for Planning and Evaluation, Judy Feder tried to help resolve the problem. She pushed for the creation of the HHS Data Council to bring together the different agencies, at least within HHS, that were responsible for providing policymakers with timely, accurate data on the uninsured and other major policy issues. It worked well for a while, but when Judy left HHS, the Data Council lost its political leadership and clout. A lack of political interest and proper funding for the kind of methodological work that would improve the estimates across surveys meant that little progress was made and the fundamental problems remained.

When I became Assistant Secretary for Planning and Evaluation in 2003, I made data policy consistency a priority issue. Through the HHS Data Council, I invited the Census Bureau team that conducts the Current Population Survey and the Survey of Income and Program Participation to discuss how to improve the estimates of the uninsured. I also asked key analytic staff from the CBO, Congressional Research Service, Treasury, and Government Accountability Office to discuss how they navigated the problems with the current set of estimates and what they thought could be done to improve the situation. Next, I directed funding to support methodological research to improve the HHS and Census surveys’ estimates of health insurance coverage and income and launched two parallel studies of the Medicaid undercount — a gap between the Current Population Survey estimate of the Medicaid population and state enrollment data. My office contracted with one team of health economists and one team of actuaries to try to understand why Current Population Survey estimates of Medicaid enrollment were always below the actual numbers of enrollees reported by states.

Studies Look At The Problems Involved In Tracking Coverage And Income Levels

Two additional studies that my office funded were recently completed and presented at the 2009 AcademyHealth annual research meeting in Chicago: “A Comparison of the Health Insurance Coverage Estimates from Four National Surveys,” by Michael Davern and his colleagues at the University of Minnesota and Mayo Clinic, and “Income and Poverty in 2002: The View from Five Surveys,” by John Czajka and Gabrielle Denmead. These two studies identify specific methodological problems and provide solutions to guide the teams working on the individual surveys. They explore the somewhat unsatisfying hypothesis that when people are asked about their insurance coverage last year, they are really answering about their insurance coverage at the time the survey question is being asked — that is, a point-in-time estimate, not an all-year estimate. (This hypothesis is unsatisfying because it seems to assume that the Census Bureau doesn’t know how to construct a proper question or that the American public doesn’t know how to give a proper answer.) Getting an accurate estimate of the number of uninsured people all year or at a particular point in time is vital to good policy making. There are many people who have brief periods without insurance — for example, between jobs, after graduation but, before starting work. These people are of much less policy concern than those who have been uninsured for an entire year or longer.

Both papers are important in helping move the policy debate forward. In the uninsured study, Davern and colleagues discovered that the problem was much more a question of the length of the recall period; whether one person answered for the whole family or individuals were asked; use of confirmation/verification questions; inconsistent definition of what it means to be uninsured; and the methods used to impute answers when a respondent left a question unanswered. In the income study, Czajka and Denmead discovered that the problem was much more focused on how “income” was defined; how “families” were defined; and measuring both income and family composition monthly rather than yearly. These are all fixable problems. Both papers do what they were asked to do, but candidly are too academically polite for most policymakers.

These papers challenge us to rethink the borders between surveys, administrative (enrollment) data, and models. For the best income data, are we better off using a hybrid approach — a targeted survey like the Survey of Income and Program Participation for the lowest quintile of the population linked to the IRS’s Statistics of Income for the other four? For the best health insurance estimates, are we better off using Medicare and Medicaid administrative data for those subpopulations and using surveys for the rest? When most surveys rely more and more on sophisticated imputation techniques, including probabilistic modeling, where does the survey end and the modeling begin? These are vital data policy issues that need to be addressed if health services researchers are to provide policymakers with the most timely and reliable data.

Adding to these challenges, a new source of survey data that either could resolve many of these concerns or make them more complex is about to appear. The American Community Survey, successor to the Census “long form,” will report detailed health insurance coverage for the first time this fall. The sample size of roughly three million people is more than ten times the size of the other major surveys. Analysts hope that this new source will resolve many of the issues around the estimates of the uninsured and not make them more confusing than they already are.

Since I have been both a senior civil servant and a political appointee, this is my takeaway: there is a dire need in the federal government for a real data policy. This data policy would ensure coordination across survey teams; provide sufficient funding of major surveys and other data collection, linked files, and models; and make an early and aggressive investment in methodological work. Unfortunately, it is unclear who will take on this cause. While civil servants, including those who sit on the HHS Data Council, Office of Management and Budget, and Census, have great research expertise, they do not have enough control over departmental priorities and budgets to get the job done. And policymakers, although they have the control over priorities and budgets, tend to focus their efforts on tackling more immediate policy issues, not longer-term data policy concerns.

It is my hope and belief that if the political leadership at Office of Management and Budget, Assistant Secretary for Planning and Evaluation, and the Under Secretary of Commerce for Economic Affairs decide to make data policy a focus and a budget priority, they are in the strongest position to do so. These positions tend to have policymakers who have both the technical expertise to understand the importance and complexity of the problem and the political resources to make change possible. Regardless of which party they may represent, the need is the same. Now is the time to make the commitment, to take a stand, and to make sure that no future presidents have to be handed a chart like the one Doug Badger gave to President Bush.

 Email This Post  Print This Post

This entry was posted on Wednesday, September 2nd, 2009 at 1:09 pm and is filed under All Categories, Coverage, Insurance, Policy, Politics. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.