Photographer: Alex Wong/Getty Images
Photographer: Alex Wong/Getty Images

For several months now, whenever the topic of enrollment in the Affordable Care Act came up, I’ve been saying that it was too soon to tell its ultimate effects. We don’t know how many people have paid for their new insurance policies, or how many of those who bought policies were previously uninsured. For that, I said, we will have to wait for Census Bureau data, which offer the best assessment of the insurance status of the whole population. Other surveys are available, but the samples are smaller, so they’re not as good; the census is the gold standard. Unfortunately, as I invariably noted, these data won’t be available until 2015.

I stand corrected: These data won’t be available at all. Ever.

No, I’m not kidding. I wish I was. The New York Times reports that the Barack Obama administration has changed the survey so that we cannot directly compare the numbers on the uninsured over time.

The changes are intended to improve the accuracy of the survey, being conducted this month in interviews with tens of thousands of households around the country. But the new questions are so different that the findings will not be comparable, the officials said.

An internal Census Bureau document said that the new questionnaire included a “total revision to health insurance questions” and, in a test last year, produced lower estimates of the uninsured. Thus, officials said, it will be difficult to say how much of any change is attributable to the Affordable Care Act and how much to the use of a new survey instrument.

“We are expecting much lower numbers just because of the questions and how they are asked,” said Brett J. O’Hara, chief of the health statistics branch at the Census Bureau.

I’m speechless. Shocked. Stunned. Horrified. Befuddled. Aghast, appalled, thunderstruck, perplexed, baffled, bewildered and dumbfounded. It’s not that I am opposed to the changes: Everyone understands that the census reports probably overstate the true number of the uninsured, because the number they report is supposed to be “people who lacked insurance for the entire previous year,” but people tend to answer with their insurance status right now.

But why, dear God, oh, why, would you change it in the one year in the entire history of the republic that it is most important for policy makers, researchers and voters to be able to compare the number of uninsured to those in prior years? The answers would seem to range from “total incompetence on the part of every level of this administration” to something worse.

Yes, that’s right, I said “every level.” Because guess who was involved in this decision, besides the wonks at Census?

The White House is always looking for evidence to show the benefits of the health law, which is an issue in many of this year’s midterm elections. The Department of Health and Human Services and the White House Council of Economic Advisers requested several of the new questions, and the White House Office of Management and Budget approved the new questionnaire. But the decision to make fundamental changes in the survey was driven by technical experts at the Census Bureau, and members of Congress have not focused on it or suggested political motives.

Sarah Kliff of Vox says we shouldn’t freak out, because these are the numbers that the census collects for 2013, so the change is actually giving us a good baseline. But I’m afraid I’m not so sanguine. As Aaron Carroll says: “It’s actually helpful to have a trend to measure, not a pre-post 2013/2014. This still sucks.”

The new numbers will suffer, to some extent, from the same bias that the old questions suffered from: People are better at remembering recent events than later ones. Quick: On what day did you last get your oil changed? What month was the wedding you attended last summer? If it was in the last few months, you probably know. If it was someone you’re not that close to … well, the summer months kind of blend into each other now that you’re a grownup, don’t they?

And what has been happening in the most recent months? A whole lot of change! Policies were canceled, benefits changed, people shifted around their coverage in anticipation of the new law. That doesn’t make for a very good baseline. It will be a very good measure of who has insurance right now, in 2014, but it’s not where I’d want to start my 2013 baseline for our new law. That’s why they should have done this for 2012 -- or waited until 2016 -- to give us actual comparable data for the transition period. So by your leave, I think I’ll continue to freak out for a bit.

I find it completely and totally impossible to believe that this problem didn’t occur to anyone at Census, or in the White House. It would be like arguing that the George W. Bush administration might have inadvertently overlooked the possibility that when the U.S. invaded Iraq, there would be shooting. This is the biggest policy debate of the last 10 years, and these data are at the heart of that debate. It is implausible that everyone involved somehow failed to notice that they were making it much harder to know the effect of this law on the population it was supposed to serve. Especially because the administration seems to have had a ready excuse as soon as people reacted to the news.

Even if the administration genuinely believes this is defensible, why would they give anyone reason to believe that it is cooking the books? Because those charges are being made, and they’re a lot harder to dismiss than the complaints about birth certificates or dark intimations that the administration has simply made up its enrollment figures out of whole cloth.

I just don’t get it.

I mean, I can certainly think of explanations, but I can’t quite bring myself to believe the worst of them. Which leaves me with the only slightly-less-utterly-appalling conclusion: At some point, very early on in the process, folks noticed that asking the new questions would make it difficult to compare Obamacare’s implementation year to prior years, and decided that assessing the effects of the transition wasn’t nearly as important as making urgent changes to … questions we’ve been asking basically the same way for a decade and a half.

No, wait, that doesn’t make any sense, either. Let’s go back to inexplicable, shall we?

If the administration is really serious about transparency and data-driven policy, as I’ve been told for a year now, then it will immediately rectify this appalling mistake and put the old questions back into circulation double-quick. But we’re more likely going to hear the most transparent and data-driven administration in history citing these data -- without an asterisk -- to tout the amazing impact of its policies.

To contact the writer of this article:
Megan McArdle at mmcardle3@bloomberg.net.

To contact the editor responsible for this article:
James Gibney at +1-202-624-1863 or jgibney5@bloomberg.net.