OK, I’m going to argue headlines.
It’s about the 2014 elections. Over at the Monkey Cage earlier this week, John Sides wrote up the latest Senate forecast from a simple model he developed with Eric McGhee and Ben Highton (Note: they’re all good friends of mine, as well as terrific political scientists). He gave it this headline:
Republicans have a 44 percent chance of taking the Senate. But it may be much higher.
Monkey Cager and statistics guru Andrew Gelman took exception to that, explaining that if the projection, depending on how they built the model, were actually for either a 44 percent chance or, say, a 64 percent chance, then the headline should have been (as Gelman titled his own post):
Republicans have a 54 percent chance of taking the Senate
So far, so good. The model seems reasonable to me, and Gelman’s critique sounds right.
But both are wrong.
The projections depend, of course, on building an accurate model (and John explains the issues involved there), but accuracy as John explains, depends on two key variables: economic growth in the first half of 2014 and President Barack Obama’s approval rating in June 2014.
We don’t know either of these values!
John gets around that problem by plugging in Obama’s current approval rating, and by projecting that economic growth in the first half of 2014 will match growth in the second and third quarters of 2013. That’s certainly possible, but it’s also a very rough estimate; both presidential approval and economic growth can change rapidly. Indeed, I suspect that there’s only a very small correlation between mid-year economic growth in one year and first-half growth in the next year.
Another way of doing it, and one which I probably would have preferred, would have been to run a table or graph with various economic and approval numbers, and let the reader eyeball the possibilities. See, for example, the way Alan Abramowitz handled his unknown variable (generic ballot) in a House projection post last year.
But mostly I’m annoyed that both authors attached very catchy, very exact numerical projections to something dependent on variables so unstable that they amount basically to guesses, and then slapped that number in a headline, where it’s bound to be noticed. (John’s version goes further, suggesting a trend in one direction when it might just as easily go in the opposite direction.)
For a better way, see John’s post on House projections, which he titled “There is no wave coming in the 2014 election.” That one worked for me; it implied that it was a static picture of a potentially changing situation. It also had the virtue of avoiding any specific numerical projection.
In general, I do worry some about the tyranny of charts and numbers. It’s a very, very, good thing that quantitative political science is reaching a larger readership, and that folks such as Nate Silver and Ezra Klein are bringing quantitative data and political science research to more people. But I sometimes worry about how these things get transmitted, with the risk of readers who have less statistical training taking specific numbers -- especially projections -- too literally.
Anyway: the articles themselves, as usual, are first rate, and I have no complaints about the model that John, Ben and Eric are using. So read the pieces! And read the Monkey Cage, in general. Just look very hard at the headlines.
To contact the writer of this article: Jbernstein62@bloomberg.net.
To contact the editor responsible for this article: Francis Wilkinson at +1-212-617-8699 or firstname.lastname@example.org.