This is because his model is a whole world to itself, and within that world the number of things that can change the probable outcome are declining every day. It is exactly like the program that calculates the odds of a Texas Hold 'Em hand winning or losing when you watch the World Series of Poker. Every time a card is turned over, if it doesn't materially affect the odds in favor of the challenger, the odds of the high-hand holder go up. This is because there are fewer cards left that might change the game for the low-hand holder.
Thus, Mr. Silver is to a large degree correct. If the state polls are correctly modeled, and if no new data out of line with the existing data is introduced, time is grinding away the opportunities for anything different to occur.
Nevertheless he makes a huge error.
But many of the pollsters are likely to make similar assumptions about how to measure the voter universe accurately. This introduces the possibility that most of the pollsters could err on one or another side — whether in Mr. Obama’s direction, or Mr. Romney’s. In a statistical sense, we would call this bias: that the polls are not taking an accurate sample of the voter population. If there is such a bias, furthermore, it is likely to be correlated across different states, especially if they are demographically similar. If either of the candidates beats his polls in Wisconsin, he is also likely to do so in Minnesota....That "sixteen percent" chance that the assumptions are flawed is entirely bogus. The model can account for sampling error of the "plus or minus three percent" variety; there's no problem with that because you can give it a percentage using known methods. But there's no way to know what the odds are that a flawed assumption is making the data itself unreliable.
My argument, rather, is this: we’ve about reached the point where if Mr. Romney wins, it can only be because the polls have been biased against him. Almost all of the chance that Mr. Romney has in the FiveThirtyEight forecast, about 16 percent to win the Electoral College, reflects this possibility.
That's the one thing that the model can't actually measure. Any attempt to estimate it is a completely unscientific guess. Investment bankers and pollsters are each a class: they make slightly different guesses, based on their position and what they want to achieve, but they inform themselves based on talking to and watching each other. If they're wrong, it's as likely as not that they're almost all wrong.
There's simply no way of knowing what that means for the model, because the whole model is built around data shaped by their assumptions. It's like trying to guess what the odds are that Elvis is still alive in a nearby possible world: there's no way to put a number on it, because the world in which Elvis is alive is a world built out of entirely different facts.
If the pollsters are biased, the facts of the true world and the facts of Mr. Silver's world are simply not the same at all. His model won't just be wrong in some way that can be estimated and worked into the model. It'll be so wrong that it simply can't be applied to the actual world. Any resemblance between his world and ours will be accidental.
8 comments:
Mr. Silver's model is not related to reality. It's more than just flawed: it is stupid.
It reminds me of the MASH episode where everyone was so tired they were dreaming, and Charles Emmerson Winchester III dreamed of trying to treat a sucking chest wound by holding sparklers and doing a football cheer.
Silver knew this from the get-go.
Valerie
The problem with calculating the odds of Elvis being alive in an alternate world is that he's still alive in this one.
This impacts the odds concerning the alternate world, as the two worlds are not unrelated to each other, even if informed by differing facts.
When I'm able to better characterize the nature of the relationship, I'll be able to estimate with a measure of specificity the odds of his living in that other place.
Eric Hines
Well, actually, no, Mr. Hines. As explicated by David Lewis, the Elvis who (is or is not) alive in this world is the only Elvis. The Elvis who might be alive any other possible world is at most the counterpart of Elvis.
Denote the Elvis in our world "Elvis-0," while the nearest counterpart Elvis is "Elvis-1." A fact about our Elvis being alive is a fact about Elvis-0, not about Elvis-1. A fact about Elvis-1 is not a fact about Elvis-0.
Thus, the two streams are actually unrelated. Their apparent similarity is accidental.
This is a highly important thesis because it prevents philosophers from doing idiotic things with possible worlds theory. Otherwise they can waste hours with it.
What you're talking about a variety of what's called systematic error. In physics we try to estimate it by making predictions using as many different models (that fit other data well) as we can and looking at the spread. I've no firm knowledge of what political pollsters do about estimating systematic errors, but since they never quote anything but statistical errors I assume they ignore them and pray we don't notice.
Insofar as it's comparable, there's this bit of advice:
"If no pattern in a series of repeated measurements is evident, the presence of fixed systematic errors can only be found if the measurements are checked, either by measuring a known quantity or by comparing the readings with readings made using a different apparatus, known to be more accurate."
Here we don't know exactly what the error is, because we don't know if we should be looking at R+1 or D+4. But the good news is, a more accurate method of measurement is at hand.
One simple thing anybody can do (and I'm too busy this morning) is look at the poll results the day before and the election results the day after for a few years, and use the spread on that to estimate the systematic contribution. Of course that's the systematic error on old polls, but that gives a track record.
That's been done, actually, but the problem is that the data aren't clear. Polls have a good track record some years; other years (like 1980) they don't.
The problem isn't a systematic bias across different election years, but a bias at work in assumptions about this year (whichever year this is). For example, I expect this to be a huge turnout year across the board. Others expect turnout to be lower than in 2008. Polls of registered voters usually go better for Democrats than polls of likely voters -- so if turnout is higher, Democrats do better than expected (because the actual voters track closer to the registered voter figure than the likely voter figure).
On the other hand, sometimes people switch parties. Guessing about what the spread of Democrats, Republicans and independents will be also is a year-by-year guess that may or may not reflect reality. In 2008, a lot of people wanted to be Democrats, wanted to be part of the wave. This year? My guess is that fewer people will want to associate with that label, after four years of disappointment.
I just listened to a poll pundit speak about what seems to be a tightening of the independent numbers, which have been running pro-Romney by double digits. If the independents are now running closer to 50/50, it could be that they're breaking more for re-electing the president, but it could also mean that more people who previously didn't want to associate themselves with the "R" label are now content to do so.
I think it's going to be a nail-biter for those of us who actually believe that one candidate is better than the other and that the outcome is important for the country. There clearly is a wave of revulsion; the question is, how broad and deep is it?
Post a Comment