How did our estimates for CHIP50 compare to what happened on election day? As with 2020, CHIP50 underestimated Trump’s vote share. If we plot our estimates against actual outcomes for the two party vote, we see that we overestimated Harris’ vote share by about 3.4 points (figure 1). In contrast, the estimate of Harris’ vote share from 538 polls was a bit closer, overestimating her vote share by a bit less, 2.1 points, aggregating 538 for the same period of data collection for CHIP50, August 19 to November 3rd. (We note that the final estimate for 538 only overshot Harris by about a little over a point, because there was a downwards trend for Harris over the period we were in the field.)
Figure 1 suggests an intriguing pattern; CHIP50 overshot Harris far more in heavily Republican states; and by very little in heavily Democratic states, underestimating her vote share in only 6 states, all of them where her vote share was well above 50%. We don’t have a definitive explanation for this pattern (if you have hypotheses, please reach out to us). In contrast, the 538 polls systematically overshot Harris’s vote share by about two points, as noted (Figure 2); with no “tilt” in the relationship, with very low spread in that relationship. (We omit 13 states because there weren’t enough polls to offer an estimate.) Note that our collective prior expectation should be that 538 would do better; it captures the input from many polls per state from professional pollsters whereas CHIP50 is a single set of polls that aren’t optimized to provide predictive signals. Beyond that, taking 538 polls as an indicator for the industry, pollsters on average did very very well in 2024 by historical standards.
Why did CHIP50 systematically overestimate Harris’ vote share? Besides the apparent trend towards Trump during October, we suspect that much of our miss was because of gaps in turnout by Democrats as compared to Republicans. As we noted in our earlier posts, we did not incorporate into our estimates the likelihood of individuals actually voting. (Professional pollsters do this, but to do this well would require far more resources than we have.)
Whether turnout was a driver of the systematic misses in figure 1 will become clearer over the next few months, as final voter files become available. However, analysis of overall turnout indicates that heavily Democratic areas had major drops in turnout relative to 2020 (and Republican areas did not). Thus, to take just a few examples of very heavily Democratic areas with major drops in turnout: Cambridge, MA, had a drop in turnout of 15%; Chicago a drop of about 8%; Cleveland a drop of 10%.These were part of a broader pattern of strongly Democratic areas having lower turnout than in 2020. And, unsurprisingly, if we assume individuals who indicated that they preferred Harris were less likely to vote, we end up with an estimate that is much closer to what actually occurred. Figure 3 provides estimates based on the assumption that voters who preferred for Harris were 10% less likely to turn out as those who preferred Trump. This is a plausible estimate, and would be roughly what would yield unbiased estimates for CHIP50.
We will return to the question of why our estimates deviated after voter data are available. Voter data include party registration and whether people voted; and we will be able to evaluate definitively whether and how much less likely Democrats were to turn out relative to Republicans.
Interestingly, we were correct in the evaluation that the electoral college was no longer biased in favor of Trump, because Harris had a much smaller drop in vote share in the swing states than in other states.
