As a high school statistical model, we strive to learn ways to improve on the complexity of our model by looking at the work of our more professional colleagues. In order to do this, we examined the model of popular magazine format newspaper The Economist. Though The Economist has far more resources to create a more advanced model, we wanted to see if Blair ORACLE’s methodology measures up to The Economist’s by completing a statistical analysis examining the difference between our predictions.
We chose to compare our own model’s and The Economist’s predictions for the Florida House races because of its history as as a key battleground state. In 2012, Obama won the state by 74,309 votes which was only .0088% of the votes. The state flipped in 2016 when Trump won Florida with 112,911 votes, 1.25% of the vote.
More specifically, we looked at districts with differences that we deemed substantial. We calculated a percentage of the prediction (POP) to see how dramatically the difference would affect our model’s determination of who the winner is and the probability of that person winning.
The POP was calculated by taking the difference in predicted Democratic percentage between our model and The Economist’s (e-p) and dividing it by our models predicted Dem percentage. This figure is our way of determining how dramatic the difference is compared to the result. The POP is the number we will be looking at when comparing the differences between the two models.
Methodology Comparison:
Our methodology and The Economist's methodology are similar in their basic structure: both factor in national mood/generic ballot, fundamentals, and partisan lean. Both of our fundamentals include whether it is a presidential year or a midterm and if it is a presidential year, whether the president is running for reelection. However, The Economist’s model, unlike ours, includes factors about the state of the economy, such as the unemployment rate. Our model looks at median household income only when there are no polls available to compare districts with Blairvoyance.
Another substantial difference between our models is the number of simulation’s ran. The election is only simulated 10,000 times in The Economist’s model, but ours 10,000,000 times for ORACLE of Blair’s. This could mean that our model produces a more statistically accurate representation of voting with less variation in the results. The Economist’s methodology also includes calculating the generic ballot accuracy by comparing it to what they a “perfect calibration” as well as accounting for the variability over the years of the number of people voting for the same party twice. This perfect calibration is the expected two-party percentage for the president’s party based on the generic ballot polls for a given year. Oracle of Blair similarly calibrates the weight to the national popular House vote from 2002 to 2016.
Economic differences:
In terms of the information factored into the models, a major difference between Blair Oracle and The Economist model is that The Economist includes information about the economy in determining the fundamentals and partisan lean. We obtained the unemployment rate for the districts from 2017 American Community Survey data and compared it to voting trends to predict how The Economist might have incorporated it into their model since they are not clear about how they incorporate unemployment data.
Analysis:
According to findings by the APM Research Lab, a lower unemployment rate tends to boost a Republican’s chances of election. The national unemployment rate is at an all-time low around 4%, so we expect that factoring in the economy into a model would skew the predictions based on historically high unemployment rates.
Districts with unemployment rates above 5% were highlighted in plum to visualise whether a high unemployment rate necessarily corresponded with a higher Democratic percentage. We noticed that the majority of the districts with high unemployment rates did correspond with a likely Democratic winner, but did not correspond with a higher vote share in The Economist’s model than Oracle of Blair. Data on unemployment rates was retrieved from sources that favored neither side, so any slightly higher percentage is due to minor differences in the model. This did not necessarily show that The Economist’s predictions favored the Democrats more than Oracle of Blair when there was a high unemployment rate. There is no noticeable pattern to the difference and the unemployment rate.Whether The Economist model predicted higher or lower Democratic vote percentages, the unemployment rate remained near 5%.
Districts 14, 15, 26, and 27 are classified as battleground districts by Ballotpedia, so we also paid close attention to those districts specifically. We looked at these districts to see if our prediction about the way the unemployment rate is used in The Economist model would demonstrate itself in closer races like these ones. In District 27, The Economist has Shalala (D) seated to win with a safe 62% of the vote. Oracle of Blair predicts Shalala winning with only 52% of the vote. In District 26, The Economist has Mucarsel-Powell (D) seated to win with 54% of the vote while Oracle of Blair predicts that it will be closer race and Mucarsel-Powell will slide through with 50.5% of the vote. In District 16, The Economist predicts Shapiro (D) will win with 51% of the vote, but Oracle of Blair has Buchanan (R) winning with only 54.6% of the vote. Only in Florida’s 15th district does Oracle of Blair predict a higher Democratic vote percentage (49% versus 45%).
The Economist did tend to predict higher percentages for Democratic candidates in these districts than Oracle of Blair by an average of 3.6%, but it does not appear that economic data is the main reason. This is because a statistically significant difference in economic data would produce a higher difference in percentages. The unemployment rate in these districts are not high, so the higher Democratic percentages refutes our prediction that The Economist’s economic data would produce lower Democratic percentages with low unemployment rates. The unemployment rates are mostly near 4%. This data suggests that there is another confounding factor or a combination of factors that could explain the true difference between our models.
Though the economy does not appear to be the most important issue to voters this election, we have to wait until Nov. 6 (and possibly later) until we can ultimately decide what factors influenced Oracle of Blair, The Economist model, and voters nationwide. Economic factors appears to be the largest difference in our methodology and The Economist's, creating significant differences in our predictions for certain highly contested districts, the largest being a 9.3 percentage difference for the swing district of Florida 27. After the election, we will take a closer look at the state of Florida to see whether our model or the The Economist's is more accurate, and if The Economist truly should have taken the economy into account.