FAQ
Q: How does the model work?
A: Using econometrics and past elections' results, we estimate coefficients to translate province-level variations to riding-level ones. Specifically, if a party increases (since the last election) by (say) 2% province-wide, the model is able to translate this 2% into variations in every riding. The calculations involve regional effects, incumbency effect as well as the past election's results of all the parties. Here are the complete details if you are interested. In rare cases, we also account for riding-specific effects, such as Elizabeth May or the ABC (Anything But Conservatives).
Here is some additional information about what the swing looks like: http://2closetocall.blogspot.com/2011/03/what-does-swing-look-like-proportional.html
Here you can find a post about how to add uncertainty (like margins of error) in the projections:
http://www.tooclosetocall.ca/2011/04/adding-uncertainty-to-model.html
Pour les lecteurs francophones, ce billet présente les nouveautés du modèle 2.0 qui sera utilisé lors de la prochaine élection Québécoise. Le pdf mentionné auparavant reste valide, mais le nouveau modèle est plus robuste à l'extrapolation (par ex, si un parti augmente de beaucoup par rapport à ses résultats électoraux précédents).
Here is some additional information about what the swing looks like: http://2closetocall.blogspot.com/2011/03/what-does-swing-look-like-proportional.html
Here you can find a post about how to add uncertainty (like margins of error) in the projections:
http://www.tooclosetocall.ca/2011/04/adding-uncertainty-to-model.html
Pour les lecteurs francophones, ce billet présente les nouveautés du modèle 2.0 qui sera utilisé lors de la prochaine élection Québécoise. Le pdf mentionné auparavant reste valide, mais le nouveau modèle est plus robuste à l'extrapolation (par ex, si un parti augmente de beaucoup par rapport à ses résultats électoraux précédents).
Q: How is that different from DemocraticSpace or Three Hundred Eight or other models?
A: There exists a lot of models and they usually share the core principle: the results in a specific riding this election is a function of the past election's results as well as of the current level of supports for each party. Simplistic models usually use an uniform or proportional variation (HKDP for instance). For the former, it means that if a party's support increases by 2 percentage points in a province (between the last election and now), the level of support for this party will increase by 2 points in every riding. For the latter, the variation in every riding is proportionnaly the same as the provincial one (i.e: if a party's level of support increases by 5% (not percentage points!), then the party will increase its share of votes by 5% in every riding or, if you prefer, it will be multiplied by 1.05.) Those two models then simply assume the form of the variation and do not allow any region-specific effects.
DemocraticSpace uses past results to estimate regional coefficients of variation. This is very smart and we openly admit it was a source of inspiration for our model. Our model includes the regional effects but also take into account the previous level of supports for each party in each riding or the fact that one party is the incumbent in one riding. On top of that, we use econometrics methods that have been proven to be efficient for forecasting.
As for ThreeHundredEight, they recently started to (finally) provide some kind of methodology, as well as riding-level projections. The model starts from the proportional change one and add an incumbency effect, as well as some star-candidat/cabinet minister effects. Those two effects are halfway estimated (meaning they are estimated using raw averages, not actual statistical tools). While I think 308 is making a mistake by using the proportional changes as the base for its model (see the link above clearly showing the swing is not proportional).
DemocraticSpace uses past results to estimate regional coefficients of variation. This is very smart and we openly admit it was a source of inspiration for our model. Our model includes the regional effects but also take into account the previous level of supports for each party in each riding or the fact that one party is the incumbent in one riding. On top of that, we use econometrics methods that have been proven to be efficient for forecasting.
As for ThreeHundredEight, they recently started to (finally) provide some kind of methodology, as well as riding-level projections. The model starts from the proportional change one and add an incumbency effect, as well as some star-candidat/cabinet minister effects. Those two effects are halfway estimated (meaning they are estimated using raw averages, not actual statistical tools). While I think 308 is making a mistake by using the proportional changes as the base for its model (see the link above clearly showing the swing is not proportional).
At the end, we welcome competition. And simply because one model performed better for a given election doesn't mean it's necessarily the best. After all, the lottery winners are not better at predicting the numbers. However, in term of the underlying logic behind each model, we believe our model is the best. In particular, our model actually nests the others. Indeed, if the variation is actually proportionnally the same in every riding, then our coefficients will show that. As it turns out, it is far from being the case.
Q: Is it reliable?
A: Yes and no. It does work very well but the accuracy is largely dependent on the reliability of the polls. Indeed, if the polls estimate the Liberals at 30% but on election day this party gets only 26% of the votes, the projections will be biased. On the other hand, thanks to the huge number of opinion polls available during a campaing (especially during a federal election where we have around 4-6 polls every week), we usually have a good idea of where the parties stand.
This is also why we allow users to use the model themselves. For our own projections, we use an average of polls. But some people might believe more in one pollster (Angus-Reid seems to have been quite close to the actual results during the last couple of election, with Nanos in close second) or simply believe that the polls are wrong Therefore, you are free to make your own projections using the percentages YOU believe are the most accurate.
Q: Ok but how accurate is the model given that we have the correct percentages?
A: The methodology gives more details about that, but for the 2008 election for instance, our model (with the percentages of the election as input) would have correctly predicted (in the sense of predicting the right winner in a riding) 283 ridings, out of 308. As opposed to 268 with the proportional changes model for instance. Do not get fooled by some websites (especially HKDP) that claim to have been correct in almost 300 ridings. This is simply incorrect. What they usually mean by that is that overall (i.e: for the projections for the country as a whole), their projections were mostly correct (for instance, they predicted the Conservatives to win 145 seats while they actually won 143). That only happens because a lot of mistake at the riding-level are actually cancelling each other! Our model makes less mistakes and is more accurate (in terms of percentages). Again, this is simple to explain since our model nests the other ones. By allowing the coefficients to vary according to the data (and thus, allowing them to potentially take the same values as the ones assumed by other websites), we cannot do worse.
Q: Can I use the model to see what would happen if the Bloc didn't exist or if the NDP was at 55%?
A: The new model (currently used to make projections for Quebec) can handle this, but you have to remember that it can be less reliable. This is pure extrapolation and therefore, the past observations are less useful in order to make predictions. The rule of thumb is to stay within + or - 10% or the highest and lowest score of each party observed during the last three elections. Beyond these margins, the model begins to extrapolate.
