May 10th 2013: Two new polls, both necessarily have mistakes

Today, we've got two new polls from two firms that quite frankly, aren't the most well known in this business. The first one, from Hill & Knowlton strategies showing a somewhat similar situation as the other polls released recently: the BC NDP still first but with a lead of around 6-7 points "only". This poll has very original questions such as the voting intentions by types of vehicles driven. Is that fun? Yes. Is that useful? I'm not sure.

The second poll, from Oracle poll research, is more surprising as it shows a tight race between the two main parties. Indeed, the BC NDP would be at 41%, only 4 points ahead of the BC Liberals. With 1000 respondents, the NDP'S lead is almost non-significant.

Technically, these two polls aren't that far from each other and could just be examples of the natural statistical variation. Although some of you could argue that the HK poll is based on an online panel while the Oracle poll is a more traditional phone poll. I personally see both types of polls as valid as long as the correct methodology is used (question, weighting, etc). Recent elections in this country haven't really helped settle this debate.

However, there is one big difference between the two: the Oracle polls has the 4 main parties summing to 100% while the HK poll has 3% for the "other" parties. These are really extreme examples and are perfect for what I wanted to talk about: even though you can't be sure how wrong a poll is, you can usually be sure that there is indeed a mistake!

What do I mean by that? Well, for the Oracle poll, we know that candidates from more than 4 parties will receive valid votes for sure. So there is no way the four main parties sum to 100%. On the other hand, having as much as 3% in this category is too high and typical of polls. In 2009, the other parties got less than 2% (except if you include the BC Conservatives in this category, which was the case of most pollsters back in 2009). With now 4 parties included in the polls, it's a lot more reasonable to assume that these four parties will sum to 98%, if not more. Similar numbers can be found for other Canadien elections.

So everytime you see a new poll, if you see the other parties getting more than 2% of the vote, you can tell almost for sure that this poll is off, at least by a little bit.

The second source of almost-sure error are the Green and Conservtives. Both are more than likely overestimated in the polls. Just look at the polls of the 2009 election and you see a constant overestimation for the Green (by about 2-3 points, which actually could explain the underestimation of the BC NDP). This usually occurs mainly because the supporters of these "smaller" parties are less sure of their choice. All polls show this very clearly. On top of that, given that this time around the Green party doesn't even run 85 candidates, it's quite likely this overestimation is at least as big as it was in 2009. Same for the Conservatives who run even less candidates.

So bottom line, conservatively, most polls out there are off by a good 5 points (1+2+2). Now the real question is really: which party is underestimated? If these 5 points are evenly distributed between Liberals and NDP, the latter keeps roughly the same lead over the Liberals and its chances of winning the election are mostly the same (the electoral system can play a role here, but for simplicity sake, let's ignore that). But recent elections all had a very similar pattern: only one party was polled wrong. Last federal election, the polls were pretty spot-on for the Liberals and NDP, but were quite off for the Conservatives. Last BC election, the polls were off for the NDP. Last Quebec election, the polls underestimated the Liberals by about 4 points while being much closer for the other parties. For the current election, how these 5-points will be re-distributed could make the difference between a landslide victory for the NDP or a surprising win of Christy Clark.

I don't think it's very likely that only one of the main parties will be underestimated. The overestimated Green support will likely go to the NDP (as in 2009) while the Conservatives will help the Liberals. Still, my objective was simply to show you that polls usually have obvious errors to spot. It's harder to correct them though.

I'll update the general projections soon. Adding these two polls didn't change anything much though.