Too Close To Call / Si la Tendance se Maintient

Among all the pollsters for the last BC election, Forum did clearly better than anyone else. Pollsters seem to go for the "late swing" excuse again, yet the best polling firm conducted its poll... one week from the election! Forum wasn't spot on, but they were close enough that a late swing could indeed explain the discrepancies. Kudos to this firm as they were already the only one to catch the much closer race in Alberta (while still being off though) and the only one to have the Liberals much higher than anybody else in Quebec. So while not perfect, it seems that Forum polls are at least within the margins of error (or very close).

So what would the projections have looked like had I used only Forum's numbers? Well, the model and simulations would still have predicted a BCNDP victory, but the actual outcome would have been much more likely to happen. When I used an average of all recent polls, I had the BC Liberals with around 13-14% of winning. But even in the scenarios where Christy Clark was winning, 50 seats was really at the extreme of the intervals (and outside of the 95% confidence intervals).

But when using Forum's numbers only, here are the seat ranges (at 95% confidence) for each party:

BC Liberals: 30-52

BC NDP: 33-54

BC Green: 0-1

BC Conservatives: 0

As you can see, the actual outcome was indeed in the 95% confidence intervals. Remember, when we do our simulations, we account for the uncertainty due to polls as well as the electoral system. Statistically speaking, it'd have been acceptable.

Moreover, the probability of winning would have been 43% for the BC Liberals and 57% for the BC NDP. So again, while not spot on, it'd have been much closer to reality.

I think that from now on, I'll publish final projections based on Forum only, just as additional information. Although, I might just have jinxed the good streak for this firm!

The day before the election, I tweeted that the BC Liberals had more chances to win the election than the Boston Bruins had to come back against the Mapple Leafs (at 4-1 in the middle of the third period). Based on the polls, I had the BC NDP's odds of winning around 86-87%. Seems that the Bruis' win should really have prepared us for last night!. But I can at least argue that I technically had the Liberals victory as a possiblity (although the magnitude of the victory, with 50 seats for the Liberals, was really at the maximum of the projection ranges). However, the actual result was incredibly unlikely. At least based on the information we had.

Before I talk about how wrong the polls were, above is what the projections would have looked like if I had the correct vote shares. Details can be found here. As you can see, the model would have performed quite well, making the correct call in 75 ridings (so a success rate of 88%). Some of the mistakes include the ridings of Christy Clark, Oak-Bay-Gordon Head (where I didn't think Andrew Weaver would get such a big boost but it was a possiblity as well in the model) or a couple of close races in the lower mainland. It also seems the model would have made quite a lot of mistakes on Vancouver Island, probably because the regional swing (of the Green especially) was so different from the provincial swing. Since some of these 10 mistakes cancel out each other, the overall projections would have been really close. The fact is that if you came to my blog and input these vote shares in my simulator, you'd have won your political pool! So "my part of the job", i.e translating votes into seats, is clearly well done. Let's not miss the achievement here: by having the provincial vote shares only, the model could have correctly called the election as well as 88 % of the ridings. On top of that, if you look through the pdf, you'll see that more often than not, not only the correct call would have been made, but the vote shares in the riding are very close to the actual ones. It just shows how important the provincial swing is. So translating votes into seats is totally possible, with very little information (but a lot of work from me). Now, even more than before, the key is really to find or project the correct vote shares.

That, honeslty, shouldn't be my job. Pollsters are there for that. We've got a lot of polls during this BC election, including like 3-4 during the last two days. They were all showing the same outcome: BC NDP ahead by 5-7 points (except Forum who had the BC NDP up by 2 only, but they released the poll about a week ago, so they might just have been lucky). So even by averaging the polls, you'd still have been far off. If I accounted for the true statistical uncertainty due to these polls (i.e: margins of error), my simulations would never have allowed for the possibility of a BC Liberals win. Indeed, when you combine polls, margins of error go down. It's only because I add a lot more randomness that I technically had the actual outcome as a possibility. My only regret is that two ridings that were called with probabilities of 100% went wrong. I thought I was accounting for enough uncertainty, but it seems I'll have to add even more, both at the provincial and riding levels. The probabilities should account for the possible error of the polls. They already do, but not fully it seems.

Honestly, it's quite depressing and frustating to work so hard on building a model just to be so wrong because all the polls are providing you with incorrect information. In the US, during the last election, you didn't have to be Nate Silver to predict the election. A simple poll average would have been (almost) spot on. So pollsters in this country should really ask themselves why they can be so wrong. And not just one or two polls, but all of them! It's the same thing that happened in Alberta (and to a lesser extend in Quebec) last year (we could also include the underestimation of the Conservatives last federal election, although some polls were quite close). It's almost funny because people argue a lot about the correct methodology (online panel vs IVR vc phone calls), but the issue lies elsewhere. I'm sure pollsters will come out (they have started already) talking about how there was a late swing and blablbla. But to me, the problem is they don't poll the right people. Ekos is trying to identify likely voters but is obviously failling miserably.

I'll continue to build model and offer projections. After the last Quebec election, I decided to add uncertainty as to account for the possibility of big mistakes in the polls. I'll thus continue that way and hopefully, polling firms will improve on their parts.

[Update: even though we've got three new polls today (Ekos, Angus-Reid and Ipsos), adding them to the projections doesn't change anything. Actually, even if I was to use these three polls only, the projections below would still be valid. So I won't be updating the infography or the pdf]

Given the polls, past results and the electoral system, taking account the uncertainty due to theses factors, Too Close Too Call projects that the BC NDP will win a majority government on May 14th 2013, while the BC Liberals will form the official opposition.

Adrian Dix and his party have 87% chances of winning the most seats and 86% of getting a majority. For the BC Liberals, it'd take a mix of a serious (and systematic) bias in the polls as well as some "luck" or efficiency with the electoral system in order to keep power.

Just a couple of weeks ago, the possibility of a BC Liberals surprise didn't even exist. So Christy Clark has at least succeeded in bringing her party back in a position where they not only can avoid a landslide victory of the BC NDP, but can even dream of beating it. Still, with the odds being around 13%, you have to like risk to gamble money on the BC Liberals.

If you want the riding-by-riding projections, you can find them here.

Let's take a look at each party. First, the very likely next governming party in BC, the BC NDP of Adrian Dix. After three consecutive elections won by the BC Liberals, all with smaller and smaller margins, the BC NDP is poised to come back to power. While the odds are very good for this party, let's not forget that 4 weeks ago, they were projected to win a majority 100% of the time. They even had the possibility of winning as many as 80 seats. So, did Adrian Dix make mistakes during this campaign? I wouldn't say so. I believe it's more a return to the natural order. In particular, the BC NDP didn't drop in voting intentions (a little bit, but at almost 45%, they are very close to their average of the last 6 months). What really happened are the smaller parties (Green and Conservatives) declining. The only negative point is that the BC NDP wasn't able, it seems, to benefit from some of the Green support going away, at least not looking at the net effect. It's well possible some of the Green supporters did indeed go back to the NDP, but at the same time, some NDP supporters moved to the Liberals. At the end of the day, the net effect has been a very stable NDP.

Christy Clark started this campaign with very little hope of winning it. With around a 20-points deficits, it would have taken a miracle for her to remain Premier. Still, an effective campaign and the collapse of the BC Conservatives gave her a chance. It'll likely not be enough, but she'll make sure the BC NDP is at least facing a real opposition in Victoria. And while she was projected to lose her own seat for the best part of the election, we now project that she'll indeed be back in Victoria. For how long? And will it be as the leader? We'll have to wait and see.

The BC Green party has great expectations this election. But while polls still look good (especially considering they could get a positive provincial swing despite running 24 less candidates, thus meaning that in ridings where they do have candidates, the swing is definitely positive), a decline towards the end of the campaign puts the chances of getting MLA(s) in jeopardy. Their best chance has always been with Andrew Weaver in Oak-Bay-Gordon Head but it'll take a significant "candidate-effect" for him to win. Something comparable to what Elizabeth May achieved last election. By Canadian standards, this is pretty unique. I believe the Green will be in the race but will ultimately fail. The main reason being the BC NDP polling so high on Vancouver Island and showing no sign of decline.

As for the BC Conservatives, the trend of the last year is quite disastrous. They went from being in the race to beat the BC Liberals for second place to most likely finishing last (among the four parties) with no MLA. An objectively weak campaign marked by candidates withdrawals and other controversies, along with what has been considered the weakest debate performance, all point to a bad night for this party and his leader, John Cummins. At 6-7% provincially without a full slate of candidates, the party could be in the race in some ridings around Kelowna or in the Interior, but as the Green, they should ultimately fall short. Still, they could be the party with the best provincial swing, something not rewarded (yet) with this electoral system, but something not negligeable.

Finally, let's nor forget there are a couple of independents candidates that have a real chances, including 3 incumbents (only 1 of them actually elected by running as ind. in 2009). Independents are very hard to project and I won't feel bad if my model is off in these ridings. It's honnestly more a guess than anything else.

I'll post later a short posts "by the numbers" to complete this one. Expect a lot of probabilities and, obviously, numbers.

I don't expect a lot of new polls today as we probably got the last Forum, Angus-Reid and Ipsos. However, Ekos is supposed to release a poll later this afternoon. So expect the final projections to be posted soon.

Two of the four main parties in BC aren't running candidates in the 85 ridings. In fact, the Green party only has 61 candidates while the BC Conservatives have 56 (or 60 if you include the 4 candidates who are on the ballot without the Conservatives identification, due to an error in registration with BC Election). For projections, this creates essentially two problems.

1) Polls (usually) don't account for that and therefore overestimate the support for these parties.

Except for last night's Angus-Reid poll who made full use of its online panel to offer respondents only the options available in their riding (i.e: if you live in a riding without a Green candidate, AR would not have offered this option to you), most (if not all) polls simply ask the same generic question to everyone. So there could be a small but significant numbers of respondents who declare wanting to vote for the Green party for instance, even though there isn't a candidate in this riding. After all, it's quite reasonable to assume that most respondents don't actually know if there is a candidate for each party in the riding and will only find out on election day.

 When that's the case, the support for such a party is clearly overestimated. The effect could be particularly important with the Green and Conservatives parties since they aren't even close to having 85 candidates.

On the other hand, if the polling firm accounts for that (as did Angus-Reid), then the support for the Green party is unbiased. And if you look at the AR poll, you indeed find the Green and BC Cons below their usual levels in other polls. But in this case, it actually underestimates the actual level of support in ridings where there is indeed a Green candidate. This post from BC Iconoclast explains it clearly.

In average though, since most polls don't account for this, expect the Green and Conservative supports to be overstimated. I don't currently correct for this in my projections but will make sure to do so for the final ones.

2) The provincial swing is affected.

My model, as most of these type of models, use the provincial swing to make projections in every riding. While I do account for regional and other effects, I have to make additional adjustments for the Green and BC Cons. Here is why: the Green party got 8.2% of the votes in 2009, with 85 candidates. There are now 24 ridings where there isn't a candidate anymore. In these ridings, the average level of support received by the Green candidates was 7%. So you see that the BC Green party is "efficient" and has decided to run candidates in ridings where it has a better chance this time around (specifically, in ridings where there is a Green candidate in 2013, the average result for this party in 2009 was 8.4%). Still, these 24 ridings with an average of 7% account for a good part of the overall 8.2% level of support. In fact, without these ridings, the Green would only have received around 6% of the votes province-wide. So when the polls (assuming we corrected for the effect explained in part 1) show the Green party at (say) 10%, it means that in the ridings where there is indeed a candidate, the provincial swing isn't 2 points (10-8), but 4 points (10-6). If you don,t account or correct for this, then you clerly underestimate the Green candidates in your projections.

To take a similar example as BC Iconoclast, I currently have the Green party at 18% on Vancouver Island. So quite similar to most polls (which shows that my regional coefficients to transpose the provincial swing actually work). But this 18% is an average out of the 14 ridings of the island. In reality, the BC Green party only has candidates in 11 ridings. So if I average over these 11 ridings only, I find the Green standing at 23%.

In conclusion, remember that most polls will have the Green and Conservatives supports wrong. But count on me for making sure to account for that in my projections (as well as in the simulator that you can use on this site)

Depuis que Philippe Couillard a remporté la course à la chefferie du PLQ, ce parti trône en tête des intentions de votes au Québec. Le dernier sondage Léger pour le Journal de Montréal continue cette série. Les Libéraux sont à 35%, le PQ à seulement 27% et la CAQ tombe encore davantage et se retrouve à 19%. Clairement, Couillard réussit à aller chercher des appuis chez les deux autres partis.

En termes de projections, vous avez les résultats affichés. Les projections par comtés sont ici. En termes de probabilités, le 7 points d'avance des Libéraux leur donnent environ 90% de chances de remporter l'élection. Les chances d'une majorité Libérale sont d'environ 55%.

Le PQ devrait espérer une sous-estimation de ses appuis ainsi qu'un peu de chance avec le système électoral afin de remporter l'élection. En effet, alors que le PQ remporterait le plus de sièges dans 90 simulations sur 1000, la formation souverainiste ne recueillerait davantage de votes que le PLQ que 7 fois! La concentration du vote Libéral fait en sorte que le PQ peut remporter l'élection en ayant moins de votes. Cela est connu, il s'agît simplement d'une autre illustration de ce phénomène.

La CAQ qui avait déjà un taux de conversion votes-sièges faible n'améliorerait pas cette situation. Dans les faits, la formation de François Legault aurait de fortes chances de se retrouver avec très peu de députés. Tellement que Québec Solidaire pourrait en fait la dépasser!

En effet, QS grimpe à 11%. Une possible sur-estimation des sondages hors campagne électorale, mais peu importe. Avec un tel résultat, couplé à la chute du PQ, la formation de gauche décrocherait 4 sièges (Gouin, Hochelaga, Mercier et SMSJ) et serait dans la course dans quelques comtés supplémentaires (Bourget, Crémazie, Laurier-Dorion, Pointe-aux-Trembles et Rosemont). La division du vote serait bien sûr un sujet d'actualité, mais serait parfois retourné. Dans Laurier-Dorion par exemple, QS a davantage de chances de gagner (35%) que le PQ (0%).

Bien sûr, les sondages hors campagne sont moins fiables. Néanmoins, avec un gouvernement minoritaire à Québec, il y a toujours une chance d'élections. Avec la CAQ si faible, il y a cependant peu de chances que Legault ne fasse tomber le gouvernement.

With the two new polls released yesterday, in particular the Oracle Poll showing a 4-points gap only between NDP and Liberals, some might be tempted to think that we now have a wild race where everything can happen. News medias sell more if they write about a race, remember the last US election?

By averaging the two polls published yesterday (so the two most recent polls), we'd get the NDP around 41%, the BC Liberals at almost 36% and the Green and Conservatives at 12.8% and 8.8% respectively. This is still a 5-points lead for the NDP. This is actually not significant. Indeed, for these levels of supports and with a sample size of 1000, the lead needs to be at least 5.4 points to be significant at 95% confidence interval). But given that we are actually using two polls with total sample size of 1800, the NDP lead is indeed significant. But remember, not being significant only means that it's possible for the Liberals to actually be ahead. It doesn't mean it's likely to be the case! If you prefer, it means that while the Liberals could be ahead of the NDP, it's just as equally possible for the NDP to actually have a very important lead of almost 13 points (margins of error work both way, remember that!).

If I run my 1000 simulations using these numbers, I get that Adrian Dix would win the election 84% of the time! Remember, this accounts for the uncertainty (or randomness) due to the polls as well as the electoral system. So it shows clearly that while we do have a much closer race than at the beginning of this campaign, the odds are still in the BC NDP favor.

Even if I was to only use the best poll for the Liberals (the Oracle poll), where Christy Clark would only be 4-points behind, I'd still find that the NDP would have 76% chances of winning. But there, any over- or under-estimation by the polls is becoming critical as each point translates into many more seats and chances of winning.

In conclusion, yes polls aren't perfectly accurate and can be quite wrong at times, but you have to wonder how likely it is. You can mention the Alberta election, but you should also remember that polls are not that wrong in average. My model actually accounts for that by going way beyond the traditional margins of error. So yes it's possible for the BC Liberals to win next Tuesday, but it isn't likely. Still, for Christy Clark, the simple fact that a victory is possible is already a huge improvement over simply 2 weeks ago!