How bad were the polls for the Calgary mayoral election of 2017?

On Monday, residents of Calgary (and elsewhere in Alberta, but I'm only focusing on Calgary here) were asked to vote to elect their Mayor. Incumbent Naheed Nenshi won after a long and difficult campaign, beating challenger Bill Smith 51-44.

I live in Vancouver and don't follow the politics of Calgary. The only reason I had an interest in this race is because pollsters had very different numbers. Mainstreet had Smith easily ahead by 16 points while Forum had Nenshi winning by 19 points! Obviously one of these firms would be wrong and I wanted to grab the pop corn and watch!

Mainstreet is obviously the one that was wrong and they already admitted it. They had Nenshi behind by 16 and he won by almost 8 points. The total difference is actually 23.7 points! That is a massive failure. We have seen polls being wrong before (Alberta 2012, BC 2013, etc) but missing by 24 points is something else. This is more similar to the infamous Brandon-Souris by-election in 2013 when Forum was predicting the Liberals to be 29 points ahead when they lost by a little bit less than 2 - so a total error of almost 31 points. However, this was a riding poll (usually less accurate) for a by-election (harder to predict) with a really small sample size (below 400). The miss by Mainstreet here, with 1500 respondents, is more stunning.

What makes it worse for this firm is how confident its president, Quito Maggi, was that his numbers were right. Maggi is often active during election periods. He will tweet his numbers, comment them, defend them. I actually truly appreciate that. Much better than some firms who publish numbers and are never accessible for any question. Mainstreet was so confident, they even publish (and shared on twitter) a polling scorecard pdf for people to be able to easily see which pollster was the most right! Talk about shooting yourself in the foot!

Since then, Mainstreet has been criticized by many. And let's face it, when you miss as badly as they did for this election, people will remember. Forum still has a bad reputation among many people even though they have had good to excellent results in pretty every single election since 2012 (to be fair, Forum does have a tendency to publish some weird polls once in a while). So of course Mainstreet will have to deal with the blow to its credibility for a while. With that said, some of the attacks go beyond what you'd expect. Because Mainstreet is the main partner for Post Media, some are quick to jump to the conclusions that Mainstreet was publishing fake numbers and/or trying to change the race. Those are very serious accusations for a pollster. The objective of this post is to determine how badly Mainstreet really did and to put this failure in perspective.

Polls were bad overall, Mainstreet was just the worse

Mainstreet was obviously very, very wrong on Monday. But Forum and the third pollster, Asking Canadians, were far from being good. They did get the winner right, but their numbers were also fairly off. Look at the table below.

Look, in particular, at the corresponding margins of error associated with the Mean Square Error for each firm. The Mean Square Error is a standard measure of accuracy and widely used to compare polling accuracy in the industry or in academic papers. If the estimator -the poll here- is unbiased, it's also an estimator of the variance and we can therefore calculate the corresponding margins of error. This is how I proceed to calibrate my model and simulations. For instance in France for presidential election, the average error is usually small and the corresponding margins of error are around 3-4%. On the other hand, for US election, they are closer to 6-7%. See it as a general measure of the effective polling accuracy as opposed to the theoretical one. At 17%, Mainstreet is doing terrible but the other two firms are way above what we usually observe in the industry. If Mainstreet was off by almost 24 points in total, Forum was off by more than 11!

What this means is that this was a difficult race to poll. One possible explanation is the increased turnout that went from 39% in 2013 to 58%! If there is one thing that is usually correlated with polling errors, it's a sharp change in turnout.

I have reached to Mainstreet to ask them if they knew what happened and they said not yet. Which is a completely fair answer. I'd actually be very skeptical if they could pinpoint exactly what went wrong. Also, they aren't hiding behind the usual "people changed their mind 2 minutes before voting" excuse often used by pollsters.

Could Mainstreet simply have been unlucky? After all, basic statistics teaches us that once in a while, you can get a bad sample and results that are very off. Technically speaking, we should actually get such a poll every 20 polls (the 19 out 20 thingy of the margins of error). For this part, I can safely answer: no, being unlucky wouldn't explain being off by 24 points. How do I know? I ran simulations. Specifically, I ran 100,000 simulations using the actual results of Monday and the sample size of Mainstreet. Essentially, I'm simulating taking 100,000 samples of 1500 respondents. The results are below with the gap Nenshi-Bill (positive means Nenshi is ahead in one simulated poll, negative means Smith was ahead).

As you can see, out of 100,000 samples of 1500 respondents, the most skewed sample I got was one were Smith was ahead by 3 points (remember, those simulations are drawing samples from a population where Nenshi is actually ahead by 8 points. This is really showing what can happen when you only have a sample of the population). So getting so unlucky that you get a sample where Nenshi was down 16 points is impossible. If anything, given that Nenshi was actually ahead by 8, getting a sample of 1500 respondents where Smith was ahead at all was incredibly difficult as you can see on the graph (very few points with a gap below 0 on the left). Sure, ti might be possible to get Smith up by 16 if you do a billion samples and you are incredibly unlucky in one of them, but this is becoming a stretch. For all intents and purposes, random sampling can't explain the error here.

While the failure by Mainstreet can't be explained by simply being unlucky, it doesn't mean the sample of Mainstreet was good though. Quito Maggi admitted it already. What it means is that this sample couldn't be bad simply because of bad luck. Other reasons were at play. At this game, your guess is as good as mine, but it could be anywhere from oversampling some demographics (maybe not enough young people as suggested by Maggi), to using incorrect weights not representative of the voters with this turnout.

Finding out why polls failed is never easy. To this day, we still don't really know what happened in Alberta in 2012 or BC in 2013. In the US last year, State level polls mostly missed because they apparently didn't sample enough white Americans without a college degree. In all likelihood, we might never know what happened here in Calgary.

The "Mainstreet was just making things up and/or had an agenda" criticism

Ok, this is getting beyond my expertise and what I like to write about on this blog. Quito Maggi runs his firm the way he wants. As I said before, I have zero problem with him being active on Twitter. I'm not going to judge what is a "professional behavior for a pollster". I myself am guilty at times of being too active online.

But I simply have no reason to believe Mainstreet would do such a thing. They have so far done fairly well in Canadian elections. They were fine in BC earlier this year (except for their riding level polls; Also, to be fair, pretty much every firm did well there this year) or in Nova Scotia (although Forum did better). They were also good for the federal election in 2015. Moreover, Maggi has previously been accused of being a Liberal, not a Conservative. So why would he use his firm to help Bill Smith who was the more right-wing candidate for the Mayoral election? Because Mainstreet has a deal with PostMedia, a group that owns more right-wing leaning newspapers such as the National Post? Sure. It's very easy to see bias where people want to find it. I mean, during an election, I can be accused daily of being a "Liberal shill" or "trying to help the Conservatives" depending on the projections I post.

And yes I'm aware of the dispute between Mainstreet and the MRIA. I have no comment on this. If you read more about the story, it sounds more and more like an internal dispute between two organizations. It seems more an issue of PR and behavior of Maggi than of the reliability of the firm and its numbers.

At the end of the day, as far as I'm concerned, Mainstreet's track record so far had been very good. This giant failure in Calgary will not change my mind. All it does is remind us why taking an average is better (you had the results pretty much spot on by averaging here) and that failure can happen for pollsters. Should the be more humble? Sure, maybe.

If a federal election was tomorrow, Liberals would likely be held to a minority

Polls haven't been good for Prime Minister Justin Trudeau and the federal Liberals. Tons of articles have surfaced online during the last week talking about it. So let's take a look at what the current polling average is telling us.

For this post, I used the most recent polls from Angus-Reid, Nanos, Ekos and Ipsos. All polls have been conducted in the last month. Both Nanos and Angus-Reid actually released more than one poll during that period but I only selected their most recent numbers. Why? The idea here is to get an average with a variety of firms as to avoid the possible bias of one individual firm. I could have included two Nanos (very good firm) but they do a rolling average anyway. I didn't include the most recent poll from Forum because it was over a month old (but notice that while Forum's numbers are still clear outliers, it remains that this firm was the first to really observe a decline in the Liberals' support. As I had said last month, dismissing Forum completely is ignorant). Also, the Nanos and Angus-Reid were published -and conducted entirely in the case of the latter- after the election of Jagmeet Singh at the head of the NDP. It's too few data points to draw any conclusion about the effect of his election (if you compare the last two Angus-Reid, the NDP is at +4 but it could very well be statistical noise and the objective of today's post isn't to assess the impact of the new NDP leader).

I don't weigh based on sample size or anything because those haven't proven to really improve polling average accuracy in Canada. Also, just to be clear, this is a very raw average of the numbers taken at face value. I didn't look into the number of undecided, incumbency, etc. We aren't in an election period, I really just want a rough idea of where each party stands.

The results? Here below.

Vote intentions; Seat projections with confidence intervals; Chances of winning the most seats

As mentioned in the title -no clickbait here!- the Liberals would still be favourite to win an election held tomorrow, but they wouldn't be guaranteed a majority. The actual odds for a majority are 34%. Just 2 months ago, it was difficult to imagine a scenario where Trudeau wouldn't win the most seats -along with a majority- we are now in a situation where Andrew Scheer has an actual path to have the most MPs at Ottawa (and a tiny chance, 2.5%, of a majority). What a dramatic change in a relatively short period of time. Why is that? As speculated before, I can only think of the tax reform proposal of the Liberals as angering some voters.

Now, make no mistake. Trudeau and his party are still in a great position and the next federal election is 2 years from now. Still, you can't deny that the numbers simply do not look as great as they used to for the Grits. They still have the Atlantic and Quebec as a source of as high number of seats while Ontario and BC are now true battlegrounds. If the election of Singh sinks the NDP even more in Quebec -a real possibility- then the Bloc could climb back to 25-30% in la Belle Province and become an actual threat again.

Andrew Scheer's chances, at around 25%, are quite close to the chances I was giving him to win the CPC leadership! If he can somehow win back some of the Blue Liberals who left Harper in 2015 while keeping Alberta and most of the Prairies, he'd then be in a position to become Prime Minister. The sore point for him is Quebec. In the latest Nanos, the CPC is at barely 8% there! I admit that I have a hard time seeing how Scheer could suddenly conquer the French province and he might need the Bloc to take seats away from the Liberals. The by-election next week in Lac-Saint-Jean will be interesting. If the Conservatives want to remain relevant in Quebec, they need a good showing there. The current projections give the Tories a 93% chance of winning the seat, but these numbers do not account for the loss of a popular candidate such as Denis Lebel (or the fact it's a by-election). I'll write a piece about the by-election later this week but for now, let's just say that a loss there would be bad news for Scheer.

For the NDP, there is no path to forming the next government, at least not currently. We'll see in the coming weeks if Singh can help his party climb back to at least 20% and above. For now, the silver lining is BC where the federal NDP is most likely helped by the provincial ones (new NDP government with a currently popular Premier).

In conclusion, there is no denying that the numbers have fallen for Justin Trudeau. To be fair, it might simply be a return a more normal situation after two years without any opposition - both the CPC and NDP were without a leader- and a very long honeymoon. I realize many have written this before but I believe the downward trend is more visible right now. Still, Trudeau remains the big favourite to win reelection in 2019 and even a majority is still perfectly reachable. Following polls and projections 2 years from an election is tricky. I don't think we should dismiss the numbers outright, but we also need to remain cautious not to see too much in them. I'll add this as a conclusion: I think it'll be very interesting to see if Singh can go and steal some of the progressive vote from Trudeau. If he can, then don't be surprised if the CPC start polling ahead next month.

La CAQ triomphe dans Louis-Hébert, le PLQ mange une volée

La CAQ triomphe dans Louis-Hébert, le PLQ mange une volée
Waou, quelle victoire pour la CAQ dans la partielle de Louis-Hébert. Et avec une participation plus que décente (52%). En dépassant, les 50%, Geneviève Guilbault fait son entrée à l'Assemblée Nationale et remplacera le député Libéral de longue date Sam Hamad.

Les projections donnaient le PLQ favori (en utilisant les sondages récents provinciaux. En utilisant les chiffres régionaux, cela variait d'un avantage au PLQ à une course serrée). Elles avaient bien sûr tort. Les partielles sont toujours difficiles à projecter, mais celle de hier a des résultats plus surprenant que d'habitude. Il faut dire qu'entre les raisons du départ d'Hamad et les histoires durant la campagne, il fallait possiblement s'attendre à des surprises.

Je veux simplement illustrer ici à quel point la contre-performance du PLQ est significative. Avec seulement 18.7%, le PLQ est en-dessous de mes marges d'erreur (qui étaient déjà doublées). Et ces projections avaient été faites avce le PLQ à environ 30% provincialement (probablement le plancher pour ce parti à l'échelle de la province). Dans ce comté, on parle ainsi d'une baisse de plus de 30 points par rapport à 2014! Le sondage local fait par Eris Media a bien fait, même si l'ampleur de la victoire de la CAQ sur le PLQ avait été sous-estimée.

Nous avons eu 15 partielles dpeuis 2014, le PLQ y avait obtenu de relativement bons résultats. Par rapport aux projections, les résultats avaient été entre 24 points supérieurs (Renée-Lévesque où le PQ s'était effondré) et 7 points inférieurs (Sainte-Henri-Sainte-Anne). En moyenne, le PLQ avait récolté environ 4 points au-dessus des projections, une erreur moyenne tout à fait correct, surtout pour des partielles (et en fait, l'erreur moyenne n'était qu'une sous-estimation de 2.6 points si on retire René-Lévesque).

Le tableau ci-dessous devrait vous convaincre que le PLQ ne mange en général pas une telle volée lors des partielles.

Différences entre les résultats et les projections (en points de pourcentage):

La candidate PLQ dans Louis-Hébert a récolté 20 points de moins que prévu! Donc oui le PLQ avait eu de mauvais résultats dans des partielles, mais ce parti n'avait encore jamais connu une soirée avec une telle contre-performance. Les électeurs de ce comté ont de toutes évidences décidé d'envoyer un message. Il faudra maintenant voir si ce message sera confirmé dans un an lors de l'élection générale.

La CAQ a fait mieux que prévue et réussit ici à décrocher un siège important. Après avoir échoué dans Saint-Jérôme l'année passée, le parti de Legault ne pouvait se permettre de laisser passer une autre opportunité dans une région avec du potentiel.

Le PQ a fait tel que prévu, peut-être un petit mieux.

QS a un peu déçu mais ce parti ne semble pas accorder la même importance à toutes les partielles.

Pour le Parti Conservateur, le 4% est un peu faible. Il semblerait que ce parti n'ait pas réussit à capitaliser sur le mécontentement des électeurs envers le gouvernment. Avec la CAQ au-dessus des 50%, le vote de droite a semble-t-il choisi son camp hier soir.

Au final, cette partielle pourrait avoir des conséquences importantes. Un remaniement ministériel est quasiment garanti. Du côté de l'opposition, le PQ et la CAQ vont continuer de proposer des visions différentes mais aucun parti ne va pouvoir prétendre être LA meilleure alternative (la CAQ va bien sûr essayer après cette victoire mais la performance du PQ montre un parti vers les 25% provincialement, donc pas loin des deux autres). Je continuerai à faire des projections pour les partielles malgré les erreurs. Tant que l'on garde en tête qu'il est difficile de les projeter, je crois que l'exercice reste intéressant.

Partielle dans Louis-Hébert: avantage PLQ

Ce lundi, une nouvelle élection complémentaire aura lieu dans Louis-Hébert. Depuis 2014, nous avons vraiment eu un nombre record d'élections partielles.

Cette élection a été marquée par pas mal d'histoires et de rebondissements concernant les candidats. Tant le PLQ que la CAQ ont dû changer de candidat par exemple!

Pour le PLQ, cela complique une tâche déjà possiblement difficile avec le départ d'un député de longue date, Sam Hamad. Ce dernier avait facilement remporté ce compté en 2014. Cependant, entre les affaires d'Hamad et la forte baisse du PLQ depuis la dernière élection, pourrait-il se produire une surprise? Regardons cela de plus près.

En me basant sur les sondages récents, j'ai le PLQ devant les autres, assez facilement. Et ces chiffres incluent des ajustments spécifics à cette partielle (voir ci-dessous). Vu qu'il s'agît d'une partielle, j'ai inclus davantage d'incertitude pour les simulations. Les projections sont ainsi:

Perdre un député de longue date peut coûter 2-5 points. Dans le cas particulier d'Hamad cependant, les chiffres ne semblent pas indiquer un fort effet personnel. Ses variations d'une élection à l'autre étaient relativement similaires à celle du parti à l'échelle de la province. En 2008, alors que le PLQ récoltait 42% des voix, il avait 49% dans son comté En 2014, avec le PLQ à 41.5%, il avait 49%... J'ai ainsi inclus un petit effet contre le PLQ (2 points) mais rien de plus.

D'autre part, la CAQ semble très élevée dans la région de Québec depuis quelques mois, plus élevée que ce que mon modèle (qui se base sur les intentions de vote provinciales) ne prédit. Je lui ai ainsi accordé un bonus de 2%. Cette partielle va en fait être un vrai test pour la CAQ, un parti qui jouit d'une poussée dans les sondages depuis quelques mois mais qui n'a pas été particulièrement impressionnante lors des partielles depuis 2014.

Le PQ n'est probablement pas dans le coup dans ce comté, alors je vais me concentrer sur le duel PLQ-CAQ.

QS voudra probablement montrer que les sondages le plaçant au-dessus des 15% sont réels. Pour se faire, un résultat au-dessus des 10% dans ce comté est nécessaire.

Le Parti Vert présente son chef, Alex Tyrrell... à nouveau! Ce parti semble complètement mort et son chef s'était déjà présenté dans Arthabaska et Gouin!

Il y a aussi le Parti Conservateur (qui continue d'être sondé vers les 2-3% chez Léger mais n'est même pas inclus chez Mainstreet...). Un résultats au-dessus des 3% est impératif pour ce parti dans cette région. Le dernier Léger le plaçait carrément à 8% dans la RMR de Québec, mais avec un très faible échantillon.

Revenons aux partis principaux et leurs candidats et allons au-delà des sondages.

Geneviève Guilbault, la candidate de la CAQ, n'a que 344 followers au moment d'écrire ce billet (vendredi soir, 1 heure du matin heure de l'Est)! Dans les faits, son compte Twitter ne m'est pas apparu en faisant une première recherche Google... Elle a 605 personnes sur sa page facebook.

Ihssane El Ghernati du PLQ a 409 sur facebook et 254 sur Twitter. Donc elle fait encore moins bien.

Juste pour comparer, le candidat Péquiste, Normand Beauregard ne semble carrément pas avoir de compte Twitter! (mise à jour: voilà son compte avec un grand total de... 53 abonnés!) Sa page facebook a 562 personnes cependant. Je ne comprends pas comment, en 2017, on peut être candidat d'un parti majeur et ne pas avoir de compte Twitter.

Quant à QS et son candidat Guillaume Boivin, il a 373 personnes sur facebook et lui aussi ne semble pas être sur Twitter...

Il semblerait donc que la CAQ ait possiblement une petite avance. Ou du moins ces chiffres montrent une CAQ dans le coup.

Google Trends ne nous est pas vraiment utile avec une faible volume de recherches pour ces candidats.

Il nous faut aussi penser à la participation qui joue souvent un rôle lors des partielles. 14.96% des électeurs ont voté par anticipation. En 2014, 33.43% des électeurs avaient voté par anticipation (et 83.66% au total). Ainsi, le 14.96% semble indiquer une participation en baisse de 50-60%, ce qui est assez typique des partielles. Cela est bon signe pour les projections. N'oublions pas cependant que les Libéraux ont tendance à faire sortir leur vote durant les partielles.

La CAQ se doit de faire une bonne performance dans Louis-Hébert. C'est bien beau être sondé 2e depuis des mois, les résultats dans les partielles depuis 2014 ont été mauvais. Souvenons-nous de l'échec de Legault de reprendre Saint-Jérôme du PQ. Si le PLQ remporte Louis-Hébert facilement lundi,il faudrait commencer à être sceptique du potentiel de la CAQ de prendre le pouvoir. Lundi représente un test pour Legault et les sondages.

En conclusion, le PLQ perd certes un député de longue date mais fait en général sortir son vote lorsque cela compte dans les partielles, donc cela pourrait fort bien s'annuler. Le PQ devrait faire un peu moins bien que prévu mais cela ne change rien à la vraie course. La CAQ ne connaît pas beaucoup de succès lors des partielles mais en même tempes est sondée haute à Québec et a une petite chance de créer une surprise.

After Forum, Nanos is also showing the federal Liberals decreasing

After Forum, Nanos is also showing the federal Liberals decreasing
Remember that poll from Forum that so many dismissed as "nonsense"? Well, it might still be the case that Forum was off, but the recent trend in the Nanos numbers are consistent with the Forum's numbers.

The 4-week average of Nanos now has the Liberals with a 6 points lead over the Conservatives (38.5% versus 32.5%). That's down from a +12 two weeks ago!

Some will say "LPC still ahead". That is technically true but Nanos is averaging over 4 weeks, so adjustments are painfully slow. Let's do some math to find the numbers of this week (or the last two only).

Let's compare to the numbers 5 weeks ago (so we are comparing two different samples). On August 22nd, Nanos had the following numbers:

LPC: 41,9
CPC: 30.9
NDP: 16.4
Green: 5
Bloc: 4.3 (18.2 in Qc)

The new numbers with data from the last 4 weeks:

LPC: 38.5
CPC: 32.5
NDP: 14.6
Green: 5.7
Bloc: 6.2 (25.3 in Qc)

The LPC's lead went from +11 to +6 in 4 weeks. On the other hand, dropping from 41.9 to 38.5 is not a statistically significant variation with 1000 observations. So keep that in mind.

Comparing the two full sample is nice but if Forum is right, the changes occurred in September only. Using the numbers from each week and doing some rough/approximate calculations, we get the following numbers for the last week only:

LPC: 29.9
CPC: 37.9
NDP: 11.3

How do I get that? As I said, approximate calculations. But if the Liberals were around 41% in the previous 3 weeks and are now at 38.5%, it means the Liberals were significantly below 38.5% this week (that's how an average work after all).

Of course, these numbers here are an approximation and are based on a very small sample size (250). Still, if there is really something happening, Forum and Nanos are all of a sudden not that far from each other.

The questions are naturally: is something really happening and why? For the former, we'll have to wait for confirmation by other polls and firms. I maintain my theory that the Forum's variations were a combinations of a house effect (Forum has the CPC higher), sampling variation and true, genuine variation. I think Nanos is about to corroborate this theory. As for the latter... I can only think of the proposed tax reform.

If this is what is really going on, then this tax reform would have done what a new Tory leader couldn't do after his election, or what the controversy about the Khadr payout didn't do: make the Liberals support drop.

New Forum poll puts the Conservative ahead, really?

Alright, I guess I can,t avoid talking about the most recent Forum poll. If you weren't on the internet yesterday (or were too busy following the results of the German election), the latest numbers from Forum have the CPC in first place with 39%. The Liberals are second with 35%, then come the NDP at 15%, the Green at 4% and the Bloc at 5% (22% in Quebec).

If you've been following Canadian polls, you know that this poll is different. I mean, just a couple of weeks ago I had an article showing that Justin Trudeau would be overwhelmingly favourite to win an election held tomorrow. So, did something happen or is this poll just an outlier? Let's take a look and use this poll to discuss the possible scenarios for the Tories.

First thing first though: for some reasons Forum still get laughed at. I had people yesterday on twitter simply saying "who cares, it's Forum?". I don't get that. It's like this firm had one crazy bad poll in the by-election of Brandon-Souris and some people never forgot. Forum's track record since 2012 for provincial and federal elections is fairly good (they were even the best for a while). Just recently in BC and NS, they did very well (in BC though, their final numbers were good but they did one at least one weird poll during the campaign). All that to say that I don't see Forum as the gold standard for Canadian polling firms, but I also don't have any reason to dismiss their numbers right away. Quite frankly, I think it's stupid to do that.

Ok, so how different is this new poll? It kinda depends what we are comparing it to. Compared to previous Forum polls, the Tories are up 4 points while the Grits are down by 7. Big variations but Forum has had the Conservatives higher than most other firms for a while now. Not sure why but it's definitely there. Can we find reasons as to why the Liberals would be down by 7 in one month? The obvious candidate is the proposed tax reform that is generating a lot of noise (I'm sure you have a couple of independent workers friends on facebook who expressed their discontent with the reform). 7 points only for that? I don't know. But the nice thing doing my job is that I don't have to justify the numbers, so let's just accept the notion that Trudeau has had less positive press recently and it might have affected his number.

Compared to the other pollsters? The CPC is 9 points above the most recent Nanos (which is super slow to adjust because it's a rolling average over four weeks!) and 7 above the latest Mainstreet. it's also 8 points more than according to Abacus. For the LPC, the situation is similar but inverse obviously. So it certainly looks like this Forum poll is an outlier, at least compared to the poll average.

What if we look at the provincial numbers? In Ontario, Forum has the CPC at 42% and the Liberals at 37%. Compared to Mainstreet and its 44-36, it's not that different once we account for the large margins of error and could indicate a tight race in this province. Compared to Abacus (28-51 respectively) and it's obviously very different. But Abacus' Ontario numbers actually look more outliers than Forum's. Campaign research was showing a situation very similar to Mainstreet. So, all in all, Forum has numbers in Ontario that are different from the average but nothing we couldn't expect with small sample sizes.

In Quebec, The Liberals's lead over the CPC has been +22 at Abacus, +27 at Mainstreet, +28 at Nanos and +33 at Campaign! For Forum? It's +14 only!

You get a similar situation in most provinces. Some of the differences could be due to the small sample sizes (in fact, I'd argue that the other pollsters are too similar to each other, especially in Ontario) but it gets harder to explain once you realize this is the case in every province. Random variation could cause the CPC to be higher in Ontario (compared to other polls) but you'd expect the CPC to be lower in others.

At the end of the day, it's clear this poll is different and if the election was tomorrow, I wouldn't bet money on Andrew Scheer winning the most seats. Forum seems to be the only firm using IVR to collect the data, this could explain why they seem to often be different from the other pollsters. But as I've said, Forum has been right enough time that they earned the benefit of the doubt. Personally, I think this Forum poll has weird numbers because it's a combination of a decrease in popularity for Trudeau (confirmed by the decrease in the rolling average of Nanos) and some random variations due to sampling (i.e: Forum got the one sample with higher CPC numbers - plus the possible "house effect" of this firm). I guess the truth is most likely that the Liberals are ahead but maybe not by 9 points as other polls have shown.

If we only use these numbers (something not very smart but interesting for the sake of an article), we get the following projections:

Voting intentions; Seat projections with confidence intervals; Chances of winning most seats

I won't analyze province by province. I'd rather use this poll to discuss the path to victory that Scheer has. Ever since he won the leadership contest, his numbers haven't been very good and a fair share of Canadians still don't know him. At the same time, he has kept the base together (the 30% of voters they had in 2015), included in Ontario. The main problem for this party is the incredible lead the Liberals have in Quebec. This is especially problematic as there is no obvious contender who could go and take votes away from Trudeau in the french-speaking province. Andrew Scheer was able to gather the rural voters to defeat Bernier but that won't be enough in a general election. Harper worked super hard in Quebec but saw how difficult it was to get over 20%. The NDP? It depends partially on the leader they'll choose. But none of the candidates appear likely to be able to do better than Mulcair. If anything, staying at the score of 2015 would already be great. The Bloc? We have seen this party bouncing back in the past but I don't see any reason to believe the Bloc will suddenly return to its score above 30% like before (separatist parties in Quebec aren't doing great right now even though the PQ seems to improve in the last month at least). This poll shows, however, that a combination of small increases of the Bloc, NDP and the CPC in Quebec could be enough to cost many seats to the Liberals. They'd still win the most seats but keeping them around 40 MPs makes the task of beating them a lot more feasible.

Scheer has a narrow path to a plurality (most likely a minority). He needs to retake the lead in Ontario (something possible as the Tory vote didn't collapse there in 2015). He could be helped there by the NDP if they were to choose Singh as leader for instance. His charisma and profile could split the left vote, especially in the GTA. Scheer also needs Trudeau to come down in Quebec. He simply can't beat the current PM if Trudeau keeps two regions (the Atlantic and Quebec) with total domination. Scheer can compensate for the Atlantic with Alberta. As for Quebec, we discussed it above.

This leaves BC. Forum has them at 40% over there. Given the recent provincial elections, I can't see the Tories really be that high. The BC Liberals, whose electors are mostly a mix of CPC and LPC at the federal level, got around 40% in May. There as well, Scheer's best shot might be for the NDP to elect a leader capable of splitting the vote.

What I'm trying to say here is that the CPC does have a narrow path to victory in 2019. I'm not in the camp of those who believe Trudeau can't lose even though I'm obviously aware that he's favourite. This poll gave us pretty much the perfect scenario for Andrew Scheer and the Tories. And in this perfect scenario, the CPC would only win 155 seats and have a 29% chance of a majority. Not ideal. On the other hand, take this Forum poll, average it with the numbers of my previous article and you get a race with the Liberals at 178 seats and the CPC at 122. Chances of winning for Trudeau would then "only" be 89% (67% for a majority).

So right now, the scenarios for the Conservatives range from 69% chance of winning the most seats (the best case scenario given by Forum) to 0.1% chances if we exclude this poll. If we average the polls, the chances are roughly at 10%, a number that seems sensible to me. So don't take this single poll are the definite illustration that Trudeau is falling. Take this poll as what a best case scenario would look like for Scheer. And while the numbers look good, it still means that even in the best case scenario, the chances are "only" of 69%. Of course, we are still 2 years away from the next election and the NDP still hasn't elected  anew leader.

Sondage Mainstreet de Septembre: une course à 3

Le sondage du mois d'août de Mainstreet montrait un PQ qui avait rebondi un petit peu. Le sondage de ce mois-ci, publié ce matin, semble confirmer cette tendance. Le sondage a été fait entre les 12 et 14 septembre via des appels téléphoniques non-automatisés. La taille d'échantillon est de 1501.

En même temps, remarquons que le Léger de la fin août (que je n'avais pas couvert car je déménageais, juste pour préciser avant de me faire accuser de ne parler que de certains sondages) montrait le PQ 3e et assez loin derrière la CAQ. Après, vous pouvez vous convaincre que vous aimez une firme plus que l'autre. Perso, je ne fais que publier et analyser les sondages disponibles.

Aussi, le sondage Mainstreet a été fait juste après la convention du Parti Québécois ce qui siginfie que ce parti avait bénéficié d'une couverture médiatique importante. Gardons ainsi cela en tête.

Quoiqu'il en soit, en se basant sur les chiffres du dernier Mainstreet, nous avons une course à 3. Pas dans le sens d'une course avec les trois partis ayant 33% de chances de gagner mais dans le sens que nous avons 3 partis qui, si l'élection était demain, auraient des chances réelles de remporter le plus de sièges. Une petite erreur des sondages et Legault ou Lisée pourraient se retrouver PM.

Les projections sont les suivantes:

Intentions de votes; Projections de sièges avec intervalles de confiance; Chances de remporter le plus de sièges

Tout d'abord, je tiens à adresser le phénomème un peu bizarre d'avoir le PQ avec moins de sièges mais davantage de chances de gagner. Ce n'est pas une erreur ou typo. L'explication est en fait assez simple: le PQ a un potentiel plus élevé que la CAQ. Mais si ces deux partis devaient réellement obtenir 26% des voix, la Coalition pourrait remporter davantage de sièges. C'est particulièrement le cas en raison de l'efficacité du vote CAQ dans les Laurentides (le modèle a cette efficacité en raison des résultats de 2014 mais il n'est pas assuré que ce soit une bonne hypothèse pour 2018). Si l'on tient compte de toute l'incertitude possible (sondages et distribution du vote), le PQ a davantage de chances de remporter le plus de sièges que la CAQ. Il reste que si ce sondage a raison, la course entre ces deux partis serait très serrée. Une autre illustration: le PQ remporterait 10 courses par moins de 5 points alors que la CAQ en gagnerait 14. Ainsi, encore une fois, la CAQ pourrait avoir un vote plus efficace et battre le PQ pour la 2e place, mais ce dernier a un potentiel de croissance réel s'il peut grimper de quelques points (ou que les sondages le sous-estiment de quelques points). En conclusion: un peu bizarre et contre-intuitif à première vue, mais pas une erreur ou une impossibilité.

Regardons les deux distributions de sièges ci-dessous:

On voit bien l'avantage du PLQ qui a aussi une variance plus faible tandis que les distributions du PQ et de la CAQ se chevauchent beaucoup. J'ai mis QS dans un autre graphique, ci-dessous, car sa distribution est juste trop différente. Elle reste cependant impressionnante. Qui aurait pensé qu'à 1 an de la prochaine élection, QS aurait un tel potentiel?

Voici les possibles scénarios avec de tels chiffres. Comme vous pouvez le voir, les chances d'une majorité sont très minces.

Le Québec est un champ de bataille un peu partout, avec vraiment seulement l'Ouest de l'île dominée outrageusement par le PLQ, comme à l'habitude. Le PQ peut cependant compter sur 10 comtés en Abitibi et Lac-Saint-Jean, ce qui compense un peu.

Montréal Est est une lutte à trois entre le PLQ, PQ et surtout QS (qui est en tête dans cette région avec 42% des voix et 7 sièges). La Rive-Nord est une chaude lutte PQ vs CAQ tandis que la Rive-Sud est une lutte à trois. La région de Québec oppose essentiellement le PLQ à la CAQ avec un avantage pour cette dernière.

La Gaspésie reste dominée par le Parti Québécois mais QS son 2e meilleur résultat avec 20%. Si ce parti veut grandir en région, la Gaspésie est probablement une candidate potentielle.

Au final le PLQ resterait favori si l'élection avait lieu demain en raison d'un grand nombres de comtés assurés. Certains me diront que le PQ est donné premier chez les francophones dans ce sondage. C'est vrai. Mais l'avance n'est pas suffisante pour compenser l'avantage des Libéraux avec au moins 15 comtés assurés. Aussi, chez les franco, nous avons les 4 partis dans un espace de 10 points (entre 19% et 29%). Cela veut dire que le Québec franco est super divisé alors que le Québec non-franco est tout acquis au PLQ. Cela signifie aussi que faire des projections pourrait s'avérer difficile avec plusieurs luttes à 3 (un peu ce qui était arrivé au Québec lors de l'élection fédérale de 2015).

Une autre façon de voir la situation est de se dire que le PQ et la CAQ se divise le vote anti-gouvernment et permettraient ainsi à Couillard de possiblement conserver sa job. Remarquez qu'il serait fort intéressant de voir si le PQ et la CAQ (ainsi que QS) seraient tentés de suivre l'exemple de la CB avec une pseudo-coalition pour renverser le gouvernement. On est loin de cette situation mais je me permets de me questionner quant à savoir si Couillard resterait vraiment PM dans le cas d'une si faible minorité.

Il y a 24 comtés où le gagnant y est projeté avec des chances de 100% (et le modèle ne s'est jamais trompé dans ces situations). Parmi ces comtés, le PLQ en a 15! Alors que le PQ n'en a que 4, la CAQ 3 et QS 2. Si vous ne me croyez pas quand je vous dis que le PLQ a un avantage certain, je ne sais plus quoi dire pour vous convaincre. Cela étant dit, la tendance est à la baisse pour le PLQ au cours des derniers mois.

Une petite remarque sur Prévost où Paul St-Pierre Plamondon a annoncé son intention de s'y présenter. Il s'agit d'un comté prenable pour lui et le PQ mais les projections actuelles donnent un petit l'avantage à la CAQ. Cela signifie que le nouveau venu au PQ (enfin, pas si nouveau que ça maintenant) devra batailler pour se faire élire. Remarquez qu'il s'agît d'un nouveau comté qui sera créé l'année prochaine.

Mainstreet continue d'avoir Québec Solidaire exceptionnellement élevé. Après une petite baisse en août, le parti de Manon Massée et GND remonte proche de la barre des 20% (une barre symbolique à défaut d'avoir une significance en termes de sièges pour ce parti. Correction: en fait la barre des 20% est importante pour être reconnu comme groupe parlementaire). Si vous regardez les projections détaillées, vous pouvez voir qu'il y a 18 circonscriptions où les chances de QS sont au-dessus des 1%. Ce que cela veut dire c'est que ce parti n'est plus concentré uniquement à Montréal. Il ne réussit probablement pas à remporter des sièges ailleurs, mais cela ne veut pas dire qu'il n'a pas un rôle à y jouer.

Au final, nous n'avons pas ici des changements majeurs mais comparé à la tendance des 6 premiers mois de 2017, le PQ a remonté et nous avons une course bien plus serrée entre les trois principaux partis.

Aussi, dans Louis-Hébert, le PLQ y est donné largement favori mais le modèle fait sa projections comme s'il s'agissait d'une électon générale.  Je reviendrai sur la partielle du 2 octobre d'ici peu. Avec tous les rebondissements pour le choix des candidats, il se peut que cette partielle soit plus intéressante que prévue.

Les projections détaillées sont ci-dessous: