Update for the Vancouver mayoral race: Stewart still ahead, Sylvester on the rise

Advance voting ended tonight for the municipal election in Vancouver. If you haven't voted yet, you'll have to do it this Saturday.

We got two new polls. One from Research Co. and one from One Persuasion. I have never ever heard of this firm. The poll was done online for The Orca. I asked their editor for the pdf of the poll and I got it (thanks!). However, this pdf didn't provide some of the information I was looking for. In particular, it isn't indicated how many respondents were undecided. I sent an email to the firm and I'll keep you updated if they respond. We know the poll had 318 respondents using an online panel. But given that other polls have had between 25 and 40% of undecided (or "don't know" if you prefer), it'd be nice to know how many, out of the 318, were decided voters.

Update: They replied! They said it was 26% undecided. I had assumed them at 25% for this article (kind of at the average of other polls), so we are all good, no need for me to redo the calculations. Still, it means the numbers are ultimately based on around 200 people only...

At least the polls all agree that Kennedy Stewart is ahead. And polls have also agreed that Vancouverites want some chance. Insight West for instance had 69% of people thinking it's time for chance, a proportion not too far from the 81% from One Persuasion. Insight West also showed how many people were undecided and/or were unsure about the candidates with 68% declaring they needed to do more research.

The latest poll from Research Co. shows a progression for Shauna Sylvester and a drop for Hector Bremner. Since One Persuasion had Sylvester 2nd, she seems to be enjoying a little momentum during this last week. Wai Young (former Conservative MP, the only CPC MP in Vancouver in 2011) is shown ahead among people who don't have English as mother tongue and while a victory seems highly unlikely for her, a 4th place finish above 10% is quite possible.

Anyway, here are the latest projections for the mayoral race. If you want more details about the methodology, you can read my previous article here.

Kennedy Stewart remains the big favourite to win this Saturday. Despite few polls (with a ton of undecided), he has been ahead in every single one of them. Also, if you use other measures such as Google Trends or Alexa (to measure the traffic to their website), you also have him ahead.

Nothing is guaranteed as we have fairly limited information but it's pretty clear Stewart is favourite and it'd be a surprise if he didn't win this Saturday. How big of a surprise? Well my model gives him quite good odds but my model also had to "guess" a lot of the uncertainty that exists in this race. I'm simply not as confident about these ranges and probabilities as I'd be for a regular provincial election.

Many variables could change this race, from the number of undecided to the turnout. Let's also remember than Ken Sim can benefit from the electoral machine of the NPA, the party currently polling ahead for the race for the city council. So hopefully I have included enough uncertainty in my simulations!

I'll try to update one last time before Saturday if we get more polls.

A look at the mayoral election in Vancouver

Vancouverites like me will elect their new mayor next Saturday (although advance voting is open from the 10th until the 17th). After 3 terms, Gregor Robertson (Vision Vancouver) decided not to run again (his party is not even running anyone for that matter!), which led to a fairly open race this year.

I haven't made projections for municipal elections in the past, except for one article last year for the Montreal race. It's quite different from regular projections since I don't need to project seats or anything. While it might seem easier (it technically is), there are a number of other differences making mayoral projections more tricky. I get back to them below.

But first, for those of you who don't have the time to read a long text, here are the current projections for the mayoral election of Vancouver. The vote range is the 95% confidence interval.

So, how did I come up with these numbers? I mostly used the few polls we got.

We got only two firms polling this race, Research Co (formerly Insight West) and Mainstreet Research. Although the latter only polled once in early September (back when Vision Vancouver still had a candidate). I asked on Twitter and Mainstreet told me they weren't sure if they'd release another poll. Research Co. will, but they want to release as late as possible.

Not only have we had few polls, municipal polls aren't usually as accurate as provincial ones. That could partially be due to the fairly small sample sizes (around 400 for Research Co., 862 for the Mainstreet one). When I took a look last year at previous elections in Montreal and Calgary, I found that the corresponding effective (i.e: empirical) margins of error were close to 10%! This is much worse accuracy than other Canadian polls. Polls did better for the previous Toronto mayoral election and they were not too bad last year in Montreal. Still, there is no question that I need to include more uncertainty into my simulations.

There is also a very large number of undecided. Provincial or federal elections usually only have around 10% of undecided by the end of the campaign. The Mainstreet poll had over 40% of voters still not sure while the most recent Research Co. has them at 35% (and 3% that wouldn't vote). This alone should convince you of the level of uncertainty that exists. Polls usually allocate these undecided proportionally (which is equivalent to either assuming the undecided will vote as the decided voters or that they won't vote at all; both assumptions being quite unrealistic but that's the typical method used by Canadian pollsters for any election).

So, how will these undecided ultimately vote? Your guess is as good as mine! But there are enough of them to allow Ken Sim to finish ahead of Kennedy Stewart despite being about 10 points behind originally. We could also imagine that Sim, being the official NPA candidate (the main party in Vancouver and favorite according to the Mainstreet poll), could benefit from a bigger "get out the vote" campaign and machine than Stewart who is running as an independent (he's a former NDP MP from Burnaby). Or Shauna Sylvester will benefit from polling better among the 55+ who are more likely to vote.

My job here isn't really to guess which scenario will happen but instead to model this uncertainty. While I hate when people say "don't trust the polls, anything can happen", I have to admit they aren't completely wrong (the most Quebec election, earlier this month, was a good example of how wrong polls can be). So I went with a mix of the polls and various allocations of the undecided, along with the general uncertainty that comes from the fact that polls aren't perfect measures.

So I went with a poll average where half the undecided would break proportionally (in average; During my simulations I varied this ratio from 0 to 100%) and the other half would be uniformly distributed among the candidates. For the simulations, I also used margins of error of 10% to include enough uncertainty (just to put that in perspective, this means having a simulated sample size of around 100 people only!). Side note: the Mainstreet poll included the candidacy of Campbell for Vision Vancouver before he decided to withdraw. I thus adjusted this poll to reflect the fact he's not candidate anymore and his withdrawal is likely going to help Stewart and Sylvester more (If you compare the two Research Co. polls before and after Campbell withdrew, you'll observe these two candidates going up). Doing so made the Mainstreet numbers fairly close to the Research Co. ones, so it seems the two pollsters are agreeing this time and we won't have a repeat of the Calgary race (although they didn't fully agree on where Ken Sim or Hector Bremner were).

As you can see, despite a ton of uncertainty included in my model, Kennedy Stewart is clearly favourite. A 10 points lead isn't completely safe, especially with that many undecided, but it remains a lead that is hard to overcome. Stewart is ahead in every poll over the last two months.

A surprise is possible but this exercise should put things in perspective: it'd take a fairly massive polling failure for Ken Sim to finish ahead. Still, the polls used in these simulations aren't super recent (except for the latest Research Co. which was given a bigger weight), something to keep in mind. I'll update as soon as we get new numbers.

Remember as well that my model doesn't explicitly account for turnout. But the very large margins of error (plus the various allocations of the undecided) should at least give us the range of possible outcomes, if not the exact probabilities.

Google Trends

We can try to use other indicators of the state of the race. One is Google Trends. I have been using it for a few years now and while I do not think it can replace polls (not even close), it does usually provide some information (for instance during the recent Quebec election, it was clearly showing that Quebec Solidaire was popular).

If we look at the last 30 days, we get the following:

Kennedy Stewart is still ahead but it's much closer behind him. Please notice that I have used the "search term" method which is inferior to the topic search one. But the latter is impossible because most of the mayoral candidates aren't even known to Google (if you search for Justin Trudeau for instance, Google Trends will know it's the Prime Minister of Canada).

Is the race closer than what the polls are showing? Maybe. It might be more indicative of the search habits of younger voters however (which would explain why Hector Bremner is higher). Still, given the limited data we have, I found it interesting. At the very least, this gives us a confirmation that Kennedy Stewart is most likely ahead.

Post mortem Québec 2018: une élection historique et un échec majeur des sondages

Voilà, les Québécois ont fait leur choix et ont décidé de donner à la CAQ et François Legault une large majorité. Exactement ce que mes projections avaient prévu. Cependant, les sondages ont passé une très mauvaise soirée. Tout comme le nouveau-venu dans le monde des projections, Qc125. Regardons cela rapidement.

1. Sondages

Le tableau ci-dessous vous montre les erreurs moyennes absolues et combien de partis étaient dans les marges d'erreur des sondeurs. En gros tout le monde a sous-estimé la CAQ et surestimé le PLQ.

Research Co. est en fait le meilleur sondage avec seulement 625 observations! Forum arrive deuxième. Mainstreet fait un peu moins bien en manquant le PQ.

Mention spéciale pour Crop qui avait le PLQ à 36% le 17 septembre lol. J'avais critiqué ce sondage et je maintiens mon opinion. Aussi, assez faible qu'une firme Québécoise ne fasse pas de sondage en fin de campagne. Mais enfin.

Ma propre moyenne bat tous les sondages. À remarquer que Qc125, curieusement, réussit l'exploit de faire une moyenne qui se retrouve pire que tous les sondages! Je crois qu'il a malheureusement écouté les mauvais conseils de Claire Durand qui pense que le PLQ est toujours sous-estimé en raison des "discrets" et qu'il faut ainsi répartir ceux-ci (indécis et refus) avec la méthode 50-25-25, soit 50% au PLQ et 25% chacun au PQ et CAQ. J'ai répété à plusieurs reprises que je trouvais cette hypothèse (car c'est bien une hypothèse) stupide dans un context où la CAQ est en tête et le PQ est à moins de 20% et aussi un vote Libéral qui ne sortait pas selon le vote par anticipation. Jean-Marc Léger sur TVA l'expliquait bien: le vote Libéral est sous-estimé quand il y a une "menace" de la souveraineté. Or ce n'était pas le cas cette fois-ci et les anglo sont restés chez eux. Le 50-25-25 marche la moitié du temps mais pour une raison que j'ignore on considère cette règle comme une règle d'or...

Je suis très fier d'avoir anticipé que le PLQ ferait probablement moins bien que les sondages. je me suis fait attaqué sur Twitter par de pseudo scientifiques mais au final j'avais raison! Pourtant j'avais de bonnes raisons de penser cela.

J'avais évoqué à plusieurs reprises que les sondages par comté de Mainstreet n'étaient pas cohérents avec leur sondages provinciaux (pas mal sûr que je suis le seul site à avoir parlé de cela). Je montrais bien que ces sondages par comté avaient la CAQ bien plus élevée et le PLQ bien plus faible. Lors de mes projections finales, j'avais fait un ajustement partiel pour en tenir compte. C'est ainsi que j'ai été celui qui avait la CAQ le plus élevée. J'aurais dû croire ces sondages encore davantage. En faisant un ajustement complet (-3 pt au PLQ, +1.5 à la CAQ), on aurait obtenu de loin la moyenne la plus proche. Et les différences entre sondages provinciaux et par comté étaient encore plus nettes pour les comtés sondés vers la fin. Le PLQ était carrément 4 points en desous et la CAQ 2.5pts au-dessus. Ainsi ils nous montraient bien le résultat de hier soir. Il fallait simplement avoir le courage de faire confiance à ces sondages au lieu des sondages provinciaux. J'ai coupé la poire en deux personnellement.

C'était déjà le cas en Ontario. À l'avenir je vais ainsi ajuster davantage avec ces sondages Mainstreet.

Est-ce la pire performance des sondeurs? Non! Si on fait l'erreur quadratique moyenne (MSE en anglais), on obtient 13.5. Plus c'est élevé et plus c'est mauvais. En Alberta en 2012, cette MSE avait été de plus de 39! (le Parti Conservateur était carrément sous-estimé de 10 points!). La CB en 2013 était aussi pire. Mais c'est la pire au Québec, bien pire que 2012 (MSE de 8.6). Donc une mauvaise soirée mais pas la pire à vie.

2. Projections

Tel que mentionné plus haut, j'étais le plus proche entre pour la CAQ parmi les trois principaux sites (CBC/308, Qc125 et moi). Mon erreur moyenne en sièges est de 4.5 sièges, comparé à 7.3 pour Qc125 (qui a des fractions de sièges, ce que je trouve un peu bizarre même si je comprends la logique et 4 pour CBC (qui avait QS à 10 sièges, bravo).

J'ai aussi correctement prédis 109 comtés sur 125, soit la même performance qu'en 2014! Amusant.

Bien sûr il nous faudrait comparer avec les vrais pourcentages. Le problème ici c'est que j'utilise en partie les sondages par régions. Ainsi, devrais-je utiliser les vrais % par région? Dur à dire. Si je conserve mes ajustements régionaux mais j'utilise les pourcentages à l'échelle de la province, j'aurais eu 80 CAQ, 34 PLQ,4 PQ et 7 QS, pour une erreur moyenne de 4 (et 16 erreurs au niveau des comtés également). Mais mes ajustements par régions aident probablement trop la CAQ. Si je les retire dans le reste du Québec, j'aurais 78 CAQ, 35 PLQ, 5 PQ et 7 QS.

J'ai manqué 3 sièges QS (Rouyn, Sherbrooke et Jean-Lesage). Dans tous les trois j'avais que QS était proche et avaient des chances. Mais QS est toujours bon pour gagner davantage de sièges que prévu grâce à leur travail de terrain. Je ne m'en veux pas trop de les avoir manqué, c'est la vie.

Je vous remercie de m'avoir suivi, ce fût le meilleur mois à vie sur mon blogue en terme de traffic, meilleur que quand j'avais le soutient du Journal de Montréal! Votre participation dans les commentaires a été très appréciée. Je vais prendre une petite pause là car c'est crevant une élection. Et je reviendrai bientôt pour l'élection fédérale (et le référendum en CB dans quelques semaines en fait). Merci!