A look at the early turnout for the BC referendum on electoral reform

The referendum on electoral reform is happening right now in BC. According to Elections BC, as of yesterday's morning, 6.5% of the registered voters have sent their ballots back (or, more exactly, Elections BC had received these ballots).

With every single poll showing pretty much a 50-50 race, including my own poll, turnout might be the key to this referendum. Which side will get its vote out the most? If Millennials and people on the Island vote the most, PR will pass. If, on the other hand, the votes come mostly from older people living in the interior, then the current system will prevail.

It's very early and therefore preliminary, but let's look at the numbers and see if we can find some patterns.

1. Turnout predictions

As mentioned above, the current turnout is 6.5% but there is considerable variation across ridings. This is mostly due to the fact that some regions (mostly in the interior of BC) got their envelops earlier. According to the website of Elections BC, there are four groups: 1 that received by October 24th, another by the 25, one by the 26 and, more importantly, a large group of 64 ridings that received their ballots until November 2nd. Among this group, you have variations as well (as indicated by the fact that some of these ridings were below 1% turnout even yesterday) but we don't have the exact date.

For us, it creates really two groups: the 23 ridings (16 in the interior, 4 in the Lower Mainland and 3 on the Island) that received early and the rest. It is important to keep that in mind during our analysis.

Elections BC started reporting on the number of ballots received on November 5th. For the first group of ridings, that means up to 12 days after receiving the envelops while the second group could have had as little as 3 days. We naturally expect the current turnout in the first group to be higher and it is. The average (not weighted, so each riding is equal) turnout is around 10.5% while it's only around 5% for the second group (another way to see the impact of the dates, the first group represents 36% of all ballots received while representing only 26% of the ridings).

This graph below shows you the turnout by date for the two groups:



As you can see the second group is catching up. As a matter of fact, at the same numbers of days since receiving the ballots, the second group is higher. But remember that the second group has more variety within it and not all ridings actually received their ballots as late as November 2nd. Also, comparing the number of days since received isn't truly an apple to apple comparison. According to Google Trends, the interest in this referendum picked up early November and peaked at the "lit" debate on November 7th (see graph below). This means that it's reasonable to assume the second group of ridings would have voted sooner while the first group wanted to wait. They should converge as we approach the deadline.



So, where is this heading? As some people right in fearing a record low turnout? Maybe.

Using this graph from David P. Ball from the Star, we see that the early turnout for this referendum is actually outpacing the previous two referendums (the 2011 one about the HST and the 2015 transit "plebiscite").


(Yes the period to return the ballots was longer for the HST referendum. This is because a postal strike happened in the middle of the campaign and Elections BC decided to extend the deadline. They might do the same this time around depending on what happens with Canada Post)

So, can we expect this referendum to break the 50% barrier? I'm skeptical.

I believe more people were interested in the HST (people actually requested this referendum after being upset at the BC Liberals for breaking their promise not to introduce this tax; Plus, you know, people really, really dislike sale taxes) or the transit plan than electoral reform. The Angus-Reid poll was clearly showing that for many people, this just wasn't a major topic.

Also, if I only use the data from the first group (the one that got their ballots earlier), we see that the best fit for the data seems to be a concave function (like a logarithm). The HST line on the graph above showed that turnout really picked up after around 16 days. If the same was to happen this year, the current data isn't showing this. But again, this is very early.

Let's do the extrapolation with multiple scenarios.

Scenario 1: we extrapolate the current concave function. In this case, we would predict a final turnout of around 20%.

Scenario 2: linear extrapolation. In this case, we end up at around 26%.

Scenario 3: it picks up. In this case, your guess is as good as mine. I can't really use the current data since it doesn't show an steepening of the slope. Based on the graph from the Star, 50% and more isn't impossible. Maybe a better way is to see where the HST turnout was at the same point as when this referendum will end. As you can see, this is just crossing the 30% mark.

Ultimately a turnout between 20 and 30% seems likely. Not only based on these graph but on the fact that during a similar referendum last year, Prince Edward Island got 36% turnout. If you account for the fact that PEI has higher general turnouts than BC, then 20-30% becomes quite good actually.


2. Any pattern?

Again, I repeat myself but this is very early and the data is not super reliable for now (the problem really is the second group of ridings that just started voting). Still, I decided to look at general correlations between the current turnout and some other variables such as the region, the electoral results in 2017 (is the riding a Liberal or NDP one?) as well as the median age and the share of Millennials in the riding (based on the 2011 census, haven't seen the updated version unfortunately).

What did I find? So far, it seems the Interior is voting more (and that's while factoring the fact most of the ridings in the first group were from this region; I used a regression to isolate each effect). For the age and shares of Millennials, they both have positive effects on the turnout! Is it possible that young people are voting (the 18-44, so that includes some Gen Z) and the older people also do? So a Baby-boomers versus Millennials while the Gen X would watch from the sideline. Finally, regarding who the riding voted for in 2017, I find that BC Liberals ridings are currently voting more (again, this is while accounting for when they received their ballots).

All of this is consistent with the Insight West poll showing that among the people who had voted already, FPTP was leading. Yes my regression is supposed to control for the fact some ridings received their ballots sooner, but it might not be doing a perfect job yet since we are too early in the process. Or turnout will simply remain higher among the BC Liberals, older ridings in the interior and proportional representation won't pass.

So right now it doesn't look too good for proportional representation. But we'll. My poll was clearly showing that the undecided were more similar to the pro-PR folks. The key now for Horgan and Weaver is to get these people (mostly Millennials) to actually vote.

As a conclusion, I think the key might be how well the Green party is mobilizing its voters. They are around 18% right now in the polls (so similar to last year). Their partisans are some of the most pro-PR, for obvious reasons. If the overall turnout does remain as low as 20%, then the Green vote could well be the difference. If this party can get its voters to vote, even half of them, that could be enough to reach a majority in favor of PR. The Liberals are definitely trying hard to get their vote out and I'm not sure about the NDP.

New Mainstreet poll confirms the BC referendum on electoral reform to be a 50-50 race

New Mainstreet poll confirms the BC referendum on electoral reform to be a 50-50 race
Just a few days after I published my own poll (conducted using Google Survey) on the referendum on electoral reform in BC, we now have a new Mainstreet poll. And it's showing exactly the same results!

Before talking about this poll, how remarkable is that? A $200 poll conducted using Google Survey can do just as good of a job as a full fledged, regular poll! You might not be excited about it but the political nerd/statistical geek in me is super excited. This is potentially revolutionary. Polling cost have already come down a lot in the last decade thanks to IVR and online sample, but Google Survey is just dirt cheap.

So, anyway, the new Mainstreet poll puts the "keeping the current system" at 50.5% among decided voter while "proportional representation" would receive 49.5%.

My own poll had 50.25% for keeping the current system (Note: yes when I published on Tuesday, "switching to PR" was slightly ahead -at 50.1%- but that's because I wanted to publish even though I still needed around 20 observations to be collected. Google Survey is great but it's crazy slow for the last few observations. It took almost a day and a half to collect these last 20 observations. When it was done, "keeping the current system" was back on top. It's irrelevant ultimately, with margins of error this was and is a 50-50 race).

The debate tonight between Horgan (NDP Premier of BC and pro-PR) and the Liberal leader Andrew Wilkinson (deeply against PR, as is most of the BC Liberals) will be interesting and incredibly important. Proportional representation was ahead in every poll done before this referendum. So Horgan and his partner, Green leader Andrew Weaver, are currently losing this campaign it seems.

Turnout so far has been... abysmal! Only 1.4% of registered voters have returned their mail-in ballot. Of course not everybody got their ballots at the same time and the postal strike doesn't help. Still, I wouldn't be surprised if the overall turnout, at the end, doesn't go above 30%.

It'll ultimately come down to turnout and undecided. Unfortunately Mainstreet doesn't indicate how many people are undecided. I'll ask them. [ Update: I asked and they didn't offer an undecided option at all; This goes a little bit against what I had seen before where the question without undecided would be more pro-PR than the question with the option]. The poll also confirms what my own poll was showing: people under 35 are the biggest supporters of proportional representation.

The Mainstreet poll also confirms what is common sense: if we switch to PR, Multi-Member proportional is the preferred system. Maybe surprisingly, the never used before Dual member proportional is third while the weird urban-rural proportional is second. I would have expected the opposite.

Exclusive poll: The referendum on electoral reform in BC is a toss-up

Full disclosure: I voted to switch to proportional representation. I have never hidden the fact I was pro-PR even if adopting PR pretty much puts me out of a job since projections wouldn't be as important anymore. Just because I have an opinion on this topic doesn't mean I can't be impartial when doing an analysis.

According to a poll conducted exclusively for this blog by Google Survey, British Colombians are almost perfectly divided on the electoral reform referendum. An important share of the population is however undecided and could ultimately make the difference.

Even though we are in the middle of the referendum period (many people, including myself, have already returned their mail-in ballot) and the debate between the NDP-Premier John Horgan and  Liberal leader Andrew Wilkinson is this Thursday, we've had almost no poll. The only one is from over a month ago! So I decided to spend some of that Google ads money my site generated during the Quebec election and to order a poll from Google Survey (I guess Google is winning at the end). I had already tried for the Ontario election with good results (especially in hindsight since most polls underestimated the PC and overestimated the NDP, although my poll was done a little bit far from the election date, so I might have been lucky). I also ordered a "fun" poll asking people what they thought of Maxime Bernier's infamous tweet. Google Survey has also been used in the US with great success for electoral polling.

Google Survey uses the usual Google database (usually used for marketing purposes) to create a representative sample of the Canadian population. They are fairly cheap (20 cents per answer as long as you only ask one question; It increases to $1 and more if you want multiple questions). This poll was done from October 31st to November 6th (so right after the peak in interest for this referendum, thus when people would likely start voting; They have until the end of the month to do so). The sample size is 1200 respondents, although the final number of observations is just over 800 because many respondents didn't have an age and/or gender and were therefore dropped (given a weight of zero). The specific question I asked was "There is currently a referendum on electoral reform in BC. You have or will receive your voting package in the mail. Which following option do you intend to vote for?". I only asked this one question and didn't try to replicate the format of the referendum where a second question exists (you have to choose between 3 potential proportional systems). I did it for monetary reasons and because, quite frankly, the main question is whether BC will switch or not. No actual margins of error since the poll is online but a probabilistic sample of that size would have margins of 3.4% 19 times out of 20.

Here are the results.


First observation: it's close! So, so close. Among decided voters, it's essentially a 50-50 race. And it has been like that all along the period during which the data was collected. "Keeping the current system" was actually ahead for the most part until the last 50 observations. This referendum doesn't have a qualified majority threshold, so 50%+1 would be enough.

Notice however the very high number of people who are undecided. I wish I had the budget to ask them a leaning question but I didn't. I'm not surprised by the high number though. Electoral reform can be seen as a complex issue, or one that doesn't matter. It also seems that online polls, and Google Surveys in particular, have a higher share of undecided. It's interesting to see more people being undecided on this topic than regarding Bernier's tweet!

Let's compare these numbers to the Angus-Reid poll of September. At that time, 33% were undecided, 31% in favour of keeping the current system and 33% wanted to switch. This meant that among decided voters, it was 52-48. My poll numbers match pretty well (not withstanding the higher number of undecided). Can we take it as evidence of a negative trend against proportional representation? Not really since once you apply the margins of error, both polls pretty much match.

With that said, we had seen a trend over the last year according to Angus-Reid. I mentioned it in my post about that poll. It's not clear if Horgan and Weaver are currently losing their referendum or if it's more a question of allowing a "not sure/Don't know" option in the poll. It is possible that pro-PR people are simply less sure of their choices than anti-PR ones. So the 40%+ of undecided in this poll might actually break slightly in favour of switching (note: I expect most of them not to vote at all). The Tyee wrote about comparing different polls with different methodologies already.

Still, I think it's fair to say that this referendum could be very close and I'm not sure the YES campaign is winning so far. Fear mongering arguments (proportional representation will literally allow a nazi party to have seats!...) and others, more valid arguments (the systems offered are half defined, in particular MMP where they haven't even chosen what kind of lists we'd use!) can easily convince some voters against taking the risk to switch. If Horgan is serious in his desire to adopt proportional representation, he'll need to give a convincing performance on Thursday. I'd suggest to maybe emphasize more on the fact that we'd vote again after 2 elections to confirm we want to keep PR. Also, I maintain my position that this referendum would be an easier sell if we could all agree that only MMP is a viable option (Dual member proportional is a system never applied anywhere, invented by a student in mathematics in Alberta and is, ultimately, just a modified version of MMP with more complex features such as vote transfers and forfeited seats; Urban-Rural proportional is too complicated -it mixes STV and MMP- and I can't see it winning). Doing so could clarify the choice for people. Just my 5 cents of course.

Ok, let's look at more numbers since, after all, I paid for this poll and I'm going to milk it. First, by age:



It seems to be Millennials vs Baby boomers (and older Gen X). The general trend isn't surprising but the extend of the difference between age groups might be bigger than expected. Millennials and later Gen X really, really support proportional representation while older Gen X and Baby Boomers, really don't.

The 18-24 is a little bit surprising and might be due to an overall small sample size (or way too many undecided, see below).

Finally, if I look at the age profile of the undecided, I find that they are younger than the population in general. For instance, over 25% of the undecided are between 18-24 while they only represent 20% of the total sample. On the other hand, only 5% of the undecided are aged 65 and over (versus over 8% of the sample coming from this age group).

Here is the percentage of undecided by age.


The fact the undecided are mostly in the age groups that are otherwise favourable to proportional representation is potentially good news for John Horgan and Andrew Weaver. But it also means they need to make sure to convince them to vote ultimately. Turnout will likely be very low (Prince Edward Island got 36% during a similar mail-in referendum last year but this province is used to turnouts much higher than in BC). So every vote could be very important. Now let's wait and see if the debate this Thursday will change this referendum.

It'd also be interesting to reweigh the data based on the age distribution of BC voters as opposed to the census. But that will be for another day for me.

If you'd like to have access to the raw polling data, email me your request along with the reason 9and who you are!).

The YES side ahead in the BC referendum on electoral reform, but it's close

It's time for me to cover my last event in 2018: the referendum on electoral reform in BC. After the Ontario and Quebec elections earlier this year, as well as the Vancouver mayoral election very recently, I hope to continue the perfect streak.

British Colombians like me have started receiving their voting package in the mail. I got mine a couple of days ago. We have until November 30th at 4:30pm to send our ballots back. BC isn't new to this voting by mail referendum thingy. We already voted this way to repeal the HST in 2011 and to refuse a 0.5% sale tax increase to finance public transit in 2015 (although it was then called a plebiscite, but whatever). This is a great way to vote and it cuts cost.

Alright, before writing a wall of text, for you busy people, here are the current chances BC will adopt proportional representation next month:
Based on the latest Angus-Reid poll and 20,000 simulations
The PR side's confidence interval is 46.4 to 56.7% while it's 43.3-53.8% for the FPTP. A close race but PR is currently ahead.

Polls on BC referendums have been pretty good in the past (see below) but we only have one poll. I used the same margins of error as I do for regular Canadian election (so around 5%, far superior to the theoretical margins of the poll itself). I believe this should be enough to capture the uncertainty that exists but I might increase this later. I also allocated the undecided proportionally, which may or may not be a good idea (see below).


Information and analysis

For some reason David Eby, the current NDP cabinet minister in charge of this referendum, has decided to make the question more complicated than it ought to be. Instead of just asking people if they want to switch to the mixed-members proportional system, he decided to first ask if we wanted to have a proportional system or keep the current first-past-the-post. This isn't a bad idea in itself. The problem I see is for the second question asking us to choose between three proportional systems. One is the commonly requested mixed-member one (MMP, i.e: German system but without the complicated overhand seats). We also have the dual-member proportional system (DMP), a system that isn't used anywhere in the world and was invented by a student in mathematics at the university of Calgary. The system itself is fine to me but I don't get the point of including it. MMP is just fine and can be adapted to have "Canadian specificity". Finally, there is the monster of a system than is the urban-rural system (RUP) where urban regions would have the STV (Single Transferable Vote, on which BC voted in 2005 and 2009) while rural areas would have MMP. This is a stupid proposal if you ask me (oh but Bryan, aren't you supposed to be neutral on this site?). It's a wet dream for nerdy political scientists (Fair Vote Canada likes this system...) that nobody outside of some political students will even take the time to research. I totally understand the theoretical arguments in favour of this system, but we live in the real world. If you think the same people who were uninformed and refused the STV in 2009 will all of a sudden learn about two systems and vote yes, you are delusional.

This whole referendum format is weird and overly complicated. And because the government decided to rush a referendum with 3 systems, all of them are half defined. Many "details" are left to be decided AFTER the referendum. And by details I mean "minor" things like the number of MLAs and the boundaries of the electoral map! This is absurd and if this referendum fails next month, Eby and the NDP should be blamed for it. We can already see how this lack of details is helping the NO side.


Lack of polls

We literally only had one poll so far, from Augus-Reid. I know Mainstreet will include 1-2 questions in their next general poll. I fully expect Research Co. to poll as well.

According to this poll, we pretty much have a perfect split between keeping the current system, switching to a proportional one and people that are undecided. However, Angus-Reid has been asking a similar question for a while now. And the answer to this question doesn't match exactly with the most recent one. Let me explain.

AR has included the following question in its BC polls for over a year:

Q13.B.C. could keep the First Past the Post system or adopt a new system that allocates seats roughly in proportion to the total number of votes a party receives.Please indicate which of these two broad options you prefer for B.C.

Only two options were offered: current system or proportional representation. No option to be undecided! At least according to the PDF of their polls. At this question a majority of people from BC usually support PR (around 56-59%). It hasn't moved much.

But in their very last poll, they decided to include questions more specific to the actual referendum. The main difference being that AR allowed respondents to be undecided (as well as not voting, etc).

They also asked a question about which of the three proposed systems is preferred and, no surprise, it's MMP. Honestly I have zero doubt that if the referendum is successful, MMP will be chosen. So I won't focus on this part.

In the same poll with a perfect 3-way splits, AR also has a question about FPTP vs PR in general and guess what? 57% of people in BC support PR!

So either allowing an undecided option affects the YES side more or people like the general idea of PR but don't like the specific project offered to them during this referendum. Or a mix of both. For my simulations, I used the numbers 31%-33%-33% for the FPTP, PR and undecided respectively. In other words, I trusted this poll and not the generic question. Some of you might be surprised that a 2 points lead with simulations using a 5% margins of error would give almost 70% chances of winning, but it is what it is. Obviously it'd be better if we could get more polls confirming the (even small) lead of PR over FPTP, but this isn't a perfect toss up right now. A 2-points lead with 5% confidence intervals can indeed mean the FPTP side is ahead, but it could also mean the PR side is ahead by 7! Uncertainty works both ways. And a 2 points lead with 33% undecided is actually a 4 points lead once we remove/allocate the undecided. So close but not a perfect toss-up, not based on this poll (and the ones before).


Undecided

When I started collecting data to write this article, I thought I'd find that past referendums had seen the undecided mostly breaking for the "safe" or "status quo" option. But actually I haven't.

First of all, polls seem to have been fairly successful with BC referendums, including the ones by mail. Insight West nailed the results of the transit plebiscite. Ipsos was almost perfect for the HST referendum. They also had the right result for the STV referendum in 2009.

It should be noted that mail-in referendums and referendums held during a general elections could be very different. The latter tend to be forgotten during the campaign but then most people still vote since they receive the ballot when they go to their polling station. We have evidence that many people had no idea there was a referendum in 2009 for instance. Mail-in could be different since you only vote on that one issue. I think it's reasonable to assume that the influence of undecided (or unaware) voters is much bigger if the referendum is held at the same as the general election as undecided might simply not send a ballot this year.

So far, I don't have any reason to try to allocated undecided non proportionally (there is a pun somewhere here I'm sure). And the Augus-Reid polls tend to indicate that undecided might actually be more likely to vote for PR than against. It seems proponents of PR are just less sure of their choice and if we allow them to declare themselves as undecided, they will do so.

Now obviously the big question is: will these undecided even vote?


Turnout

Mail-in votes in BC have been quite successful regarding turnout. The HST one got 53% of registered voters sending back a ballot, quite similar to general election (of course, some might argue that it should be higher since you can vote by mail and it's easier). The transit plebiscite got 48%.

At the same time, it is reasonable to assume people cared more about a sale tax and transit than the electoral system. You might feel strongly differently, but it's likely because you are the type of people who read blogs like mine. So keep that in mind. The Angus-Reid poll shows that many don't consider it a priority.

Maybe our best look at turnout is from the mail-in referendum on electoral reform in Prince Edward Island last year. With only 36% of registered voters, this was far below the norm for the small province. If the same pattern were to hold true in BC, we could end up with a turnout below 30% (since BC has a relatively lower turnout in general).

This referendum has also not generated much coverage so far. The YES camp's site is only ranked 26850 on Alexa while the NO site is even lower (72,674). Google Trends shows a rising interest however.



With the municipal election behind us and the ballots being received, this campaign might finally get started. According to Mike Smyth from the Province, the YES side won the first week.

Still, I think a turnout of 40% would already be great.

Maybe the last source of uncertainty is the fact all three major political parties are involved in this campaign, as opposed to 2009 where only the Green even mentioned the referendum. This might help with the turnout. It might simply come down to which party gets its vote out the most. The BC Liberals absolutely want to avoid PR as they think it'd mean no chance to form the government in the future. NDPers and Green voters are in favour.

Update for the Vancouver mayoral race: Stewart still ahead, Sylvester on the rise

Advance voting ended tonight for the municipal election in Vancouver. If you haven't voted yet, you'll have to do it this Saturday.

We got two new polls. One from Research Co. and one from One Persuasion. I have never ever heard of this firm. The poll was done online for The Orca. I asked their editor for the pdf of the poll and I got it (thanks!). However, this pdf didn't provide some of the information I was looking for. In particular, it isn't indicated how many respondents were undecided. I sent an email to the firm and I'll keep you updated if they respond. We know the poll had 318 respondents using an online panel. But given that other polls have had between 25 and 40% of undecided (or "don't know" if you prefer), it'd be nice to know how many, out of the 318, were decided voters.

Update: They replied! They said it was 26% undecided. I had assumed them at 25% for this article (kind of at the average of other polls), so we are all good, no need for me to redo the calculations. Still, it means the numbers are ultimately based on around 200 people only...

At least the polls all agree that Kennedy Stewart is ahead. And polls have also agreed that Vancouverites want some chance. Insight West for instance had 69% of people thinking it's time for chance, a proportion not too far from the 81% from One Persuasion. Insight West also showed how many people were undecided and/or were unsure about the candidates with 68% declaring they needed to do more research.

The latest poll from Research Co. shows a progression for Shauna Sylvester and a drop for Hector Bremner. Since One Persuasion had Sylvester 2nd, she seems to be enjoying a little momentum during this last week. Wai Young (former Conservative MP, the only CPC MP in Vancouver in 2011) is shown ahead among people who don't have English as mother tongue and while a victory seems highly unlikely for her, a 4th place finish above 10% is quite possible.

Anyway, here are the latest projections for the mayoral race. If you want more details about the methodology, you can read my previous article here.


Kennedy Stewart remains the big favourite to win this Saturday. Despite few polls (with a ton of undecided), he has been ahead in every single one of them. Also, if you use other measures such as Google Trends or Alexa (to measure the traffic to their website), you also have him ahead.

Nothing is guaranteed as we have fairly limited information but it's pretty clear Stewart is favourite and it'd be a surprise if he didn't win this Saturday. How big of a surprise? Well my model gives him quite good odds but my model also had to "guess" a lot of the uncertainty that exists in this race. I'm simply not as confident about these ranges and probabilities as I'd be for a regular provincial election.

Many variables could change this race, from the number of undecided to the turnout. Let's also remember than Ken Sim can benefit from the electoral machine of the NPA, the party currently polling ahead for the race for the city council. So hopefully I have included enough uncertainty in my simulations!

I'll try to update one last time before Saturday if we get more polls.

A look at the mayoral election in Vancouver

Vancouverites like me will elect their new mayor next Saturday (although advance voting is open from the 10th until the 17th). After 3 terms, Gregor Robertson (Vision Vancouver) decided not to run again (his party is not even running anyone for that matter!), which led to a fairly open race this year.

I haven't made projections for municipal elections in the past, except for one article last year for the Montreal race. It's quite different from regular projections since I don't need to project seats or anything. While it might seem easier (it technically is), there are a number of other differences making mayoral projections more tricky. I get back to them below.

But first, for those of you who don't have the time to read a long text, here are the current projections for the mayoral election of Vancouver. The vote range is the 95% confidence interval.


So, how did I come up with these numbers? I mostly used the few polls we got.

We got only two firms polling this race, Research Co (formerly Insight West) and Mainstreet Research. Although the latter only polled once in early September (back when Vision Vancouver still had a candidate). I asked on Twitter and Mainstreet told me they weren't sure if they'd release another poll. Research Co. will, but they want to release as late as possible.

Not only have we had few polls, municipal polls aren't usually as accurate as provincial ones. That could partially be due to the fairly small sample sizes (around 400 for Research Co., 862 for the Mainstreet one). When I took a look last year at previous elections in Montreal and Calgary, I found that the corresponding effective (i.e: empirical) margins of error were close to 10%! This is much worse accuracy than other Canadian polls. Polls did better for the previous Toronto mayoral election and they were not too bad last year in Montreal. Still, there is no question that I need to include more uncertainty into my simulations.

There is also a very large number of undecided. Provincial or federal elections usually only have around 10% of undecided by the end of the campaign. The Mainstreet poll had over 40% of voters still not sure while the most recent Research Co. has them at 35% (and 3% that wouldn't vote). This alone should convince you of the level of uncertainty that exists. Polls usually allocate these undecided proportionally (which is equivalent to either assuming the undecided will vote as the decided voters or that they won't vote at all; both assumptions being quite unrealistic but that's the typical method used by Canadian pollsters for any election).

So, how will these undecided ultimately vote? Your guess is as good as mine! But there are enough of them to allow Ken Sim to finish ahead of Kennedy Stewart despite being about 10 points behind originally. We could also imagine that Sim, being the official NPA candidate (the main party in Vancouver and favorite according to the Mainstreet poll), could benefit from a bigger "get out the vote" campaign and machine than Stewart who is running as an independent (he's a former NDP MP from Burnaby). Or Shauna Sylvester will benefit from polling better among the 55+ who are more likely to vote.

My job here isn't really to guess which scenario will happen but instead to model this uncertainty. While I hate when people say "don't trust the polls, anything can happen", I have to admit they aren't completely wrong (the most Quebec election, earlier this month, was a good example of how wrong polls can be). So I went with a mix of the polls and various allocations of the undecided, along with the general uncertainty that comes from the fact that polls aren't perfect measures.

So I went with a poll average where half the undecided would break proportionally (in average; During my simulations I varied this ratio from 0 to 100%) and the other half would be uniformly distributed among the candidates. For the simulations, I also used margins of error of 10% to include enough uncertainty (just to put that in perspective, this means having a simulated sample size of around 100 people only!). Side note: the Mainstreet poll included the candidacy of Campbell for Vision Vancouver before he decided to withdraw. I thus adjusted this poll to reflect the fact he's not candidate anymore and his withdrawal is likely going to help Stewart and Sylvester more (If you compare the two Research Co. polls before and after Campbell withdrew, you'll observe these two candidates going up). Doing so made the Mainstreet numbers fairly close to the Research Co. ones, so it seems the two pollsters are agreeing this time and we won't have a repeat of the Calgary race (although they didn't fully agree on where Ken Sim or Hector Bremner were).

As you can see, despite a ton of uncertainty included in my model, Kennedy Stewart is clearly favourite. A 10 points lead isn't completely safe, especially with that many undecided, but it remains a lead that is hard to overcome. Stewart is ahead in every poll over the last two months.

A surprise is possible but this exercise should put things in perspective: it'd take a fairly massive polling failure for Ken Sim to finish ahead. Still, the polls used in these simulations aren't super recent (except for the latest Research Co. which was given a bigger weight), something to keep in mind. I'll update as soon as we get new numbers.

Remember as well that my model doesn't explicitly account for turnout. But the very large margins of error (plus the various allocations of the undecided) should at least give us the range of possible outcomes, if not the exact probabilities.


Google Trends

We can try to use other indicators of the state of the race. One is Google Trends. I have been using it for a few years now and while I do not think it can replace polls (not even close), it does usually provide some information (for instance during the recent Quebec election, it was clearly showing that Quebec Solidaire was popular).

If we look at the last 30 days, we get the following:



Kennedy Stewart is still ahead but it's much closer behind him. Please notice that I have used the "search term" method which is inferior to the topic search one. But the latter is impossible because most of the mayoral candidates aren't even known to Google (if you search for Justin Trudeau for instance, Google Trends will know it's the Prime Minister of Canada).

Is the race closer than what the polls are showing? Maybe. It might be more indicative of the search habits of younger voters however (which would explain why Hector Bremner is higher). Still, given the limited data we have, I found it interesting. At the very least, this gives us a confirmation that Kennedy Stewart is most likely ahead.

Post mortem Québec 2018: une élection historique et un échec majeur des sondages

Voilà, les Québécois ont fait leur choix et ont décidé de donner à la CAQ et François Legault une large majorité. Exactement ce que mes projections avaient prévu. Cependant, les sondages ont passé une très mauvaise soirée. Tout comme le nouveau-venu dans le monde des projections, Qc125. Regardons cela rapidement.


1. Sondages

Le tableau ci-dessous vous montre les erreurs moyennes absolues et combien de partis étaient dans les marges d'erreur des sondeurs. En gros tout le monde a sous-estimé la CAQ et surestimé le PLQ.



Research Co. est en fait le meilleur sondage avec seulement 625 observations! Forum arrive deuxième. Mainstreet fait un peu moins bien en manquant le PQ.

Mention spéciale pour Crop qui avait le PLQ à 36% le 17 septembre lol. J'avais critiqué ce sondage et je maintiens mon opinion. Aussi, assez faible qu'une firme Québécoise ne fasse pas de sondage en fin de campagne. Mais enfin.

Ma propre moyenne bat tous les sondages. À remarquer que Qc125, curieusement, réussit l'exploit de faire une moyenne qui se retrouve pire que tous les sondages! Je crois qu'il a malheureusement écouté les mauvais conseils de Claire Durand qui pense que le PLQ est toujours sous-estimé en raison des "discrets" et qu'il faut ainsi répartir ceux-ci (indécis et refus) avec la méthode 50-25-25, soit 50% au PLQ et 25% chacun au PQ et CAQ. J'ai répété à plusieurs reprises que je trouvais cette hypothèse (car c'est bien une hypothèse) stupide dans un context où la CAQ est en tête et le PQ est à moins de 20% et aussi un vote Libéral qui ne sortait pas selon le vote par anticipation. Jean-Marc Léger sur TVA l'expliquait bien: le vote Libéral est sous-estimé quand il y a une "menace" de la souveraineté. Or ce n'était pas le cas cette fois-ci et les anglo sont restés chez eux. Le 50-25-25 marche la moitié du temps mais pour une raison que j'ignore on considère cette règle comme une règle d'or...

Je suis très fier d'avoir anticipé que le PLQ ferait probablement moins bien que les sondages. je me suis fait attaqué sur Twitter par de pseudo scientifiques mais au final j'avais raison! Pourtant j'avais de bonnes raisons de penser cela.

J'avais évoqué à plusieurs reprises que les sondages par comté de Mainstreet n'étaient pas cohérents avec leur sondages provinciaux (pas mal sûr que je suis le seul site à avoir parlé de cela). Je montrais bien que ces sondages par comté avaient la CAQ bien plus élevée et le PLQ bien plus faible. Lors de mes projections finales, j'avais fait un ajustement partiel pour en tenir compte. C'est ainsi que j'ai été celui qui avait la CAQ le plus élevée. J'aurais dû croire ces sondages encore davantage. En faisant un ajustement complet (-3 pt au PLQ, +1.5 à la CAQ), on aurait obtenu de loin la moyenne la plus proche. Et les différences entre sondages provinciaux et par comté étaient encore plus nettes pour les comtés sondés vers la fin. Le PLQ était carrément 4 points en desous et la CAQ 2.5pts au-dessus. Ainsi ils nous montraient bien le résultat de hier soir. Il fallait simplement avoir le courage de faire confiance à ces sondages au lieu des sondages provinciaux. J'ai coupé la poire en deux personnellement.

C'était déjà le cas en Ontario. À l'avenir je vais ainsi ajuster davantage avec ces sondages Mainstreet.

Est-ce la pire performance des sondeurs? Non! Si on fait l'erreur quadratique moyenne (MSE en anglais), on obtient 13.5. Plus c'est élevé et plus c'est mauvais. En Alberta en 2012, cette MSE avait été de plus de 39! (le Parti Conservateur était carrément sous-estimé de 10 points!). La CB en 2013 était aussi pire. Mais c'est la pire au Québec, bien pire que 2012 (MSE de 8.6). Donc une mauvaise soirée mais pas la pire à vie.


2. Projections

Tel que mentionné plus haut, j'étais le plus proche entre pour la CAQ parmi les trois principaux sites (CBC/308, Qc125 et moi). Mon erreur moyenne en sièges est de 4.5 sièges, comparé à 7.3 pour Qc125 (qui a des fractions de sièges, ce que je trouve un peu bizarre même si je comprends la logique et 4 pour CBC (qui avait QS à 10 sièges, bravo).

J'ai aussi correctement prédis 109 comtés sur 125, soit la même performance qu'en 2014! Amusant.

Bien sûr il nous faudrait comparer avec les vrais pourcentages. Le problème ici c'est que j'utilise en partie les sondages par régions. Ainsi, devrais-je utiliser les vrais % par région? Dur à dire. Si je conserve mes ajustements régionaux mais j'utilise les pourcentages à l'échelle de la province, j'aurais eu 80 CAQ, 34 PLQ,4 PQ et 7 QS, pour une erreur moyenne de 4 (et 16 erreurs au niveau des comtés également). Mais mes ajustements par régions aident probablement trop la CAQ. Si je les retire dans le reste du Québec, j'aurais 78 CAQ, 35 PLQ, 5 PQ et 7 QS.

J'ai manqué 3 sièges QS (Rouyn, Sherbrooke et Jean-Lesage). Dans tous les trois j'avais que QS était proche et avaient des chances. Mais QS est toujours bon pour gagner davantage de sièges que prévu grâce à leur travail de terrain. Je ne m'en veux pas trop de les avoir manqué, c'est la vie.

Je vous remercie de m'avoir suivi, ce fût le meilleur mois à vie sur mon blogue en terme de traffic, meilleur que quand j'avais le soutient du Journal de Montréal! Votre participation dans les commentaires a été très appréciée. Je vais prendre une petite pause là car c'est crevant une élection. Et je reviendrai bientôt pour l'élection fédérale (et le référendum en CB dans quelques semaines en fait). Merci!