Mainstreet admits 'big polling failures' after predicting Nenshi would lose Calgary election

The polling firm that consistently predicted Bill Smith would be elected mayor of Calgary and challenged third-party observers who raised questions about its results now admits to "big, big polling failures."

"We were the worst by far," MainstreetResearchpresident Quito Maggisaid the day after incumbent Naheed Nenshi defeated Smith by eight percentage points.

• Nenshi wins 3rd term as Calgary's mayor

For weeks prior to Monday's vote, Mainstreetreleased poll after poll suggestingSmith was leading Nenshi by between nine and 17 points.

"These numbers point to the near certain election of Bill Smith on Oct. 16," Maggisaid in an Oct. 7 news release.

"It's likely Nenshi will not be the only incumbent to be the victim of this change election," he said.

In the actual vote, not only did Nenshi win so, too, didall 10 incumbent councillorswho ran for re-election.

'Wonky sample'

Maggibelieves his company's methodology isn't fundamentally flawed, but said it'sinvestigating whether it failed to connect with some younger voters and those who don't speak Englishas their first language.

The company is also looking into the possibility that some people were simply hanging up on the automated calls that Mainstreet relies on to reach voters and gather their opinions.

In particular, Maggiadmitted the second poll his company did, which hadSmith ahead by 17 points, was based on a "wonky sample."

"We knew that it wasn't a great sample, but it's the sample we get," Maggi said.

"Our code says we have to publish polls, even when we think it might be an outlier. And I certainly think that second one was, for sure, an outlier, absolutely."

The demographic breakdown of that poll put Smith ahead by 19 points among female votersand gave Smith a staggering 36-point lead among voters aged 18 to 34.

Both of those demographics are traditionally stronger forNenshi, which promptedseveral political scientists to raise questions about Mainstreet'sresults.

Other polls put Nenshi ahead

Mainstreet did the bulk of the public polling during the election campaign, releasing three polls over a period of three weeks that were paid for by Postmedia, the company that owns the Calgary Sun and Calgary Herald.

But late in the race, a poll commissioned by the LRT on the Green Foundation a non-profit citizensgroup dedicated to light-rail transit expansion in Calgary was released, which hadNenshi in the lead.

Then, in an unusual move, a group of academics opted to release partial results from a larger study they were conducting that included public polling of voter preferences in Calgary.

That polling, conducted by Forum Research, also suggested thatNenshiwas leading Smith.

Both of the non-Mainstreet polls included demographic breakdowns more in line with what political observers expected when it came to young voters and female voters.

"Part of the reason we felt compelled to release some of these results is we understand the power of polls," saidMichael McGregor, a Ryerson University professor and lead researcher inthe academic group that commissioned the Forum poll.

"I'm not going to go out there and say, perhaps like some other people have said, that this is how the election is going to be. I will say that this is, I think, a good indication of how the electorate feels about the candidates."

Mainstreet vs. political scientists

As pundits and political scientists publicly questioned Mainstreet's results during the campaign, Mainstreetdidn't sit back and accept the criticism lightly.

Maggipersonally engaged onTwitter, in particular, with Duane Bratt, a professor at Mount Royal University.

Bratt was especially critical of the demographic breakdown in Mainstreet's numbers, whileMaggiquestioned Bratt's credibility.

Meanwhile, in an interview with 660 Newsthe day before the election, Mainstreetexecutive vice-president David Valentin suggested political commentators in Calgary were biased.

"Certainly I've seen a lot of behaviour from political scientists that I would say is quite shocking in this election campaign, and some of it, quite frankly, is quite appalling," he said.

In that same interview, Valentin said pundits who were critical of Mainstreetcould expect a response after the vote, as the company planned on "singling people out" for "what exactly it is they said and did" about the poll results.

"I think anyone who comments to the media should expect that their comments are going to receive scrutiny after the fact," he said.

"I think that's fair."

Pre-election bravado

Mainstreet also released a "polling scorecard" just before the election, encouraging Calgarians to compare its results and those of the other polling firms to the actual vote totals on Monday.

(Neither Forum Research nor Asking Canadians, the firm that conducted the poll for LRT on the Green, asserted that their polling results were predictive of actual voting outcomes.)

The scorecard was the latest in a series of moves from Mainstreet asserting confidence in the firm's results.

"If polling were poker, this is the part where I would go all in," Maggi tweeted on Oct. 11, the day the LRT on the Green poll was released.

"I would bet $10 million we're closer than that pseudo poll today," he said.

On the Friday before the election, the day the Forum poll was released, Valentin also reiterated his faith in Mainstreet's numbers.

"Some people are going to have a very bad Monday, but not me," he tweeted.

Post-election humility

As the election results rolled in and Nenshi's victory became clear, Maggi said he was incredulous, at first.

"It was just unbelievable to us what the result was," he said.

"I had actually said as much as it would take a miracle for Naheed Nenshi to get re-elected, and then he won by seven per cent."

But as the night wore on, Maggistruck a more conciliatory tone, even agreeing with one online commenter that the result made Mainstreetlook "silly."

"Silly is an understatement," Maggi tweeted in response. "I take full responsibility for this error and launching a full investigation on the polling error."

He spent the next couple of days defending Mainstreet from legions of online critics.

"Admittedly, the industry andmyself could use a touch of humility," he tweeted on Wednesday.

"Will do my part."

Edmonton mayor pans automated polls

Meanwhile, Edmonton Mayor Don Ivesonsaid automated phonepolls, in general, are unreliable and harmful to the public discourse, in his view.

"We should stop reporting things that are off by 10 or 15 per cent and distort or oversample or undersample certain groups. But that's just me," he told reporters.

"And I know some of your outlets have decided to do that, but I'm just going to say as a free-time, political commentary that those polls are just not useful. They distort the public discourse," Ivesonsaid.

"They do a disservice to citizens, quite frankly, because it's not what a credible poll with good methodology shows."

• Polling associations look to increase transparency with merger, but not everyone is on board
• Canada election 2015: Online polling defended, while some still question reliability