Calgary election polling failures caused by 'perfect storm,' Mainstreet research firm says

Mainstreet Research says its internal investigation into the polling failures that led the firm to wrongly predict Calgary Mayor Naheed Nenshi would lose the October municipal election concludes they were caused by a "perfect storm" of factors.

This includes a failure to poll in non-official languages, a tendency among Nenshi voters not to respond to the firm's polls, and most crucially a misgauging of youth voter turnout and their voting intentions, Mainstreet said in a release Monday morning.

• Mainstreet considers legal action against Nenshi pollster

Before the election,MainstreetResearch released three surveys on behalf ofPostmedia, which owns the Calgary Sun and Calgary Herald.

All suggesteda relatively unknown challenger, Bill Smith, was leading Nenshi by between nine and 17 percentage points, with Mainstreet Research presidentQuito Maggi predicting on Oct. 7the "near-certain election of Bill Smith" and adding he likely wouldn't be the only one to lose his seat in "this change election."

Instead, Nenshi won re-election to a third term as mayor in the Oct. 16 election by eight points. All 10 incumbent councillors were also re-elected.

• Read the key findings of Mainstreet'sinternal investigationreport here.

Many long-time political watchers and polling expertsquestioned Mainstreet's resultsduring the campaign, prompting Mainstreet to conduct an internal reviewled by Mainstreet Research VP of analytics Dr. Joseph Angolanoand vetted by a panel of independent polling experts.

It found"absolutely no evidence that the polling errors were intentional or malicious," Mainstreet said in its release.

The report concludesthat severalfactors led to "a near 25 per cent deviation" between Mainstreet's last pre-election poll and the actual outcome.

"The ratio of young people in Calgary, and a few other western urban centres is unlike anything we see in other parts of the country," said Maggi in the release.

"We missed a lot of these young voters, because they are harder to reach. Even those we did reach had significant response bias as revealed by the report."

'Humbling' experience

Maggi said the experience has been a humbling one that will ultimately make the firm better.

The review was also critical of the tone taken by Maggiin his response to critics during the campaign. Maggiaccepted the criticism.

"In the heat of an election race, myself and some of my staff were not as respectful as we should have been," he said.

"We received more than a healthy dose of skepticism and critique. We also faced allegations on social media and elsewhere that our intent was malicious, or that our polls were rigged. We took it personally and responded emotionally."

Maggi vowed to change the way he and his firm communicate during election campaigns, and let the firm's research speak for itself.

Mainstreet conducted Interactive Voice Response (IVR) polling, which is also known as robo-call or push button polling.

Critics of the method say IVR has a low response rate and tends to over-represent older, more conservative-leaning voters.

Angolano's report found that growing youth demographics and declining response rates puts the future utility of IVR in doubt.

"This review has cast a harsh light on some of the shortcomings of IVR (Interactive Voice Response) in polling," the release said.

Next-generation polling

While Mainstreet remains convinced the technique has merit, with some adjustments, the firm is also working on next-generation polling technology, it said.

Maggiapologized to the Calgary Herald, the Calgary Sun and media in Calgary generally.

"You deserve better and I regret any embarrassment it caused the media and Postmedia specifically," he said, vowing that the experience will make him a better, more vigilant and cautious pollster.

To Calgary's mayoral candidates, including the victor, Nenshi, Maggialso offered apologies.

He said he especially regrets any effect his firm's polls had on Andre Chabot's chances, by prompting strategic polarization of support to either Naheed Nenshi or Bill Smith.

The Mainstreet polling data, predicting big lead for Smith, might also have led to complacency among his supporters and negatively affected voter turnout, for which Maggialsosaid he was sorry.

"By setting the expectation of your victory, the true significance of your achievement was lost, the third largest vote count in Calgary mayoral election history, and getting 44 per centof the vote against Naheed Nenshi was unimaginable just a few short weeks before the election," he said.

• Poll puts Bill Smith in the lead over Naheed Nenshi in Calgary's mayoral race
• Bill Smith leads Nenshi by 17 points in latest poll
• Smith leads Nenshi by 13 points in final Mainstreet poll before Calgary election