Canada election 2015: Online polling defended, while some still question reliability

Election campaigns are notoriously unpredictable but one thing is certain: Canadians will be bombarded with public opinion polls until the federal vote on Oct. 19.

But how accurate and representative are the data?

• Longer federal election campaign forces pollsters to adjust
• PODCAST: The polling challenges of a long campaign

Statisticians and survey experts say the industry is going through upheaval as an ever-diminishing number of people are willing to participate in polls, and there is no longer a reliable and complete registry of citizens that can be used to randomly select people for research.

Moreover, the number of polling firms has increased significantly over the past several years, creating a "Wild, Wild West" environment for public opinion research due to the various kinds and quality of methodologies used to collect data, according to Andrea Perrella, director of the Laurier Institute for the Study of Public Opinion and Policy.

The polling industry is going through "dark days," he said.

No single methodology

"Fewer people are answering surveys," Perrella explained. "There is no longer a single methodology and even in the most carefully administered telephone polls, response rates are so low we are not too sure what those non-respondents are thinking."

Gone are the days when the vast majority of citizens had landlines and firms had reliable lists of people they could randomly call.

Today, firms need to spend more money to include cellphone users in phone surveys, while other companies build online panels of people who volunteer to be solicited to answer questions online.

Pollsters aware of 'near crisis'

Cliff Zukin, professor of public policy and political science at Rutgers University, penned an op-ed in the New York Times in June where he said "election polling is in near crisis, and we pollsters know."

• Poll Tracker: Follow the latest numbers and seat projections
• Analysis: Parties woo parents in search of 'persuadable' voters
• Analysis: Poll Tracker: Conservative swing voters could drift because of Duffy trial

In an interview, Zukin said online, panel-based polls are problematic because most are not probability-based, meaning people aren't selected randomly because there is no complete list of voters' IP Internet addresses.

"[Online panels]are starting with a badly selected sample," he said. "It's not a sample of anybody, it's people who have volunteered to take polls that's not a good sample of the population."

He said the only way to get a truly representative sample for an online poll is to randomly select voters and give laptops to people who don't have Internet access in order for them to complete the survey which a handful of U.S. firms are doing at a high cost.

Fewer landlines

Moreover, as a result of low response rates and the decrease in the number of landlines, Zukin considers a telephone poll reliable only if 50 per cent of the sample includes people reached on their cellphone.

Sebastien Dallaire of the Leger polling firm disagrees with Zukin on the reliability of online panels.

Leger boasts a panel of 400,000 households from which it draws to conduct its online surveys.

But online databases growing

While Dallaire says "not all panels are created equal," he maintains Leger has created a strong database that reflects the Canadian population.

"Our entire reputation rests on the fact we can produce quality results that are a [near]match with actual voting on voting day," he said.

"If we couldn't do this it would be terrible for us. If we didn't believe in it we wouldn't be using (panels) because our success rests on the trust people have of what we do."

Dallaire wouldn't use the words "dark days" or "crisis" to describe the industry but said it was "going through a transition."

"Response rates have diminished and it poses a problem," he said. "From that point of view it's something that everyone is thinking about."

Too early to get solid results

But aside from the tumult rocking the industry, experts question the usefulness of the high number of polls that have already been released in the early stages of an 11-week election campaign.

Craig Worden of polling firm Pollara Strategic Insights, said surveys focusing on the election horse race between the Liberals, the NDP and the Conservatives this early are "pretty ludicrous."

"At this point the polls [that have come out]are useful as a general snapshot of where people stand at a point in time when they aren't engaged in the campaign and won't be for another month and a half," he said.

Perrella said horse-race polling is problematic for another, more serious reason.

Pick issues over horse-race approach

"The horse-race approach makes voters want to pick a horse and that's not really the intent of our democratic discourse," he said.

He said polls should ask many questions, focus on underlying trends and try to offer voters a detailed and in-depth understanding of how Canadians think about important issues.

Perrella added that horse-race polling turns voters off the idea of coalition governments or a more collaborative approach to politics.

"Coalitions are the backbone of democratic consensus-building but a horse race is not about a consensus," he said.

"It's about a single winner and all the others are losers. That kind of frame limits the ability of people to look at another party as little more than an adversary."