Polling associations look to increase transparency with merger, but not everyone is on board

Two organizations representing market research firms and pollsters in Canada have announced they are entering into merger talks to speak on behalf of the public opinion polling industry with one voice and apply a uniform set ofstandards for their members.

But at least one prominent polling firm has concerns that some aspects of those standards could undermine the level playing field in the industry.

• Donald Trump and the war on polls
• ricGrenier's Pollcast

Formed in 2004 out of a merger of three industry bodies, the Marketing Research and Intelligence Association representsindividuals andorganizationsacross themarket intelligence and survey research industry, including firms that conduct public opinion surveys.

Last year,a groupof pollsters, researchers and media members who felt MRIA'sdisclosure requirementswere not up to international norms, formed the Canadian Association forPublic Opinion Research to create a code of conduct they felt had higher standards of disclosure.

Earlier this spring, however, MRIA announceda new set of standards that brought the association closer in line with their American and European counterparts. According to MRIA's press release, the "new standards are also very much in the spirit of the standards put in place by CAPOR."

That led to talks over the summer to bring the two organizations together. Once negotiations are concluded, the merger proposal will be put to a vote by members early next year.

Kara Mitchelmore, CEO of MRIA, says a merger will allow the industry to speak with one voice. She says there had been a consensus among its membership for the need forCanadian disclosure standards in public opinion polls the most visible work the industry does.

Darrell Bricker, president of CAPOR and CEO of Ipsos Public Affairs, says that with these new standards itmade sense to bring the two organizations together.

'The more exposure the better'

The new disclosure standards adopted by MRIArequire polling firms to release more information when reporting new polling resultsthan hadpreviously been the case.

Among other things, the standardsrequire polling firms to report, whenever any poll is publicly released, the size of the sample, the field dates of the survey, the method of polling and how any weighting was applied. These are details that polling firms have reported for years.

But additionally, the new standards require the reporting of the number of "call-backs" conducted (when pollsters call numbers again if they get no initial response), the proportion of undecided respondents, the "reasons for any 'rogue' polls or drastically different results between polls" conducted on the same subject, and both weighted and unweighted data tables.

"The more exposure, the better," says Mitchelmore, the idea being that if polling firms are forthcoming with the details of how they get their results, it will help foster public trust in their findings something that has been lacking after a few high-profile polling missesin recent years.

Violations of these standards could result in censure, suspensionor expulsion from MRIA.

There have been criticisms in the past that MRIA has not enforced its standards rigorously enough, but Mitchelmore says the new complaint resolution process will work better, more quickly and treat all parties fairly.

These minimum disclosure standards, says Bricker, make it possibletodetermine the quality of a poll.

With telephoneresponse rates plunging in the industry and pollsters increasingly moving online to reach people, there are concerns thatpollsters are struggling to gatherrepresentative samples. Weighting can correct for this, but there are limits.

The recent case of theUSC/Los Angeles Times poll's extremeover-weighting ofone unrepresentative black supporter of Donald Trump enough to move Trump's national support in polls by a point or two alone is one example of what full disclosure can reveal.

Standards for accuracy, 'not process'

But this push for more transparency, while generally welcomed by most pollsters, has raised concerns in some corners of the industry.

Quito Maggi, president and CEO of Mainstreet Research (political pollster for the Postmedia network of newspapers), feels that these disclosure requirements may force polling companies to reveal proprietary information.

Maggi says that publicizing unweighted data tables would allow other polling firms to reverse-engineer Mainstreet's weighting algorithm, which Mainstreethas put a lot of time and resources into developing. Correctly weighting a sample according towho is likely to vote in an electioncan make the difference between nailing and missing a call.

"Standards should be about accuracy, not process," says Maggi.

Maggisays that his company'sunweighted tables are shown to itsclients and are hosted at the Fields Institute at the University of Toronto (Forum Research, a firm that also uses interactive voice response technology, also houses its data at the university).Butaccording to Maggi,a wider dissemination of Mainstreet'sunweighted tables would lead to a loss of competitive advantage.

Bricker says that kind of disclosure is not good enough, and that if a company like Ipsos one of the largest polling organizationsin the world can adopt these disclosure standards, others can as well.

But according to Maggi, the law is on his side.

"We had previously sought our own legal opinion which affirmed to us that the standards that were being proposed by CAPOR did not comply with Canada's Competition Act and we filed a competition complaint against CAPOR at that time," says Maggi.

Brickersays thatCAPORhas not yet heard from the Competition Tribunal regarding this complaint.

ButMaggi believes "the new MRIA reporting guidelines are similarly non-compliant with Canada's Competition Act as it pertains to industry associations" and is waiting to receive word from the MRIA on how it will address his concerns.

Mitchelmore says MRIA is sensitive to worries about proprietary information, but is moving towarda full disclosure model. For her, the priority is ensuring that widely reported polls do not misleadthe public.

"There's no doubt that polls influence public opinion," says Bricker. "This level of influence places a special responsibility on pollsters to be accountable for their work. Accountability starts with being open and transparent about your methods. That's why there needs to be a set of unified professional standards and disclosure rules for all of Canada's ethical pollsters to follow."