Pubdate: Tue, 19 Dec 2017
Source: Truro Daily News (CN NS)
Copyright: 2017 The Daily News
Contact:  http://www.trurodaily.com/
Details: http://www.mapinc.org/media/1159
Author: Russell Wangersky
Page: A4

MAKE NO MISTAKE, NOT ALL POLLS ARE CREATED EQUAL

They're great for entertainment purposes, but then again, so are
fortune-tellers and horoscopes.

If you're looking at public opinion polls to predict things, well,
there are wins and there are losses. And lately, more losses.

We've got to start looking at polling differently - because though
some of the tools get better all the time, the results are not the
gospel we pretend they are.

Two polls in the past week or so highlights that for me: a Toronto
poll suggesting Doug Ford could be competitive in a race against
Toronto Mayor John Tory, and a Corporate Research Associates poll
about marijuana use in the Atlantic provinces.

For the Toronto poll, I'd question why the numbers were even
used.

Certainly, the media shares some of the blame: in a world of
short-staffed newsrooms, the "instant news" of free polling results is
a ready-made page-filler. Heck, the pollsters often essentially write
the story for you. But putting polls up in news stories - even
warning, like the CBC did with the Doug Ford poll, that the numbers
might be off - can skew results. The Ford poll was wonky enough that
the CBC said, "CBC Toronto is not publishing the full polling results
because of a number of concerns raised by its internal research
department, primarily that the poll lacks a randomized sample."

That being said, they published it anyway.

The poll contacted people by robocall - 233,640 people. Only 15,576
Torontonians actually answered the calls and, really, you have to
wonder how representative that subset of people would actually be. (I
think the only empirical evidence you can take from that is people
hate robocalls.)

Then, there's CRA's marijuana poll. Right now, marijuana use is
illegal. I don't know about anyone else, but if someone called me out
of the blue said they were calling from a polling agency, and asked me
about future marijuana use, I'd certainly be more than circumspect in
the way I answered. Numbers for prospective post-legalization
marijuana users across the Atlantic provinces in the poll looked like
this: 15 per cent in P.E.I., 19 per cent in Nova Scotia, 23 per cent
in Newfoundland and Labrador, and 20 per cent in New Brunswick.

I'll be very interested in what the numbers are actually like,
post-legalization.

Pollsters are clearly facing problems reaching a representative sample
of the public: regular telephone numbers are shrinking fast, and
cellphones don't have easy, geographical-searchable listings.
Web-based polling is often a crapshoot - like firing a shotgun into a
fogbank and hoping for ducks. The recent Calgary municipal election
saw polling company Mainstreet forecast that candidate Bill Smith
would win by between nine and 17 percentage points. Smith actually
lost, with Naheed Nenshi taking eight per cent more votes; Mainstreet
had called Smith's election a virtual certainty.

Being off by 23 per cent is a real kick in the teeth.

It's fine to go back after the fact and dissect what went wrong, and
even apologize for it, as Mainstreet did after the Calgary election.
Mainstreet also used robocalls to sample people, but found after the
election that it skewed the sample away from young people, and away
from voters who didn't speak English as a first language. Polling
companies do free polls and release them to the media to promote and
advertise their own business - they do political polls often for the
very people who are running.

If the numbers are wrong, the damage is often already done - people
hear the numbers, and think it's science.

And sometimes, those polls persuade people to put their support behind
a cause that's nothing more than a sampling illusion.

I'd say simply, buyer beware. Nineteen times out of 20.
- ---
MAP posted-by: Matt