10/6/13

Internet Surveys: Bad Data, Bad Science and Big Bias


Falacious reasoningBack in 2012, I wrote a blog piece about internet polls and surveys, asking whether internet polls and surveys could be – or should be – considered valid or scientific. I concluded, after researching the question, that, since the vast majority lack any scientific basis and are created by amateurs – often with a goal to direct rather than measure public opinion – that,

Most internet polls are merely for entertainment purposes. They harmlessly allow us to believe we are being engaged and are participating in the process, and they make pollsters happy to receive the attention. They are, however, not appropriate tools for making political or social decisions unless they are backed by rigid, scientific and statistical constraints.*

Earlier that year, an editorial in a local newspaper wisely drew similar conclusions (emphasis added):

It’s said that there’s lies, damn lies, and statistics. You could also throw Internet polls into that mix. …

But anyone who takes the results to heart, or attributes any level of credibility to them, is horribly mistaken. We post those polls to gauge how the community feels about one issue or another, but otherwise there is little to no scientific basis to them.

And unlike a poll that’s usually conducted by the likes of Ipsos, Leger and Gallup – who use scientific principles to conduct many of their public-opinion polls – the results of an Internet poll can easily be ‘plumped’ by one side or the other enlisting family, friends and associates to vote, regardless of their level of understanding of an issue.

I wanted to follow up my earlier piece with some more information from the professionals in the polling and statistical analysis fields, as well as some journalistic comments. The reasons internet polls are unscientific and lack credibility has been addressed by many universities, professional polling companies and associations.

The National Council on Public Polls weighed in on this question, stating (emphasis added):

While different members of NCPP have different opinions about the potential validity and value of online surveys, there is a consensus that many web-based surveys are completely unreliable. Indeed, to describe them as “polls” is to misuse that term.

The NCPP then suggests a list of ten questions for journalists to ask to help clarify the results and the scientific methods used to create and assess the poll:

  1. Is the internet-based survey designed to be representative, and if so, of what population? If not, it is not worthy of being reported.
  2.  What evidence is there that the sample is representative of the population it claims to represent? Unless the internet-based survey can provide clear evidence that the sample is representative by demographic and/or other relevant information it is not worthy of being reported.
  3. How was the sample drawn? Many internet-based surveys are just “call-in” polls or are asked only of people who happen to visit a particular web site. These surveys usually do not represent or make any pretense to represent any other population, and are not worthy of being reported.
  4. What steps does the organization take to prevent people from voting more than once? Any poll which allows people to vote twice, or more often, is not worthy of being reported.
  5. How were the data weighted? Survey data may contain biases from a variety of causes. The magnitude of these biases and random errors are usually unknown to the researcher. Even so, weighting may minimize these biases and errors when there is a strong relationship between the weighting variable and data in the survey. If there is not a strong relationship weighting may make the survey results worse. Demographic weighting of internet-based surveys is essential but is not sufficient. Some firms, in addition to demographic weighting, are weighting on other variables in an attempt to reduce the biases of online data.
  6. What is the evidence that the methodology works and produces accurate data? Unless the organization can provide the results of their other internet-based surveys which are consistent with other data, whether from the Census or other surveys, the survey results are not worthy of being reported.
  7. What is the organization’s experience and track record using internet-based polls? Unless the organization can demonstrate a track record of obtaining reliable data with other online surveys, their online surveys should be treated with great caution.
  8. What is the organization’s experience and track record as a survey researcher using traditional survey methods? If the organization does not have a track record in designing and conducting surveys using the telephone or in-person surveys, it is unlikely that they have the expertise to design and conduct online surveys.
  9. Does the organization follow the codes of conduct of AAPOR, CASRO, and NCPP (whether or not they are members)? If they follow none of these, they are probably not a qualified survey research organization. The more of these Codes they follow, the more likely their data are to be reliable and be trusted.
  10. Is the organization willing to disclose these questions and the methods used (as required by the codes of conduct referred to in #9 above)? If the organization is unwilling to disclose, or unable to provide, the relevant information the survey is probably not worthy of being reported.

The NCPP also has a list of 20 questions journalists should ask about all poll results. They state in the introduction (emphasis added):

The only polls that should be reported are “scientific” polls. A number of the questions here will help you decide whether or not a poll is a “scientific” one worthy of coverage – or an unscientific survey without value. Unscientific pseudo-polls are widespread and sometimes entertaining, but they never provide the kind of information that belongs in a serious report. Examples include 900-number call-in polls, man-on-the-street surveys, many Internet polls, shopping mall polls, and even the classic toilet tissue poll featuring pictures of the candidates on each roll.

One major distinguishing difference between scientific and unscientific polls is who picks the respondents for the survey. In a scientific poll, the pollster identifies and seeks out the people to be interviewed. In an unscientific poll, the respondents usually “volunteer” their opinions, selecting themselves for the poll.

The results of the well-conducted scientific poll provide a reliable guide to the opinions of many people in addition to those interviewed – even the opinions of all Americans. The results of an unscientific poll tell you nothing beyond simply what those respondents say.

Continue reading

09/8/13

Reflecting on our successes this term


Collingwood CouncilWith just over a year left to go in this term, I’d like to take a few minutes to consider all the accomplishments of this council over the past three years. They are not inconsiderable, and worth celebrating, I think you’ll agree.

Most recent are the two new state-of-the-art recreational facilities; jewels in our community. Centennial Pool Aquatic Centre is now open, to the public’s delight and great excitement. If you haven’t seen it, stop by and look inside. Everyone who does is impressed. It certainly exceeded expectations. Bring your bathing suit and come for a swim.

It proved to be a good choice – and not merely economically – that the whole community can take pride in. It has a full-sized, FINA-endorsed, 25m, six-lane competition pool, with touch pads and starting blocks, as well as a separate warm-water (94F) therapeutic pool residents can use every day (cost to swim is a mere $3; we are looking into options for passes or memberships).

Centennial PoolPlus we can now host those aquatic competitions that are so important to the swimming community. That will prove a benefit to the whole community, economically and in public relations. The Collingwood Clippers held their open house at the new pool, Sept. 5, and presented the town with the anticipated donation cheque for the pool upgrades: $158,000. Thank you, Clippers!

The new (as yet unnamed) arena in Central Park will open this fall. We will have enough ice available to allow groups time that previously we could not accommodate. Women’s and girls’ hockey, and sledge hockey come to mind. Figure skating, too.

It will have a viewing area-mezzanine (with kitchenette for catering), which groups can rent for events – an economic bonus.

Both facilities offer something else, perhaps even more important: safety and time. Local kids and adults can play and practice here in town, and not have to drive to other communities for ice or swim time.

Parents no longer have to get up well before dawn to drive their kids to a distant rink or pool in the early hours of a winter morning, risking whiteouts, rough driving conditions and bad visibility to get their kids to and from a far-away site. Our community’s children and their parents will be much safer, and will have more time to spend together at home or in play, rather than on the road.

That’s worth celebrating.

That we did it all without having to increase taxes is just icing on the cake.

This new arena will allow us to upgrade and enhance the venerable and much-loved Eddie Bush Arena next year. We will be able to use it for other purposes in the non-hockey season: car shows, conventions, entertainment, indoor soccer and lacrosse – the possibilities are great. We will use it to expand our local events and activities, which turn will draw more visitors and improve business. This opens all sorts of possibility for economic development.

Fiscal responsibility has been the watchword for us this term. We have kept the town’s portion of your property taxes from rising for three straight years, while still maintaining services and infrastructure. We have an excellent staff which has been galvanized to hold their budgets by council’s determination and our forward thinking approach to financial responsibility. We plan to continue that trend through 2014.

Continue reading

09/3/13

What is a councillor’s role?


A question was asked of me recently about the appropriateness of the Deputy Mayor being at a meeting last summer to discuss the possible purchase of the new recreational facility structures. From the question I inferred that the asker did not approve of a politician being there.

I disagree, and made my point that it was appropriate. It was hardly a secret meeting – it included numerous staff, plus the acting CAO. And the DM was invited to attend by staff, not the other way around.

First,  the DM is both chair of the budget process and council’s liaison with the Works Department. Who on council would be more appropriate to have at that meeting?

The DM would know and understand the fiscal challenges and opportunities better than any other member because of his history guiding the budget process every year. Plus as Works liaison, he knows what other projects are underway and being contemplated – and how they have to be coordinated with staff in planning and parks, recreation and culture, what services and resources are necessary and available.

The DM alone can’t direct staff – it takes a majority (usually at least at least five members) of council to do that. The proposal as staff determined it had to come to council for approval and confirmation. The DM could hear the arguments pro and con, and raise questions and concerns so that at least the presenters might be able to prepare for possible questions or objections from the table. Staff can make sure these salient points get included in any presentation.

In that way, council might avoid the sort of hour-long round-robin discussion we had about the proposed dog park (much of which seemed to revolve around questions about the choice of base material in the park). That meandering debate ended up going nowhere because staff were unprepared to answer the questions raised at the table.

Having someone to suggest possible objections or questions can streamline the process and make it more efficient. We have no need to return to the often indecisive and divisive five- and six-hour meetings of last term.

Second, as the council member who asked staff to look into the structures, the DM would be the one to present any motion to the table. Isn’t it appropriate that he learn all the sides, all the issues, understand the costs and the complexities, before presenting it to the table? You can’t defend what you don’t understand (or at least you shouldn’t try to). See the notes from the Municipal Act, below.

Third, politicians are elected to lead. Not to rubber stamp staff’s ideas or proposals. We should be meeting with staff, with the private sector, with residents, looking for ideas, opportunities and challenges trying to uncover solutions, partnerships and innovations. We have a larger role outside the table than is seen in our Monday night meetings. Our work at the table is only the tip of the iceberg of work we do. Or should do.

We are not elected to sit at home, in some cocoon, avoiding any contact with the outside world. We need to be active and engaged, if we are to champion or challenge issues. A good politician is one who is actively engaged, not just passive.  We are part of the process, not separate from it.  We are expected to use our own judgment in these situations.

To be able to do our job properly and effectively, we need to get all the input we can garner, to hear people’s ideas and concerns, discuss their projects, discuss the implications with staff. As long as there is no overriding legal issue – such as a potential breach of confidentiality or a liability concern – not to meet with our constituents and with staff is a failure to perform our roles.

Politicians have to get involved, get their feet wet. We can’t sit on the sidelines. But we are not dictators who rule by autocratic decree. We need input from the people who have to implement our decisions in order to accomplish our goals – and the way to get that input is to meet with them.

So, yes, it was appropriate. It’s almost always appropriate that council meet with staff and the public to discuss upcoming issues, motions, initiatives and projects because we were elected to represent the populace and we can’t do it without being engaged.

Continue reading

08/25/13

Why Quebec’s Proposed Headgear Law is Wrong


The Province of Quebec is proposing to ban the wearing of any and all religious headgear (including hijabs, turbans and yarmulkes, as well as other religious symbols) from public facilities and public service – affecting teachers, hospital workers, daycare workers, nurses, civil servants and (we assume) politicians.

The law would also cover “ostentatious” crucifixes, which has led to darkly humorous speculation about police stopping people to measure the size of their crucifix.

What about a tattoo of a giant “Om” on someone’s back? Would the owner have to remove it? Cover it up?

What about priests’ collars or nuns’ habits, worn in every Catholic school in the province. They’re public sector, aren’t they?

Atheist AAnd what about displays of the Flying Spaghetti Monster? Evolvefish? Or the red atheist “A”? Atheist symbols aren’t religious, so would they be exempt under this new law? *

It would be a bad law, as bad if not worse than the province’s repressive language laws. And it will spark innumerable challenges based on the Charter of Rights’ declaration of religious freedom.**

Not long ago, Quebec’s soccer federation tried to ban the wearing of Sikh turbans on the field. Why? Do turbans proffer a hithertofore unknown benefit to the wearers; performance-enhancing headgear like drugs? Or the opposite? The federation mumbled incoherently about safety, although there has been no turban-related injury ever recorded.

Could it be that turbans have no effect whatsoever on the player or his game and it was just racially-motivated? The general consensus was the latter, and the federation had to backtrack and rescind the ban after it became an international issue. The backlash was embarrassing and awkward, but the PQ government stood by the soccer federation.

While the PQ government – Canada’s answer to the Tea Party, it seems – remains tight-lipped about the upcoming ban on religious displays, the speculation is that Quebec wants to enforce a secular state by legislation. It has announced plans to release a Quebec secular “values” document this fall.

Clearly they haven’t learned from history: Lenin and his successors tried to do that and failed. The more the state interferes with religion, the more it thrives. I suspect that this attempt by the PQ will galvanize the province’s fading Catholicism.  Nothing like a good punch-up to bring out the believers. Britain has its soccer hooligans; Quebec may have its Catholic hooligans if this passes.***

Values cannot be legislated. Instilled, taught, expressed and promoted, but – like taste and talent – not legislated. Lead by example, Marois, not by the iron fist.

Continue reading

08/25/13

Creating an Age-Friendly Community


AgingThe most interesting and inspiring seminar I attended during the recent AMO (Association of Municipalities of Ontario) convention was “Synergies for Senior Friendly Communities.” It was about creating “age-friendly communities,” not simply for seniors, but heavily tilted in their direction.*

Speakers included Mario Sergio, Minister responsible for seniors (Ontario Seniors’ Secretariat); Mayor Debbie Amaroso, of Sault Ste. Marie, Mayor Jim Watson, of Ottawa; and Dr. John Lewis, associate professor of planning for the University of Waterloo. I also benefitted by sitting beside Mayor Rick Hamilton, of Elliot Lake, who gave me commentary from his town’s perspective on many of the issues raised, as well as speaking to me afterwards about specific issues I questioned.

The session provided a lot of ideas and processes that I believe we can bring to Collingwood. We do many things right, here, and we have a generally senior-friendly community, but the seminar told me we can – and should – do more. And it talked about the need to formalize our approach, strategize and create a long-range plan. We can’t do this ad hoc.

Provided for all participants were two publications: “Finding the Right Fit: Age-Friendly Community Planning,” a 112-page manual produced by the Province’s Seniors’ Secretariat, and the City of Ottawa’s 40-page “Older Adult Plan, 2012-2014.” Both are invaluable guides for the process. The Secretariat also publishes “A Guide to Programs and Services for Seniors in Ontario.” All of these are available in PDF format, online.

AFC is a designation, not simply a philosophy or policy behind planning and recreational activity programming. You have to apply for the designation, perform several steps, and obtain your certificate from the World Health Organization (WHO). As the WHO site notes,

The WHO Global Network of Age-friendly Cities and Communities (GNAFCC) was established to foster the exchange of experience and mutual learning between cities and communities worldwide. Any city or community that is committed to creating inclusive and accessible urban environments to benefit their ageing populations is welcome to join.

Cities and communities in the Network are of different sizes and are located in different parts of the world. Their efforts to become more age-friendly take place within very diverse cultural and socio-economic contexts. What all members of the Network do have in common is the desire and commitment to create physical and social urban environments that promote healthy and active ageing and a good quality of life for their older residents.

There’s nothing onerous in the process, but it has to be followed closely to avoid being rejected. The WHO recommends a 1-2 year planning process of four steps:

  1. Establishment of mechanisms to involve older people throughout the Age-friendly City cycle.
  2. A baseline assessment of the age-friendliness of the city.
  3. Development of a 3-year city wide plan of action based on assessment findings.
  4. Identification of indicators to monitor progress.

This should be followed by an implementation program in years 3 to 5. WHO notes:

On completion of stage 1, and no later than two years after joining the Network, cities will submit their action plan to WHO for review and endorsement. Upon endorsement by WHO, cities will then have a three-year period of implementation.

Following this, there’s an evaluation process at the end of year 5.

At the end of the first period of implementation, cities will be required to submit a progress report to WHO outlining progress against indicators developed in stage 1.

Nothing we can’t do here. In fact, I think we could become a shining example for AFCs in Southern Ontario.

Continue reading

07/12/13

Casinos redux


Seniors and slotsFirst let’s clarify the terms. A “casino” was never really in the discussion, although just about everyone used that term. What the Ontario Lottery and Gaming Corp. (OLG) offered was a “gaming facility” as they euphemistically called it. A gambling joint, others said.

It was to be a warehouse-like, windowless building with up to 300 slot machines. No keno, no gaming tables for poker or blackjack, no roulette. Up to (and maybe less than) 300 slot machines. No guarantees on the number, just up to 300.

The OLG decides how many: not the town, not the operator. The town can’t even comment on who the operator will be. That decision is in the OLG’s hands.*

Locals also referred to it s a “slot barn,” underscoring its aesthetic deficits. Casino, however, stuck as the word for general palaver.

The OLG made an enthusiastic pitch to every municipality in its artificially-created and somewhat illogically-determined  “zone seven.” Do you want to be a host, they asked, assuming a civic stampede to their door. They held out the promise of money. Who doesn’t want money? It helps grease the wheels of municipal progress. Continue reading

07/9/13

Troilus and Cressida


Telegraph UKI’ve always found Troilus and Cressida a difficult play. The characters all seem jaded, cynical, opportunistic, stuffily sanctimonious, lecherous or simply underhanded. Some are merely unpleasant, others are despicable, reprehensible. All seem self-serving, more concerned with their own gains and goals than that of the greater war around them.

It’s difficult to understand why it was listed as a “comedy” (although oddly placed in the folio, between the histories and tragedies).

Even Harold Bloom calls it “difficult” and “elitist” (although he later qualifies it as Shakespeare’s “most sophisticated” and “intellectualized” play). And Joyce Carol Oates called it, “…that most vexing and ambiguous of Shakespeare’s plays.”

UK reviewer, Charles Spencer, wrote of a 2008 production,

This is surely the bleakest, nastiest and most nihilistic of all Shakespeare’s plays… The whole drama is obsessed with disease and bad faith. The cynicism, and sense of exhausted contempt, undoubtedly makes the play seem extremely modern. It also makes it hard to sit through, and a relief to escape from.

There is no real honour among warriors as we see in Shakespeare’s history plays (Hector’s single moment of gallantry, allowing the exhausted Achilles to live, is marred by his immediate desire to kill a different, unknown Greek solider, solely for his shining armour).

There’s no redemption, either, just self-serving action, personal agendas, vengeance, adolescent lust, scheming, cynicism and politics. There seems no remorse, no guilt, no shame like you might find in Hamlet or Macbeth.* Hector, when he tells Troilus,

There is a law in each well-order’d nation
To curb those raging appetites that are
Most disobedient and refractory. (2.2.180-183)

suggests that laws, not conscience, not guilt, are all that controls humanity. This is the opposite of the message in Shakespeare’s other plays, which almost always explore the effect of shame and conscience on the protagonists and their actions. Continue reading