10/6/13

Internet Surveys: Bad Data, Bad Science and Big Bias


Falacious reasoningBack in 2012, I wrote a blog piece about internet polls and surveys, asking whether internet polls and surveys could be – or should be – considered valid or scientific. I concluded, after researching the question, that, since the vast majority lack any scientific basis and are created by amateurs – often with a goal to direct rather than measure public opinion – that,

Most internet polls are merely for entertainment purposes. They harmlessly allow us to believe we are being engaged and are participating in the process, and they make pollsters happy to receive the attention. They are, however, not appropriate tools for making political or social decisions unless they are backed by rigid, scientific and statistical constraints.*

Earlier that year, an editorial in a local newspaper wisely drew similar conclusions (emphasis added):

It’s said that there’s lies, damn lies, and statistics. You could also throw Internet polls into that mix. …

But anyone who takes the results to heart, or attributes any level of credibility to them, is horribly mistaken. We post those polls to gauge how the community feels about one issue or another, but otherwise there is little to no scientific basis to them.

And unlike a poll that’s usually conducted by the likes of Ipsos, Leger and Gallup – who use scientific principles to conduct many of their public-opinion polls – the results of an Internet poll can easily be ‘plumped’ by one side or the other enlisting family, friends and associates to vote, regardless of their level of understanding of an issue.

I wanted to follow up my earlier piece with some more information from the professionals in the polling and statistical analysis fields, as well as some journalistic comments. The reasons internet polls are unscientific and lack credibility has been addressed by many universities, professional polling companies and associations.

The National Council on Public Polls weighed in on this question, stating (emphasis added):

While different members of NCPP have different opinions about the potential validity and value of online surveys, there is a consensus that many web-based surveys are completely unreliable. Indeed, to describe them as “polls” is to misuse that term.

The NCPP then suggests a list of ten questions for journalists to ask to help clarify the results and the scientific methods used to create and assess the poll:

  1. Is the internet-based survey designed to be representative, and if so, of what population? If not, it is not worthy of being reported.
  2.  What evidence is there that the sample is representative of the population it claims to represent? Unless the internet-based survey can provide clear evidence that the sample is representative by demographic and/or other relevant information it is not worthy of being reported.
  3. How was the sample drawn? Many internet-based surveys are just “call-in” polls or are asked only of people who happen to visit a particular web site. These surveys usually do not represent or make any pretense to represent any other population, and are not worthy of being reported.
  4. What steps does the organization take to prevent people from voting more than once? Any poll which allows people to vote twice, or more often, is not worthy of being reported.
  5. How were the data weighted? Survey data may contain biases from a variety of causes. The magnitude of these biases and random errors are usually unknown to the researcher. Even so, weighting may minimize these biases and errors when there is a strong relationship between the weighting variable and data in the survey. If there is not a strong relationship weighting may make the survey results worse. Demographic weighting of internet-based surveys is essential but is not sufficient. Some firms, in addition to demographic weighting, are weighting on other variables in an attempt to reduce the biases of online data.
  6. What is the evidence that the methodology works and produces accurate data? Unless the organization can provide the results of their other internet-based surveys which are consistent with other data, whether from the Census or other surveys, the survey results are not worthy of being reported.
  7. What is the organization’s experience and track record using internet-based polls? Unless the organization can demonstrate a track record of obtaining reliable data with other online surveys, their online surveys should be treated with great caution.
  8. What is the organization’s experience and track record as a survey researcher using traditional survey methods? If the organization does not have a track record in designing and conducting surveys using the telephone or in-person surveys, it is unlikely that they have the expertise to design and conduct online surveys.
  9. Does the organization follow the codes of conduct of AAPOR, CASRO, and NCPP (whether or not they are members)? If they follow none of these, they are probably not a qualified survey research organization. The more of these Codes they follow, the more likely their data are to be reliable and be trusted.
  10. Is the organization willing to disclose these questions and the methods used (as required by the codes of conduct referred to in #9 above)? If the organization is unwilling to disclose, or unable to provide, the relevant information the survey is probably not worthy of being reported.

The NCPP also has a list of 20 questions journalists should ask about all poll results. They state in the introduction (emphasis added):

The only polls that should be reported are “scientific” polls. A number of the questions here will help you decide whether or not a poll is a “scientific” one worthy of coverage – or an unscientific survey without value. Unscientific pseudo-polls are widespread and sometimes entertaining, but they never provide the kind of information that belongs in a serious report. Examples include 900-number call-in polls, man-on-the-street surveys, many Internet polls, shopping mall polls, and even the classic toilet tissue poll featuring pictures of the candidates on each roll.

One major distinguishing difference between scientific and unscientific polls is who picks the respondents for the survey. In a scientific poll, the pollster identifies and seeks out the people to be interviewed. In an unscientific poll, the respondents usually “volunteer” their opinions, selecting themselves for the poll.

The results of the well-conducted scientific poll provide a reliable guide to the opinions of many people in addition to those interviewed – even the opinions of all Americans. The results of an unscientific poll tell you nothing beyond simply what those respondents say.

Continue reading

10/1/13

The (sometimes violent) urge to write


Scribble, scribbleAs of this writing, I will have published 253 posts since I began this blog at the ending week of December, 2011. Two hundred and fifty three posts in 21 months. Just over one post every two-and-a-half days, on average. Plus 30 or so still in draft mode. Another half-dozen scribbled in word processing notes or notebooks.

And that doesn’t include the six years of blog posts – a list of 91 pages – on my previous blog site (still available in archive format, although some formatting issues have developed after some code updates).

“Scribble, scribble, scribble,” as Prince William Henry, Duke of Gloucester and Edinburgh said to Edward Gibbon. *

Approximately 500,000 words on this blog as published, public material. Uncounted numbers on my other sites, forum and blog, in draft or other formats.

That’s a lot of writing – and it still doesn’t include the writing I have done for my Municipal World articles or books (more than 75,000 words in two published books, one submitted and in editing at 45,000, and the fourth still being written – about 20,000 so far), my Machiavelli book (more than 75,000 words), and a novel I started more than a year ago (approx. 50,000 so far).

Or the writing I’ve done for an upcoming convention talk, my websites, and the innumerable Facebook (and the pages I maintain), Twitter, LinkedIn and forum posts. The ukulele and harmonica reviews, the motorcycle essays, the blog pages, and pre-blog material, the tequila guide. Or numerous emails to staff and fellow councillors, the work I did for a local political party – including crafting their newsletter.

Continue reading

09/28/13

The Unknown Monk Meme


Cisterian monksThis pseudo-poem popped up on Facebook today. It’s been around the Net for a few years, without any source attributed to the quote, but it seems to be making its comeback in the way these falsely-attributed things do:

When I was a young man, I wanted to change the world.
I found it was difficult to change the world, so I tried to change my nation.
When I found I couldn’t change the nation, I began to focus on my town.
I couldn’t change the town and as an older man, I tried to change my family.
Now, as an old man, I realize the only thing I can change is myself,
and suddenly I realize that if long ago I had changed myself,
I could have made an impact on my family.
My family and I could have made an impact on our town.
Their impact could have changed the nation and
I could indeed have changed the world.

It’s recently credited to an “unknown monk” from 1100 CE, and sometimes just to “anonymous.” Since the latter can be anyone, any time, anywhere, it’s less than helpful. Citing the source – at the very least where you found it – is helpful. Anonymous could as easily be one of those crank posters who reply to news stories with snippets about the New World Order or conjure up conspiracies about the local rec facilities.

And the monk from 1100 CE? Not likely. It reads to me like New Age piffle, something regurgitated without understanding.

So let’s look at the attribution. First 1100 CE is in the High Middle Ages. It was shortly after the Norman Conquest of 1066, so if the monk was in England it was a time of chaos, while the Normans dispossessed the English aristocracy (those few left) and took the lands for themselves.

Not as much secular literature survives from that era as religious writing, in large part because the majority of literate people were in the church. Keep in mind that everything was handwritten, mostly on sheepskin: vellum or parchment. Printing was another 450 years away.

The 12th century literature shows nothing like this “poem” anywhere.

Second, a monk would have practiced asceticism, a lifestyle…

…characterized by abstinence from various worldly pleasures, often with the aim of pursuing religious and spiritual goals.

Celibacy was one of those practices. Hence the monk would not likely have had his own family – wife and children. Parents of course, but likely left behind at an early age to be a novice initiate. How much “impact” – a word that didn’t appear in English until 1601, derived from the Latin impactus: to push against (not the same meaning as today’s usage) – a child could have had on his family is unclear, but I’m guessing little.

We of course don’t know if this alleged monk came from a wealthy or poor family. If the latter, their impact on their town – more likely a village  at that time – would likely have been minimal at best, non-existent at worst. Twelfth century village life isn’t what we think of today. There was no central governing body like a municipal council. All land was owned by the lord, and villagers rented from him. Those who were free and not bound to service:

The 12th Century society and village
What defined your status in medieval England was whether you were free or unfree, and how much land you had.
Some rough proportions: About -
15% of people were free
40% of people were Villani (villeins) – they had substantial land (c. 30 acres) but owed service
35% were cottars or bordars – unfree, less land
10% were slaves or as near as darn it
Not all villages were the nucleated village that we think of today – but it’s far and away the most common model. Each village was composed of a number of tofts (or crofts) – areas of 1/4 – 1 Acre, rented from the lord. each croft held the medieval house – typically 24 x 12 feet, 2 rooms, 5+ people and not a lot else.

Continue reading

09/21/13

Chemtrails redux: the attack of the tin-foil hat brigade


Normal clouds mis-identified by wingnutsMy earlier post on the nonsensical chemtrail conspiracy has generated quite a lot of activity recently (more than 1,000 views in a few days – thanks!). So much so that I decided to look online again to see why – had this silliness abated? Were people waking up and laughing at their former craziness? Or was it spreading more among the hard-of-thinking and the anti-science crowd?

Sadly, it seems the latter is the case. And after a bit of research, I became deeply distressed that it seems to be spreading rapidly.

Or maybe the overall number of gullibles is simply growing larger. They band together into cult-like groups that reject outsiders; refuse to allow debate or questions; that turn inwardly and reinforce their own beliefs among one another. Dissonance reduction in numbers.

I found a Facebook group page with more than 11,000 chemtrail-believing members (that’s scary enough right there). Imagine 11,000 people dedicated to this silliest and most risible of all the recent conspiracy theories. But they’re hardly alone.

The conspiracy works like this: every world government, every airline, air force, every pilot, every airline and air force ground and cabin crew, millions of government employees worldwide, engineering firms, chemical manufacturers, scientists, NGOs, meteorologists, NASA, reptilians, and the darkly secretive (and entirely imaginary) “New World Order,” the Illuminati (or the Zionists, Bill Gates or President Obama, since they are implicated – not a little racism runs through these posts) have conspired and are conspiring to secretly spray toxins (or drugs or biochemicals or alien lifeforms) into the atmosphere from stratospheric heights in order to pacify/poison/control (your choice, it seems) the population and/or the weather/crops.

But no matter how you laugh at the gullibility of these folks, no matter how their photographs and wild imagining are easily debunked by science, meteorology, rational thought and common sense, they seem to persist. And grow. (I blame TV, but that’s a digression.)*

More normal sky and cloudsThe conspiracists’ approach to science, natural phenomena, logic and fact is stunningly medieval. Of course, back in the Medieval days the motivating agents were demons, ghosts, imps, sprites and other invisible figments of their imagination. Today, it’s no less superstitious; just the imagined instigators have been given a modern facelift: big government, big pharma and secret societies. They’re still the scary things that go bump in the night, though.

Superstition is still superstition even when wrapped up in technology. The Skeptics’ Dictionary describes superstition as:

…a false belief based on ignorance (e.g., if we don’t beat the drums during an eclipse, the evil demon won’t return the sun to the sky), fear of the unknown (e.g., if we don’t chop up this chicken in just the right way and burn it according to tradition while uttering just the right incantations then the rain won’t come and our crops won’t grow and we’ll starve), trust in magic (e.g., if I put spit or dirt on my beautiful child who has been praised, the effects of the evil eye will be averted), trust in chance (if I open this book randomly and let my finger fall to any word that word will guide my future actions), or some other false conception of causation (e.g.,  homeopathy, therapeutic touch, vitalism, creationism, or that I’ll have good luck if I carry a rabbit’s foot or bad luck if a black cat crosses my path).

The conspiracist view of  government and politics goes beyond superstition, beyond the bizarre and into the pathological.

Some less-than-civil folks online call the chemtrail believers “chemtards.” Others ascribe malicious intent to them:

The Chemtrail looks like a normal contrail in reality. However, there is a conspiracy on the internet that has been passed on for some time and gaining in strength about the Chemtrail. But the truth is that the chemtrails are a hoax and rumour on the internet by people who are looking to create some kind of chaos or just trying to make an impact on others by giving false importance to the chemtrails.

Certainly some chemtrail promotes have engaged in deliberate hoaxes as this news story tells:

The Penticton RCMP is investigating a fraudulent letter that began circulating in the city on Monday.
According to Sgt. Rick Dellebuur, bogus alert notices regarding hazardous chemtrails were put on vehicles at Shoppers Drug Mart.
The letter has city letterhead, but was not issued by the city.
“There is no environmental department in the city and Penticton did not issue this,” he said.
The letter signed by someone named Susan Smith, environmental department manager, states “we are suspecting that unidentified planes are deliberately spraying chemicals over the city of Penticton.”
It further covers how to identify hazardous chemtrails and who to contact if you see them.
Dellebuur said they are investigating to see who is behind this.
“We are following up on leads,” he said. “It’s just one of those things out there in this day and age.”

I have no doubt some of these promoters are the internet versions of televangelists: they prey on the gullible, the hard-of-thinking, the susceptible and the ill-educated, conning them through sales and marketing, through aggressively encouraged “donations.”

One of the most telling indicators of these conspiracy fantasies is that they seem to be held predominantly by those of the libertarian or uber-right-wing political stripe. Blaming Obama for anything spooky, inexplicable, disagreeable or simply misunderstood seems de rigeur among the conspiracists, even if it’s blatantly stupid or illogical to draw even the vaguest of connections between events and the administration. They finally got down to blaming the government for the weather.

Continue reading

08/25/13

Why Quebec’s Proposed Headgear Law is Wrong


The Province of Quebec is proposing to ban the wearing of any and all religious headgear (including hijabs, turbans and yarmulkes, as well as other religious symbols) from public facilities and public service – affecting teachers, hospital workers, daycare workers, nurses, civil servants and (we assume) politicians.

The law would also cover “ostentatious” crucifixes, which has led to darkly humorous speculation about police stopping people to measure the size of their crucifix.

What about a tattoo of a giant “Om” on someone’s back? Would the owner have to remove it? Cover it up?

What about priests’ collars or nuns’ habits, worn in every Catholic school in the province. They’re public sector, aren’t they?

Atheist AAnd what about displays of the Flying Spaghetti Monster? Evolvefish? Or the red atheist “A”? Atheist symbols aren’t religious, so would they be exempt under this new law? *

It would be a bad law, as bad if not worse than the province’s repressive language laws. And it will spark innumerable challenges based on the Charter of Rights’ declaration of religious freedom.**

Not long ago, Quebec’s soccer federation tried to ban the wearing of Sikh turbans on the field. Why? Do turbans proffer a hithertofore unknown benefit to the wearers; performance-enhancing headgear like drugs? Or the opposite? The federation mumbled incoherently about safety, although there has been no turban-related injury ever recorded.

Could it be that turbans have no effect whatsoever on the player or his game and it was just racially-motivated? The general consensus was the latter, and the federation had to backtrack and rescind the ban after it became an international issue. The backlash was embarrassing and awkward, but the PQ government stood by the soccer federation.

While the PQ government – Canada’s answer to the Tea Party, it seems – remains tight-lipped about the upcoming ban on religious displays, the speculation is that Quebec wants to enforce a secular state by legislation. It has announced plans to release a Quebec secular “values” document this fall.

Clearly they haven’t learned from history: Lenin and his successors tried to do that and failed. The more the state interferes with religion, the more it thrives. I suspect that this attempt by the PQ will galvanize the province’s fading Catholicism.  Nothing like a good punch-up to bring out the believers. Britain has its soccer hooligans; Quebec may have its Catholic hooligans if this passes.***

Values cannot be legislated. Instilled, taught, expressed and promoted, but – like taste and talent – not legislated. Lead by example, Marois, not by the iron fist.

Continue reading

06/8/13

Creationism’s stench still lingers in American education


CreationismCreationism (and it’s dressed-up-in-drag younger brother, “intelligent” design) is the black mold of education. It’s an insidious infection of the mind, an intellectual parasite. And like real-life black mold, it creates a toxic environment – for learning and critical thinking.

This week, creationism again came up in American school board discussions. According to the HuffPost, the American Taliban* – the Tea Party – is behind the debate at a Springboro, Ohio, school board, to add the pseudoscience of creationist claptrap to the curriculum. The school board president, Kelly Kohls, is also head of the local Tea Party.

Hardly any surprises there.

It’s a sad, creepy tale. Creationism just won’t get cured. At least not by having such myopic fundamentalists in positions of authority. How do people with closed minds get on school boards in the first place?

Continue reading