When you combine bad data with misrepresentation of the sort that Whitehead has been engaging in, it becomes more difficult for our elected leaders to make well-informed decisions.
By Ryan McGreal
Published January 20, 2015
Ward 8 Councillor Terry Whitehead commissioned a city-wide telephone survey last week to gauge public opinion on the controversial transit-only lane on King Street between Mary and Dundurn.
At last Wednesday's General Issues Committee meeting, Whitehead suggested that public opinion was strongly opposed to the bus lane.
However, the poll results themselves, which Whitehead posted on his website on January 16, present a different picture.
Five of the six questions concerned the bus lane (the sixth was about the police budget, for some reason). Here are the questions and result summaries:
Question 1 – How often would you say you take the bus in Hamilton?
- Often – 17.0%
- Occasionally – 17.5%
- Not At All – 65.5%
- Unsure undecided – 3.2%
Question 2 – Would you say you have had a good experience, or a bad experience with the bus lanes?
- Good Experience – 44.0%
- Bad Experience – 56.0%
- Unsure Undecided - 36.4%
Question 3 – Have you found the dedicated bus lanes useful, or do you find them not useful?
- Useful – 39.1%
- Not Useful – 60.9%
- Unsure or Undecided – 24.0%
Question 4 – Are you in favour of the city keeping these bus lanes, or are you not in favour of keeping these bus lanes?
- In Favour – 40.7%
- Not in Favour – 59.3%
- Unsure or Undecided – 17.29%
Question 5 – Are you in favour of suspending and removing the bus lanes to obtain further community engagement on this issue? Or, are you in favour of keeping the bus lanes active while further community engagement takes place on this issue?
- In favour of suspending or removing pending further community engagement – 49.1%
- In favour of keeping the bus lanes active during further community engagement – 50.9%
- Unsure or Undecided – 11.40%
Note the response to Question 5: 50.9 percent of respondents are in favour of keeping the bus lane (it's actually only one lane, not multiple lanes) active during further community engagement, compared to 49.1 percent in favour of suspending the bus lane during further community engagement.
Even though more people supported keeping the lane open than suspending it, Whitehead brought forward a motion to suspend it (the motion failed).
You may notice another thing about the results: the totals add up to a lot more than 100 percent. That is because the percentages for the various options exclude people who were unsure or undecided.
Using Question 5, 11.4 percent of respondents were unsure or undecided; so the 50.9 percent who support keeping it open is actually 45.1 percent and the 49.1 percent who support suspending it is actually 43.5 percent.
Likewise with Question 4, 17.29 percent of respondents were unsure or undecided; so the 40.7 percent in favour of keeping the bus lanes is actually 33.7 percent and the 59.3 percent not in favour of keeping the bus lane is actually 49.1 percent.
Note that the responses to Questions 4 and 5 contradict each other. This is what happens when you are not careful about wording: a "leading question" is a question that frames the issue to suggest a certain response.
In a survey, the use of leading questions introduces a bias for a given response: as these survey results demonstrate, the same group of people can be led to contradict themselves based on what questions they are asked.
And that is not the only methodological problem with the survey. According to Question 1, 63.4 percent (displayed as 65.5 percent) of respondents never take the bus. Yet all respondents were asked whether they found the bus lane useful (Question 3).
29.72 percent (displayed as 39.1 percent) said the bus lane is useful, while 46.28 percent (displayed as 60.9 percent) said it is not useful and 24 percent were unsure or undecided.
In response to Question 2, 28 percent (displayed as 44 percent) of respondents said they had a good experience with the bus lane, while 35.6 percent (displayed as 56 percent) said they had a bad experience and a whopping 36.4 percent were unsure or undecided.
There are other problems with the data as well. The survey was by telephone and only the White Pages was used to generate a random sample of respondents, meaning anyone without a land line was excluded by design.
This means the survey will skew to an older demographic, as older people are more likely to own land lines than younger people.
Since the survey did not include any demographic questions, there is no way to control for this bias.
Bottom line: this survey is a methodological swamp, and even given this fact, the results do not support what Whitehead is saying about them:
over 60% of the people city wide do not support bus lane that number is significantly more. [sic]
Again, Whitehead's own survey results display the percentage of people not in favour of keeping the bus lane as 59.3 percent (which is less than 60 percent, incidentally), but the actual percentage is 49.1 percent once you include respondents who were unsure or undecided.
Public opinion surveys that are this flawed are worse than useless. When you combine bad data with misrepresentation of the sort that Whitehead has been engaging in, it becomes more difficult for our elected leaders to make well-informed decisions.
Please add your voice to the Support Hamilton Transit campaign to keep the bus lane.
You must be logged in to comment.