Bike Helmet Study Confuses Relative Risk

By Nicholas Kevlahan
Published October 16, 2012

Several recent media articles have reported on a new study, based on 2006-2010 data from the office of the Ontario Coroner, finding that cyclists who die of a head injury in a crash or collision were three times less likely to be wearing a helmet than cyclists who died of other injuries. The Globe and Mail reports:

Researchers looked at 129 accidental cycling deaths in Ontario between 2006 and 2010, using data from the Office of the Chief Coroner.

They found cyclists who didn't wear a helmet were three times more likely to die of a fatal head injury than those who wore head protection while riding.

The study, titled Nonuse of bicycle helmets and risk of fatal head injury: a proportional mortality, case-control study [PDF], was published in the October 15, 2012 issue of the Canadian Medical Association Journal.

The study notes that the researchers used an adjusted odds ratio to analyze the correlation between helmet use and reason for death.

An odds ratio is not a measure of relative risk but rather of the strength of the association between two values being measured.

As Wikipedia warns:

Odds ratios have often been confused with relative risk in medical literature. For non-statisticians, the odds ratio is a difficult concept to comprehend, and it gives a more impressive figure for the effect.[13] However, most authors consider that the relative risk is readily understood. In one study, members of a national disease foundation were actually 3.5 times more likely than nonmembers to have heard of a common treatment for that disease - but the odds ratio was 24 and the paper stated that members were 'more than 20-fold more likely to have heard of' the treatment.A study of papers published in two journals reported that 26% of the articles that used an odds ratio interpreted it as a risk ratio.

This may reflect the simple process of uncomprehending authors choosing the most impressive-looking and publishable figure. But its use may in some cases be deliberately deceptive. It has been suggested that the odds ratio should only be presented as a measure of effect size when the risk ratio can not be estimated directly.

In the case of the bicycle helmet study, the reporting makes it sound like a cyclist not wearing a helmet is three times as likely to die as a cyclist wearing a helmet, which is incorrect.

In fact, the authors themselves seem to confuse odds ratio and relative risk, as the title is Nonuse of bicycle helmets and risk of fatal head injury: a proportional mortality, case-control study and the introduction states, "We sought to determine whether cycling without a helmet was associated with an increased risk of sustaining a fatal head injury."

The study results do suggest that bicycle helmets provide some protection in the case of collision with motor vehicles, but the use of odds ratios (standard for this sort of case control study) rather than relative risk may exaggerate the benefit significantly.

The actual data on the effects of mandatory helmet laws do not support the principal authors' claim that their results strongly support such legislation. Specifically, helmet laws tend to deter cycling, and lower rates of cycling are associated with a higher risk of collision for each cyclist.

This is a case where an evidence-based decision requires looking at everything affecting road safety, including avoiding accidents, and international best practice in countries such as the Netherlands and Denmark, neither of which emphasize helmets as the most important aspect of safe cycling.

Nicholas Kevlahan was born and raised in Vancouver, and then spent eight years in England and France before returning to Canada in 1998. He has been a Hamiltonian since then, and is a strong believer in the potential of this city. Although he spends most of his time as a mathematician, he is also a passionate amateur urbanist and a fan of good design. You can often spot him strolling the streets of the downtown, shopping at the Market. Nicholas is the spokesperson for Hamilton Light Rail.


View Comments: Nested | Flat

Read Comments

[ - ]

By Berndt (anonymous) | Posted October 16, 2012 at 14:43:23

I find it's always best to maximize odds of survival. Part of that is taking reasonable steps to reduce risk and devoting more attention to the traffic around me than the minutiae of flawed studies or facile reporting of same.

Permalink | Context

By Statistician (anonymous) | Posted October 18, 2012 at 17:08:10 in reply to Comment 81754

The problem is that flawed studies such as this convince politicians to introduce bicycle helmet laws. This discourages cycling, reduces safety in numbers and increases the risk of some stupid driver hitting you, despite the attention you devote to the traffic around you!

The biggest flaw in this and other studies is that it compares cyclists who die of head injuries with cyclists who die of other injuries.

The real test should be to compare cyclists who die of head injuries with uninjured cyclists - i.e. the cycling population. This provides a more sensible estimate of the risk to the average helmet wearer vs non-wearer.

In Holland, cyclists who wear helmets are much more likely to be injured and end up in hospital that non-wearers -

It's natural to take more risks when you wear a helmet, research also shows overtaking drivers left less room when overtaking a helmeted cyclist (Dr Ian Walker was hit twice while carrying out this research, both times when wearing a helmet). Cyclists also choose to wear helmets for risky situations such as sports cycling or mountain biking.

The much higher injury rate for Dutch cyclists wearing helmets shows they are pretty good judges of the risks.

In countries with helmet laws, the proportion of fatally injured cyclists wearing helmets is almost identical to the population wearing rate, but helmet-wearers are over-represented in the injury statistics. There are two reasons for this: 1) cyclists are more likely to wear helmets for sports and mountain biking where with significant risk of falling off the bike (but low risk of head injury compared to bike/motor vehicle collisions) 2) when cyclists are hit by motor vehicles, non-helmeted are unlikely report the crash because they would be fined, as well as the at-fault motorist.

So it’s a flawed study. As pointed out elsewhere, the odds ratio substantially over-estimates the risk ratio for cyclists who crash or fall off their bikes. And the real risk ratio, comparing cyclists who die of head injury with population helmet-wearing rates suggests the opposite – that cyclists who wear helmets have similar, or higher, fatality rates than non-helmeted cyclists

Permalink | Context

By Brandon (registered) | Posted October 17, 2012 at 13:07:52 in reply to Comment 81754

The best way to maximize your odds of safety on a bike is to have more cyclists on the road. If laws like this reduce the number of cyclists on the road, it is making reducing your survival odds, hence the argument against the law.

Permalink | Context

[ - ]

By John Neary (registered) | Posted October 16, 2012 at 15:16:53

Not wearing a helmet while cycling was associated with an increased risk of dying as a result of sustaining a head injury (adjusted odds ratio [OR] 3.1)

Cyclists less than 18 years of age are required by law to wear a helmet in Ontario. That 88% of decedents in our study were older than 18 years (and 18% were > 60 yr) suggests a gap in public policy.

Although I agree that the authors are missing the big picture on bicycle safety (and that the OR overestimates the RR), it is nevertheless true that their study provides strong evidence that helmets do prevent death from head injuries. (At least within our current transportation system.)

However, I completely disagree with their argument re: public policy.

If we assume that the average person in Ontario drives an automobile for one hour per day, then here's my reinterpretation of their data:

Operating a motor vehicle was associated with an increased risk of cyclists dying as a result of sustaining a head injury (odds ratio 100)

Persons less than 16 years of age are required by law not to operate a motor vehicle in Ontario. That 77% of decedents in our study were killed in collisions with motorists over 16 years of age suggests a gap in public policy.

Now, no one is going to argue based on the results of this study that motor vehicle use should be banned, even though three quarters of cycling deaths occur because of collisions with motor vehicles. So why should helmet use be mandatory?

(Disclaimer: I wear a helmet every time I cycle, and I encourage others to do the same. But I think that helmet laws can do more harm than good by sending a message that cycling is dangerous and by making it impossible to set up bike-share programs.)

[Edited for typos]

Comment edited by John Neary on 2012-10-16 15:17:28

Permalink | Context

[ - ]

By Shawn Selway (anonymous) | Posted October 16, 2012 at 20:56:19

Um, if you thought that it was good to wear a helmet, and helmet laws would discourage people from riding without helmets, wouldn't you logically favour the law?

Permalink | Context

By John Neary (registered) | Posted October 17, 2012 at 11:06:35 in reply to Comment 81774

Yes, if I had no regard for individual rights, or if I were ignorant of the other consequences of helmet laws.

People are allowed to do all sorts of things (smoke, eat terrible food, operate automobiles) that cause far more harm to public health than cycling without a helmet does. I don't propose to ban all of those activities because I don't believe that the state should have that much power over citizens.

And as Ryan has pointed out over and over again, helmet laws have deleterious secondary consequences, such as discouraging people from cycling. I would rather see people cycle with helmets than without, but I would also rather see people cycle without helmets than drive SUVs.

Permalink | Context

[ - ]

By Julie (registered) | Posted October 16, 2012 at 22:11:42

Thank you for your clarification, Nicholas. I read the article in the Spec and was left baffled. I understand my confusion now.

I do wonder whether the authors of the study were confused about the math or being deliberately misleading.

Permalink | Context

[ - ]

By Statistician (anonymous) | Posted October 17, 2012 at 06:40:09

Although the odds ratio is not the same as relative risk, the odds ratio is a good estimate of relative risk under certain circumstances and does not over-accentuate the reported effect. This happens when the incidence rate is small in the unexposed, some say less than 10% is acceptable. In the cycling helmet paradigm relative to this article, this is the percentage of new head trauma related deaths among cyclists who don't wear helmets. That percentage is certainly much less than 10% and thus the study's conclusions stand. Think about it this way, of 100 unhelmeted cyclists will 10 or more die of head trauma? BTW, the Wikipedia entry calls this the rare disease assumption.

Permalink | Context

By kevlahan (registered) | Posted October 17, 2012 at 11:39:16 in reply to Comment 81785

The tricky point is that the control cases are "[cyclist] fatalities for which the cause of death was not a head injury" and the cases are "fatalities included in the coroner’s review for which the cause of death was a head injury" and the population being analysed is "cyclists who died in traffic accidents".

In the controls, 37/58 were not wearing a helmet (64%), while in the cases 58/71 (82%) were not wearing a helmet. It would have been better for the media to have also reported these raw figures.

The (unadjusted) odds ratio is then:

(58/71)/(1-58/71) / ((37/58)/(1-37/58)) = 2.53

Thus, of those cyclists who died, the odds of a non-helmeted cyclists dying of head injuries compared to helmeted cyclists dying of head injuries is 2.5 times higher than the odds of non-helmeted cyclists dying of other injuries compared to helmeted cyclists dying of other injuries (it is not a straightforward statistic!).

The "rare disease assumption" holds (thanks for pointing this out), if we consider the population to be everyone who has cycled in Ontario over the five year period.

But it does not hold if we consider the population to be those who died of injuries in the five year period, since in this case p=58/71, q=37/58, which are not small. Since it is these probabilities that are used to calculate the odds ratio, I would argue that it is misleading to interpret the odds ratio as relative risk for the entire population of cyclists over the five year period.

However, the news reports said that relative risk of non-helmeted cyclists dying of head injuries is 3 times higher, which implies an overall risk for the entire population of cyclists. This is what I meant by not interpreting odds ratio as relative risk (although I wasn't explicit enough).

The other point is that the risk for both groups is relatively low given the total number of cyclists: only 71 deaths from head injuries over 5 years, and only 43 deaths from only head injuries only over 5 years (compared with about 3700 total traffic fatalities over 5 years in Ontario).

The more convincing odds ratio is in fact 3.6 for cyclists who died of head injuries with no other injuries present.

This strongly suggests helmets are beneficial, although the uncertainty is very large due to the small sample size: [1.2, 10.2] 95% confidence interval, which suggests the need for a much larger follow-up study.

There are also questions about competing risk factors: what proportion of cyclists without helmets were riding drunk (this is a major factor in bike accidents)? Are they less law abiding and careful?

We also don't know for sure that helmets did provide protection: this can be assessed by seeing whether the foam actually compressed (rather than the helmet breaking). But this data was not considered.

My main problem is the leap from an odds ratio that shows that a cyclist who dies from a head injury is more likely not to be wearing a helmet to the strong assertion in the article that Ontario should introduce mandatory helmet laws for cyclists.

The international data on cycling accidents, helmet protection and the effect of helmet laws is extremely unclear and the article quotes only a single study from Australia that suggested helmet laws decreased the number of fatalities of all types.

The evidence suggests that there are other much more effective ways of reducing all cycling injuries and deaths (not just head injuries) and that mandatory cycle laws often have negative overall effects.

As this discussion shows, interpreting an odds ratio for a control group study is subtle, and the reporting of the article was confusing.

(Just to be clear: I always wear a helmet myself, but I am not convinced that the balance of benefits and risks is strong enough to force everyone to wear them, and forcing everyone to wear them could actually produce an overall negative outcome.)

Comment edited by kevlahan on 2012-10-17 12:35:10

Permalink | Context

By seancb (registered) - website | Posted October 17, 2012 at 07:48:38 in reply to Comment 81785

Can the actual risk ratio be calculated using the data we have at hand?

How about this: let's calculate the relative risk of a cyclist dying from a head injury while riding on a separated bike lane versus a painted bike lane versus the shoulder of a road versus a rural road with no shoulder.

And then lets implement road design laws which provide the safest environment for cyclists.

I suspect we'll never see this kind of study since it doesn't place quite enough blame on the victim.

Permalink | Context

By jason (registered) | Posted October 17, 2012 at 10:05:41 in reply to Comment 81787


Permalink | Context

[ - ]

By DigitalCyclist (registered) | Posted October 17, 2012 at 08:13:11

It's really good to see a reasoned discussion on this issue, rather than the usual venting or anecdotal "evidence" put forward in blogs (and newspapers!).

The difference between odds ratios and relative risk is quite significant in this particular study.

Here's a different take, looking at relative risk.

According to statistically robust research published by Share the Road in April 2012, 4% of Ontarians cycle daily (let's say 300 rides/year) and another 25% cycle weekly or monthly (let's say 25 rides/year). With Ontario's population of 12.8 million, that gives us approximately 1.17 billion bicycle rides over the 5 year period of the Coroner's study.

In that 5 year period, 77 cyclists died from head injuries sustained in a bicycle crash of some sort. That gives us a probability of a cyclist dying of a head injury on any given ride of approximately 1 in 15,000,000.

Wearing a helmet might (depending on severity of crash) reduce that probability, based on Dr. Persaud's study, to 1 in 45,000,000. (applying his odds ratio)

In comparison, in Canada, about 10 people die from lightning strikes every year. That's a chance of being killed by lightning of 1 in 3,300,000.

So, on any given day when you go for a bike ride without a helmet, your chance of being killed by lightning is about 5 times greater than being killed by a head injury sustained in a bike crash. (inappropriately putting the risk into an odds ratio).

Permalink | Context

By j (registered) | Posted October 17, 2012 at 22:42:14 in reply to Comment 81794

interesting. One medical response might be: if we could guard against lightning strikes we would.

However, seems to me given the nature of the helmet - protecting against slow crashes of the kind where you hit or are hit by a rapidly braking vehicle, or where you fall onto the pavement or into something, it would actually be rarer for a helmet to protect people from dying than it would lessen their severity of a head injury. So if 1000 people hit their heads on pavement falling off their bike going at a very slow speed, I would think a tiny number would die, but a large number would get concussions. That's where I expect the helmet would be effective.

Permalink | Context

By seancb (registered) - website | Posted October 17, 2012 at 08:42:09 in reply to Comment 81794

It's high time we had a law mandating lightning rods on bicycles.

Permalink | Context

[ - ]

By Kiely (registered) | Posted October 17, 2012 at 10:10:19

Given the PR issues the cycling community has with the automobile obsessed majority, rallying against bike helmet laws doesn't seem to be the best use of time… but maybe that's just my way of thinking.

Permalink | Context

[ - ]

By Richard (anonymous) | Posted October 17, 2012 at 14:46:50

None of the people using results like this to recommend helmet laws every consider the unintended negative consequences, which almost always outweigh the benefits of a helmet law significantly. See eg:

Bob Wheeler has estimated the relative risk from the odds ration in one such study:

And not long ago the Transportation Research Lab in the UK also reviewed coroner reports of cylists killed

They found that in only 10% of the deaths, a helmet had a chance of helping. In other words, if everyone wore helmets, about 7 out of every 100 cyclist deaths would be prevented. At the same time, thousands would be deterred from riding and be killed by heart disease. In this case they also had the coroner and accident reports at hand and could have answered that question. Missed opportunity.

Permalink | Context

[ - ]

By Dazed & Confused (anonymous) | Posted October 17, 2012 at 14:59:06

D - get this: I commented on this CBC story this morning. Pointed out a journal article that said car drivers will drive closer to a cylist wearing a visible helmet. That the article did not mention non-head injuries, nor consider how fast a car would hit a bike from behind, exceeding what a helmet is rated for.
They deleted it.
I'm stunned.

I've commented a dozen times on CBC Hamilton since they set up shop.
I will NEVER do so again. I have no time for aspiring censors.

Permalink | Context

By highwater (registered) | Posted October 17, 2012 at 23:50:57 in reply to Comment 81845

I can't believe they would delete a civil comment for no reason. More likely there was some kind of error. Did you include a link to the article? Maybe their system doesn't like links.

Permalink | Context

By D & D (anonymous) | Posted October 17, 2012 at 15:08:04 in reply to Comment 81845

Correction - this link.

Permalink | Context

[ - ]

By Colin (registered) | Posted October 17, 2012 at 15:53:54

Actual data from New South Wales, Australia may also indicate an issue to consider in detail.

Some non-wearers may typically be a higher risk due to other factors. Between 1996 and 2011, of known cases that had been drinking alcohol, 10 were helmeted and 12 without helmet. Nine of the 12 non-helmeted cyclists had a BAC of 0.150 or above and only 1 of the 10 helmeted had this level. Six of the 10 helmeted at low levels of between 0.001-0.019.

I think it is likely that many of the 30 deaths related to alcohol and drugs were not wearing helmets. Also helmet wearers may have been more likely to have had lights, these aspects could also be investigated.

Permalink | Context

[ - ]

By SpaceMonkey (registered) | Posted October 17, 2012 at 20:50:07

Would the ("invisible") helmets in the following link help some you anti-helmeters to change your stance on helmets?

Permalink | Context

[ - ]

By SpaceMonkey (registered) | Posted October 17, 2012 at 21:04:06

What about this? What if there continues to be no law that says one must wear a bike helmet, but if a cyclist suffers a brain injury while cycling without a helmet, then they must pay a fine, or a percentage of the medical bills incurred as a result of the injury? I realize that the person may have suffered from a brain injury even if they had been wearing a helmet. Please don't start with the blame the victim stuff... I cycle over 150km/week, so it's not like I'm anti-cycling or anything.

Permalink | Context

[ - ]

By SpaceMonkey (registered) | Posted October 18, 2012 at 10:12:27

I'm curious about why some people choose not to wear a helmet. To those who do not wear a helmet when riding, could you answer why?

1) Can't afford 2) Too much of a hassle 3) Appearance/Image 4) I'm safer without a helmet 5) I do it to make everyone else safer 6) Other (explain)

Permalink | Context

By moylek (registered) - website | Posted October 18, 2012 at 20:26:02 in reply to Comment 81905

I'm curious about why some people choose not to wear a helmet. To those who do not wear a helmet when riding, could you answer why?

Let us assume that Mr. Monkey is asking in good faith.

I choose not to wear a helmet when riding my bike for the same reason that I choose not to wear a helmet while driving my truck or walking on the sidewalk: the inconvenience is not warranted by the risk.

I might add that I have whacked my head mightily several times - once badly enough that I probably should have gone to hospital to make sure I didn't have a concussion. But that time I was walking in Westdale; the other times, I was driving, sailing or skating.

Permalink | Context

By SpaceMonkey (registered) | Posted October 18, 2012 at 21:14:23 in reply to Comment 81957

Thanks Kenneth,

I appreciate it. Some others might even appreciate the insight too.

Permalink | Context

By seancb (registered) - website | Posted October 18, 2012 at 11:40:06 in reply to Comment 81905

Is your problem one of reading comprehension? There are dozens of articles and hundreds of comments on this site alone that answer option 6 in excruciating detail.

Permalink | Context

By SpaceMonkey (registered) | Posted October 18, 2012 at 15:49:11 in reply to Comment 81927

Really? It seems I'm not the one with reading comprehension problems. I didn't ask about what studies have to say about anything.

Permalink | Context

By highwater (registered) | Posted October 18, 2012 at 10:58:29 in reply to Comment 81905

I'm curious if you're asking this in good faith.

Permalink | Context

By SpaceMonkey (registered) | Posted October 18, 2012 at 15:46:39 in reply to Comment 81912

If being curious is "in good faith", then yes I am.

Permalink | Context

[ - ]

By John Neary (registered) | Posted October 18, 2012 at 11:34:14

It's funny that the same authors, in their Coroner's report, had this to say:

The issue of mandatory helmet legislation for all ages is much more controversial, and was the subject of much debate among the members of the Expert Panel. While Expert Panel members were in agreement about promoting helmet use by all cyclists in Ontario, there was disagreement as to whether mandatory legislation was the best way to achieve this goal. There were three general arguments advanced against mandatory helmet legislation.

The first related to the potential for mandatory helmet legislation to decrease the overall number of cyclists. Proponents of this view cited the experience in Australia, where the introduction of mandatory helmet legislation was associated with a drop in cycling activity. Some research exists which suggest that the health benefits of helmets may be outweighed by the detrimental effects on overall health in the population through the decrease in cycling activity in jurisdictions where helmets have been made mandatory.

The second argument against mandatory helmet legislation relates to the view that government may see mandatory helmet legislation as “the answer” to cycling safety, with the result that other measures recommended in this Review (improved infrastructure, legislative review, education and enforcement activities) are de-emphasized or not acted upon.

The third point raised by members of the Expert Panel is that helmets are, indeed, the last line of defence and of value only after a collision has occurred. Instead of mandating the use of helmets, it was argued that efforts should be focussed on preventing the collision (through strategies such as improved infrastructure and expanded public awareness and education programs) – in other words, if one prevents the collision, helmets become unnecessary. In addition, some stakeholders felt that mandatory helmet legislation sent the message that the responsibility for safety rests with the cyclist alone, rather than being a shared responsibility of all road users.

In other words, in their role as public servants, Cass et al take a nuanced view of the benefits (and potential harms) of helmets, and acknowledge the tradeoffs inherent in legislation. In their role as researchers, they exaggerate the importance of an adjusted odds ratio derived from a study with low-quality methodology and loads of potential biases, and argue that poor evidence should dictate public policy. Given the propensity of researchers of all stripes to exaggerate the importance of their own results, I'm not the least bit surprised.

Permalink | Context

[ - ]

By SpaceMonkey (registered) | Posted October 18, 2012 at 21:17:12

To those who are against cycling helmet laws, what are your views on motorcycle helmet laws? What about scooters? (To avoid any questioning posts, I'm asking in good faith, out of curiosity)

Permalink | Context

By moylek (registered) - website | Posted October 19, 2012 at 07:53:40 in reply to Comment 81959

To those who are against cycling helmet laws, what are your views on motorcycle helmet laws?

Riding a motorcycle seems sort of risky to me, what with the speed and mass involved and the highway traffic. It seems like the sort of thing which warrants a helmet much of the time.

But this brings to mind something else: we keep talking about "cycling" and the risks involved in that activity. But there are - broadly speaking - five different kinds of cycling with very, very different risk profiles:

  • fast, long-distance road biking - think hunched riders all in spandex flying along the shoulders of country roads;
  • mountain biking;
  • recreational biking on trails and such;
  • utility/commuter cycling;
  • "downtown-Hamilton cycling" - typified by a rough sort of fellow, riding on and off sidewalks, an unpredictable anarchist on wheels (apologies to Undustrial), who never signals, ignores traffic signs and might be with or without a case of beer on the handlebars.

And a mental scan of the news articles about accidents and deaths mostly brings up the first and last sorts of cycling, both of which are clearly more dangerous than biking to work from Dundas to downtown, or going for a ride on the rail trail. And when one hears of accidents involving commuters downtown, it seems to be those terrible crushed-under-the-wheels affairs.

All very vague, of course, and subject to qualification by actual data. There is, perhaps, an article here. If not a whole study.

Comment edited by moylek on 2012-10-19 07:58:21

Permalink | Context

[ - ]

By kids bike helmets (anonymous) | Posted November 26, 2012 at 06:38:32

I have been looking for a rare blog because I am tired of accessing almost the same topic discussed in a website. I have read this post and if I could I wish to suggest you some interesting things or advice. Perhaps you could write next articles referring to this article. I want to read more things about it!


kids bike helmets

Permalink | Context

View Comments: Nested | Flat

Post a Comment

You must be logged in to comment.

Events Calendar

There are no upcoming events right now.
Why not post one?

Recent Articles

Article Archives

Blog Archives

Site Tools