Wednesday, May 9, 2012

Evidence based?

In its 25 year history, the Transport Accident Commission has based its public education campaigns on data and evidence.
source: TAC

In 2001, motorcycle fatalities in Victoria were 64, the highest in more than a decade.  In 2002, the TAC produced the Vice Versa campaign, which encouraged drivers and motorcyclists to "put yourself in their shoes."  In that year, fatalities dropped to 56, and to 39 in 2003, and 37 in 2004.

The next four years saw fatalities in the high to mid 40s, then a drop in 2009 (the awful summer of 46.5 degrees and black Saturday).

Late in 2009 TAC released the "reduce the risks" campaign.  The following year, fatalities were up to 49, and there were 49 fatalities again in 2011.  2012 has also had an horrific start in terms of motorcycle fatalities.

I am no statistician, and I shall not fall into the trap of correlation-therefore-causation and draw far reaching conclusions based upon these very broad numbers.

I will however ask one question, in light of the above trends: upon what evidence does the TAC base its approach of shock ads and single sentence sound-bites as the means of addressing motorcycle safety?

If we were to look for a correlation between TAC campaigns and motorcycle fatalities we can only conclude that the "reduce the risks" campaign failed to have a positive impact upon the fatality rate of motorcyclists, whereas in the years following the first release of the "vice versa" campaign, fatalities dropped.

So what evidence is the TAC using to choose its current message and approach?


  1. My biggest problem with this campaign hasn't actually changed - inappropriate speed is most assuredly an issue, but a speed may be inappropriate for the conditions even if it is within the legal speed limit.

    Speeding is something to be deplored, and I appreciate that there is a percentage of riders who do speed, just as there is a percentage of drivers who speed.

    The TAC seems to be suggesting that the major cause of serious injury or death to Motorcyclists is exceeding the speed limit by the motorcyclist, but I'm by no means convinced that this is actually substantated in the raw data.

    The unfortunate result of the approach taken in their most recent ad is that many motorcyclists will in fact reject the message they are trying to push, which is surely counter-productive, even if their underlying premise was correct.

    I'm still waiting for any solid evidence to actually convince me that they haven't just taken the easy line here...

    Speed is certainly *one* factor. But as state earlier, inappropriate speed may still be legal - this ad does not address that core issue. Nor does it address the many skills a rider really does need to keep him or herself safe on the roads.

    I suspect that statistics will point out that this advertising campaign has no positive effect on motorcyclist fatality rates, which is, ostensibly at least, what it is trying to achieve.

    New blood needed at TAC perhaps?

  2. I agree with everything you've said.

    Furthermore, something else that came out in the parliamentary enquiry is the reporting methods employed by Victoria Police.

    In short, when they are reporting on the causes for an accident, the officer has some tick boxes to be checked. One of those tick boxes is "speed." There are no comments, no ability to qualify or expand upon why this box was ticked. It is a binary "was speed a factor in this accident: yes / no."

    Frankly, unless all parties involved in the accident were stationary, it is hard to imagine a scenario when the police would not tick this box. After all, velocity is always a factor when an injury is sustained.

    So it would appear that TAC is spending money telling us to slow down rather than spending money teaching us how to ride more safely ... because the data collection only allows the reporting officer to select 'speed' without explaining or qualifying this selection.

    Note, AFAIK it is "speed", NOT "excessive speed" or "speeding". This has to be the clearest example of Garbage In / Garbage Out I have ever seen.

  3. Transcript relating to speed:

    Mr LANGUILLER — Through the Chair if I may, I am particularly interested in the issue of speed. Can you elaborate on what your data would be on speed?

    Ms REBEIRO — We have the field ‘speed’, but we do not provide too much qualifying data around it. So it is just a tick list, speed, and there is no further qualifying data on that.

    Mr LANGUILLER — What would be the question there for the tick?

    Ms REBEIRO —In the members opinion, ‘What was the cause of the accident?’, they are able to select the field ‘speed’.

    Mr LANGUILLER — So, for example, would that translate into a illegal speed or inappropriate speed?

    Ms REBEIRO — Currently it does not allow us to provide any additional information.

    Mr LANGUILLER — Are you looking at that?

    Ms REBEIRO — Not at the moment.

  4. Figures. Nor does it qualify (I am guessing) which vehicle was speeding where multiple vehicles are involved. It really does make this an almost useless piece of information if it doesn't have the relevant context available.