The 2018 Lancaster Lecture was deferred to May due to the incoming President Jake Olivier being called away to the US in March. Jake spoke on The Importance of Statistics in Shaping Public Policy. Much of Jake’s research, mostly by accident, has ended up having significant impact in the formation of public policy. Four case studies were presented: Queensland’s Graduated Licensing Scheme (GLS); ABC Fact Check; the National Firearms Agreement (NFA); Bicycle Helmet Laws (BHL).
In most cases discussed the statistical object under study could be interpreted as an interrupted time series: an intervention of some kind is assumed to possibly interrupt a time series of measurements on some outcome of interest. Furthermore, data is usually not collected for the purpose of the study being undertaken, rather it is routinely collected for other purposes. This general scenario poses notable challenges: “ideal” data does not exist and there is no obvious “best” analysis.
The Queensland GLS introduced an extra step in the restriction regime for new drivers. Instead of the old system where Learners graduated to “red P’s” for 3 years with only a zero alcohol restriction, under the new system red P’s lasted for (at least) 1 year with some additional restrictions:
- phone restrictions;
- night passenger restrictions;
and then drivers graduate to “green P’s” with these extra restrictions removed for (at least) a further 2 years. Also, extra restrictions were introduced for all classes, including Learners. The purpose of the change was to reduce the risk of accidents among novice drivers.
One analysis focussed on the proportion of crashes involving novice drivers and from this point of view did not indicate that the change had led to any improvement. However, Jake’s more sophisticated analysis using counting process techniques focussing on the number of crashes per 1000 novice drivers did indicate an improvement.
Jake discussed some examples of political fact-checking he undertook for the ABC Fact Check unit: assessing some comments made by Barnaby Joyce (“Our [the Nationals’] constituents are the poorest…”) and Michaelia Cash who made claims about how the introduction and abolition of the Australian Building and Construction Commission (ABCC) impacted days lost due to industrial action in the construction sector. By analysing 3 different datasets (proportion below the poverty line, National Centre for Social and Economic Modelling; Index of Relative Socio-Economic Disadvantage, ABS; Dropping off the Edge report 2015) it was concluded that Barnaby Joyce’s comments “checked out”. Michaela Cash’s comments were more interesting: her statements actually checked out but it was also concluded that “there was more to it” – other factors also affect industrial disputes and so to infer that the ABCC was the sole cause of the changes was deemed problematic.
The NFA and its effectiveness has been continually debated, mostly in the US; at home most people are happy that it was a good idea and more, that it has had a positive impact. In particular, there has been a dramatic decrease in the incidence of “mass shootings” (depending on the definition) since the laws were changed under the NFA in 1996. Anti-gun-control activists, particularly in the US, have been saying the opposite: that mass shootings are “rare events” and as such, it could easily come to pass that (almost) none would occur, even if there was no real change in the underlying process driving mass shootings. Jake made comparisons to an article by the reviewer, whereby he used a geometric change-point model to illustrate that the relative risk changed signficantly after the NFA.
Jake finished with “the most controversial topic” – not firearms legislation, but bicycle helmet laws! He presented the results of a meta-analysis across many different studies, giving some interesting stories about less-than-friendly responses when asking authors for the data used in their study, with a graphic inspired by John Oliver! He also discussed the impact of BHL on various kinds of injury and while some injury types do not show any change before and after the introduciton of BHL, there is a clear reduction in the incidence of serious head injuries. Another question he has examined is “Do BHL deter cycling?”; this is a claim made by opponents to BHL. Again, by considering 23 studies only 2 supported this claim. He finished with an interesting discussion on the use of the ABS Method of Travel to Work (MOTW) data.
Overall it was very interesting to see the pitfalls in dealing with such curly policy questions: there are indeed further barriers trying to prevent the statistician from uncovering the truth – it’s hard enough under ideal conditions! After the talk a lively bunch, including Jake’s family, marched up to Glebe for very satisfying Mexican cuisine.
NSW Branch Vice President