SSA Statement on the Australian Marriage Law Postal Survey released on 27 September 2017
The Statistical Society of Australia (SSA) is concerned that the Australian Marriage Law Postal Survey will not definitively reflect the views of the majority of Australians, and that subsequent Government decisions could perhaps be more usefully based on more representative information. The SSA is also concerned that the reputation of the Australian Bureau of Statistics (ABS) and the statistical profession may be negatively impacted by misrepresentation of the Survey’s results by uninformed community groups.
From a statistical point of view the Postal Survey is not ideal, but in order to get the most representative data possible, the SSA encourages all Australians to have their say and record their preference before 27th October 2017.
Will the Postal Survey represent the views of all Australians?
The answer to this depends on the question being asked.
The Government direction to the Australian Statistician and the ABS on the implementation of the Postal Survey, asks that the ABS give all eligible Australians the opportunity to express their support for or against the law being changed to allow same‑sex couples to marry. This information, broken down to the level of electoral divisions, will then be used to inform the Government’s decision to table a subsequent bill in Parliament. Because this question is being asked of all eligible Australians, in this sense, the results of the Postal Survey will be representative of their views.
However, as by design it is not compulsory to vote (see the focus on “participating electors” in Sections 3(1)(a)-(c)), this means that the Postal Survey outcomes – that is, the proportion of participating electors “in favour” or “against” – will only be representative of the views of those who respond, rather than representative of the views of all Australians. To make this point explicit, the ABS will publish the Survey results on 15th November 2017, along with a statement on their quality and integrity by the Australian Statistician. This will include the proviso that the results only apply to the Survey respondents and can not be generalised to the Australian population as a whole.
So from this perspective, the results of the Postal Survey, will not represent the views of all Australians.
Marriage equality is a sensitive and emotive issue. The SSA is concerned that, as a result, the correct interpretation of the Survey results will be missed or ignored by some community groups, who may interpret the resulting proportion for or against same-sex marriage as representative of the opinion of all Australians. This may subsequently, and erroneously, damage the reputation of the ABS and the statistical community as a whole, when it is realised that the Survey results can not be understood in these terms.
The natural question must then be asked: Should the Survey results represent the views of all Australians? The Government direction states that it should not, and that this is adequate for decision making. (It also conveniently produces numbers of voters at the electoral division level, for which this is an important issue.)
It is the professional opinion of the SSA, the body representing professional statisticians in Australia, that a perhaps more useful Government direction could have asked the Australian Statistician to collect statistics to estimate the proportion of all Australians in favour of or against same-sex marriage. Such a direction would avoid any chance of mis-interpretation, and would arguably be more useful for decision making in terms of understanding the views of Australians as a whole.
The value of statistical surveys
So if the aim was to understand the views of all Australians, and even though it was designed to answer a different question, how would the current Survey fall short?
In her article in the Conversation of 18 August 2017, Professor Louise Ryan (Distinguished Professor of Statistics at UTS) highlights some of the potential statistical problems with the Postal Survey. These include the non-obligation to vote, some community groups possibly boycotting the vote, the potential for proportionately fewer younger voters to vote in a postal survey, and the inability to properly adjust for non-response among different community sectors and age groups because appropriate demographic information will not be collected (under the Government direction).
This latter aspect can lead to certain groups being disproportionately overrepresented. For example, if younger voters have a lower response rate and are more likely to vote in favour of marriage equality, then the Survey will underestimate the true level of support. The opposite would happen if younger voters had a disproportionately higher response rate.
The SSA is not aware of any official statistics based purely on unadjusted respondent data alone. The ABS routinely adjusts population numbers derived from the census to allow for under and over enumeration issues via its post-enumeration survey. However, under the Government direction, there is no scope to adjust for demographic biases or collect any information that might enable the ABS to even indicate what these biases might be.
If the aim was to understand the views of all Australians, an opinion survey would be more appropriate. High quality professionally-designed opinion surveys are routinely carried out by market research companies, the ABS, and other institutions. Surveys can be an efficient and powerful tool for canvassing a population, making use of statistical techniques to ensure that the results are proportioned according to the demographics of the population. With a proper survey design and analysis, public opinion can be reliably estimated to a specified accuracy. They can also be implemented at a fraction of the cost of the present Postal Survey. The ABS has a world-class reputation and expertise in this area.
The SSA holds high standards of professional integrity, and is committed to public advocacy of the proper understanding and use of statistics and statistical methods. The SSA is concerned that limitations in the Government direction to the Marriage Law Postal Survey mean that, whatever the result, there will still be considerable uncertainty about the actual views of all Australians on the matter. This may lead to the outcome being disputed or misrepresented, and the reputation of the ABS and the statistical profession being erroneously damaged.
Professor Scott Sisson, SSA President
Dr Peter Baker
Chair of SSA Media and Communications Committee, [email protected]
Download a pdf of this media release here.
21 April 2017
The Statistical Society of Australia is proud to be involved in a recent “Open Letter for Science” organised by our partners of Science & Technology Australia, and which was printed in the Sydney Morning Herald. Its purpose was to illustrate the importance of science and technology to our way of life, and to ensure these sectors are unobstructed and supported by the community, government, and industry.
SSA President Scott signed the letter along with a list of scientific luminaries and their supporters on the eve of the Global March for Science last Saturday, 22 April 2017.
You can read the letter here or go to http://www.smh.com.au/technology/sci-tech/tim-minchin-and-australian-of-the-year-alan-mackaysim-join-forces-in-support-of-science-20170421-gvpiu3.
Statement on the 2016 Australian Census – Released on 8 August 2016
The Statistical Society of Australia is concerned that the current controversies surrounding the 2016 Census may impact upon the quality of the data collected and may be raising unnecessary fears in the community.
Australia is fortunate to have a history of highly regarded Censuses, amongst the best in the world. This has come about through the highly professional work of the Australian Bureau of Statistics (ABS) and has meant that the Census results have never been seriously questioned as they have been in many other countries, including leading countries such as the USA. Governments and the community rely upon and use data from the Census in many ways, perhaps more widely than many people realise. For example, the standard measures used to identify places of disadvantage and social need are all based upon the Census.
The ABS has had a history of being highly protective of the confidential information it collects, even before the current legislation that makes it clear that confidentiality is paramount. For many years it has collected names and addresses, so that field staff can check returns in initial stages of processing; these are not retained. The ABS already has systems in place to manage that information and keep it confidential. What is new is not the collection of names and addresses, but the retention of returns with identifying names and addresses for a longer time, and in that time use it to create more comprehensive and useful data sets by linking to other data, including previous Censuses.
As statisticians we recognise the potential value of more comprehensive data sets to enable better decision making in government. This can be to the benefit of all. We understand that access to this more comprehensive data will be subject to the same or stronger restrictions that are currently applied.
The Statistical Society of Australia is concerned that these changes, brought in with the 2011 Census and repeated in 2016, and that have many potential benefits, have not been handled well. In particular, the public whose cooperation is critical for a successful Census does not appear to have been adequately involved, and the reasons for the changes are even now not well publicised. This is an issue of transparency where the ABS needs to do better.
The Statistical Society of Australia hopes that the Australian public fully engages with the Census on 9th August 2016. The Society also hopes that the ABS adopts an approach of taking the community into its confidence in explaining what it is doing and why.
Download a pdf of this media release here.