The great Australian survey

In the 1947 film 'Magic Town', James Stewart plays an opinion pollster who discovers the town of Grandview: a perfect statistical mirror of the United States. Anything you want to know about the country can be found out merely by tallying the residents of Grandview.

Increasingly, we're starting to feel like Grandview residents. A few decades ago, one survey found that 23 percent of Americans are been polled annually – a figure that's probably risen as telephone calls have become cheaper and demand for poll results has grown.

But the deluge of polls makes us more reluctant to participate. So surveys have become less representative, with response rates mostly below 50 percent and falling. Consequently, the only way of making surveys accurate is either to pay respondents a fee (as the longitudinal HILDA survey does), or to compel a response (as the Australian Bureau of Statistics does). And for the ABS, the mother of all its surveys is the quinquennial census.

Censuses have their origins in assessing military strength. From 508 BC, the Romans conducted censuses to determine the tax base and number of men who could be called to arms. Military expansion under the Han Dynasty was facilitated by China's first census in 2 AD. William the Conqueror's 1086 census (which resulted in the Domesday Book) ensured that he could properly tax the country he had recently invaded.

The questions we ask say something about Australia as a society. For a century, we have asked Australians their religion, but it wasn't until 1976 that income became a regular question. By contrast, the US census has asked about income since 1940, but never about religion. Over the past century, the Australian census has ceased asking people how long they've been married and whether their house has a bathroom, and added questions about unpaid work and internet access.

Census questions have political power. Historians Len Smith and Tim Rowse point out that it was not until 1966 that the ABS was confident it had enumerated all Indigenous Australians (previous censuses had assumed a few thousand undetected Aboriginal people in remote areas). Full enumeration led to a 1969 Census publication that compared socio-economic outcomes for Indigenous and non-Indigenous Australians: a forerunner to today's Closing the Gap reports.

Similarly, the recent inclusion of a question about ancestry has led some migrant groups to lobby their members to provide a consistent answer, in order to gain greater public recognition. Meanwhile, there are still members of the Jewish community who say that the lesson of the twentieth century is never to write 'Jewish' on any official form.

For researchers, the ABS provides a 1 percent sample of the Australian census, which represents one of the largest datasets around. When I was an economics professor, I found the census particularly valuable for looking at rare events – such as whether couples with a son and a daughter are more likely to be married than those with two children of the same gender (answer: yes).

From a public policy standpoint, censuses are one of the ways that we decide how to allocate resources across communities. A constituent emailed me recently to point out that elections are virtually never decided by a single vote, but every additional person who answers the census brings resources to the area. She argued that it was more important to fill out the census form than to vote (a conclusion I could not possibly endorse, since I encourage everyone to participate in both activities).

Will we ever abandon the census? The Scandinavian countries have done just that, taking the view that their extensive national registers (linked by a unique identification number) make census-taking unnecessary. In the United States, questions about income, education and disability have now been taken out of the decennial census and moved into a new 'American Community Survey' which samples 3 million people annually. With fewer people to survey, the US Census Bureau is able to devote more resources to obtaining a properly representative sample, which is available on an annual basis.

When you're filling out your census form tonight, think of the myriad purposes for which it will be used. Whether you're a researcher or a curious citizen, Australia's census helps us all to better understand the nation. Better yet, if you tick the box to allow your information to be revealed in 99 years, you might even help your descendents with their genealogical explorations.