No matter how you view it, Bush is on the nose

What do Australians think of the US or of President George W. Bush? How malleable are their views on the wars in Iraq and Afghanistan?

And how many support the US-Australia alliance?

A new survey, commissioned by the US Studies Centre at the University of Sydney and conducted among 1,213 respondents nationally, shows that how researchers ask questions such as these, the order in which they ask them and the response options they provide often, though not always, affects the answers they get. Public opinion is often more complex, more malleable and more context-dependent than the answers to single questions suggest.

What do we think, for example, about the impact of US foreign policy on Australia? In the USSC survey, half the sample were asked about whether they were worried about the impact of US foreign policy and the other half were asked whether they were pleased about it. While 38 per cent said they were very worried (rather then somewhat worried or not at all worried), no fewer than 52 per cent said they were not at all pleased (rather than somewhat pleased or very pleased).

And the pattern of response - many fewer respondents very worried than not at all pleased - was reproduced in relation to every other aspect of American influence on Australia the survey addressed in this way, including: trade subsidies (30 per cent, 56 per cent), policies on climate change (31 per cent, 52 per cent), business practices (19 per cent, 39 per cent) and workplace practices (20 per cent, 39 per cent). Advisers asked to look at the questionnaire anticipated a difference in the way the two questions would be answered, but the pattern they anticipated was precisely the reverse.

Sometimes, however, different ways of asking a question make no difference. The questions about Bush in the USSC survey are a case in point. Regardless of whether respondents were simply asked their opinion of the President, asked their opinion after saying how they thought most Australians viewed the President (75 per cent said that the overall opinion of most Australians was unfavourable) or asked their opinion after being told that most Australians seem to have an unfavourable view of the US President, the pattern of response was remarkably similar: 63 per cent unfavourable, 68 per cent unfavourable and 66 per cent unfavourable, respectively.

Nor is there any evidence that respondents' views were affected by a sense that expressing hostility to Bush was politically correct. Respondents were less unfavourable to Bush (68 per cent) than they estimated most Australians to be (75 per cent).

Even where the wording of the questions is the same, the order in which questions are asked can affect responses. In the USSC survey, what respondents said about Iraq and Afghanistan partly depended on whether they were asked about these conflicts before or after being asked questions about the threat posed by al-Qaida or Islamic fundamentalism and whether Australia's decision to join the US government in the war on terror had made Australia more of a terrorist target.

Those who were asked about Australia's military presence in Iraq before being asked these questions were less likely (33 per cent) than those who were asked after (39 per cent) to support Australia's presence. Similarly, those who were asked about Australia's military presence in Afghanistan before being asked these questions were less likely (44 per cent) than those who were asked after (50 per cent) to support Australia's presence.

This suggests that the framing of these conflicts matters; more particularly, that the threat from al-Qaida or Islamic fundamentalism (about which respondents were divided) is a more powerful mobilising device than the idea that the war on terror has made Australia more of a terrorist target (notwithstanding that three-quarters of the respondents said it had).

The connection between the nature of the threat and attitudes to the wars is confirmed by looking at the process in reverse.

While those who were asked about the threat from al-Qaida or Islamic fundamentalism after being asked about the wars in Iraq and Afghanistan were less likely to say the threat has been exaggerated (40 per cent) than those who were asked before (46 per cent), views about whether Australia had become a greater terrorist target remained the same.

Finally, there are the ways in which changes in response options can inform our understanding of what it is we are trying to measure. Asked how important the US alliance under ANZUS was for protecting Australia's security, 79 per cent said it was very important (37 per cent) or fairly important (42 per cent); few said it was not important (11 per cent) or not at all important (5 per cent); while 6 per cent didn't know or refused to say.

Contrast this with a poll, conducted a few weeks earlier for the Lowy Institute, in which no more than 63 per cent said our alliance relationship with the US was very important (36 per cent) or fairly important (27 per cent). In that poll, 27 per cent said the alliance was somewhat important, 9 per cent said it was not at all important and 1 per cent didn't know or refused to say.

An explanation for the contrast is not far to seek. Unable to choose Lowy's response of somewhat important, most of those in the USSC study who might have done so chose the more positive option instead, hence the higher proportion (41:27) choosing fairly important. A smaller proportion opted for not important or found the choice too difficult (6:1). In short, the USSC poll reported a different result, one more favourable to the alliance, because it offered respondents a starker choice.

As with differences, or the absence of them, generated by questions that are differently worded or by the order in which questions are asked, the consequences of varying response options can be significant, not only statistically but politically, too.