Anonymity may spoil the accuracy of data collected through questionnaires

Thousands of psychology papers are based on data derived from questionnaires that were filled out anonymously. That's because most psychologists have reasoned that the way to get people to be honest about their practice of undesirable behaviours is to promise them anonymity. But in a new analysis, Yphtach Lelkes and his colleagues point out that anonymity comes with a price. Participants will feel less accountable and may be less motivated to answer questions accurately.

To test this, Lelkes' team devised a cunning methodology in which dozens of undergrads conducted internet research for what they thought was a study into the way that people search for information on the web.

After each student had spent 45 minutes researching the mountain pygmy-possum, a researcher made a show of deleting the student's search history before their eyes, ostensibly to prevent the next participant from accessing the browser's archives. In fact, a spyware programme was installed on the computer and kept track of all the sites visited. After the research session, each student answered a questionnaire about their use of the internet in general and their experience of the internet research task, including which sites they'd searched. Crucially, half the students were instructed to fill out their name and other personal details at the top of the questionnaire; the others were told to leave it blank to ensure anonymity.

Students who answered the questionnaire anonymously admitted to more embarrassing internet behaviours in general, such as looking at porn, but regards their searches specifically during the research task, they answered with less accuracy. There was also evidence of a lack of variety in many of the anonymous students' later answers, consistent with the idea that they were putting less thought and effort into the questionnaires as they grew tired.

Two follow-up studies involved dozens more students having the opportunity to eat M&M sweets and jelly beans while they completed questionnaires. A question at the end asked them to report how many they'd eaten and once again, students who answered anonymously were less accurate about how much they'd indulged. This was the case whether anonymity was promised before or after the opportunity to eat the snacks.

Lelkes and his colleagues were cautious about how far these findings can be generalised. For example, the same problems might not apply when people are interviewed face-to-face but promised confidentiality. However, they warned researchers against assuming that promising participants anonymity means that they will provide better quality answers. "Particularly among college students who often complete questionnaires to fulfil course requirements, such a guarantee may serve to sanction half-hearted survey completion rather than freeing students up to respond with greater honesty."

_________________________________

Yphtach Lelkes, Jon A. Krosnick, David M. Marx, Charles M. Judd, and Bernadette Park (2012). Complete anonymity compromises the accuracy of self-reports. Journal of Experimental Social Psychology DOI: 10.1016/j.jesp.2012.07.002

--Further reading-- Another recent study found that anonymous web participants provided quality data for psychological experiments.
Our bias for the left-hand side of space could be distorting large-scale surveys.
Is a taste for extreme answers distorting cross-cultural comparisons of personality?

Post written by Christian Jarrett for the BPS Research Digest.