In our continuing quest for intelligence about different types of survey formats and suitability for different question types, our latest Research on Research compares card sorts and radio buttons.
When respondents were asked their level of agreement with 24 statements presented as a card sort or series of radio buttons we found significantly different responses depending on the layout of the question used. Their options were shown as a verbally labeled 5 point scale.
Attitude Statements - Don't Knows
With attitude statements there are fewer "don't knows" with card sorts than with radio buttons. Radio button responders are considerably more likely to straight-line and choose don't know for all of the statements than card sorter responders. The card sorters also had a higher number of responders who didn't choose "don't know" for any of their questions.
Attitude Statements - Mid-point
Using a verbally labeled five point scale and a question about "trustworthiness", radio buttons result in a slightly greater use of the midpoint and less use of the topbox than responses with the card sort.
In previous Research on Research we have seen even more pronounced mid-point effects and topbox drop-off with radio buttons compared to visual grids and sliders.
Behavioral Questions - Mid-point
Using a verbally labeled scale and question about media consumption patterns there is no discernable difference in scale usage.
Why Should This Be?
Card sorts collect better data with attitude statements, compared to radio buttons. The hypothesis is that, with a card sort, people are more engaged and encouraged to actually read and think about the statement.
This makes them less likely to just blow off the exercise, dash off a run of radio buttons, and straight-line with "don't know".
If you are asking about attitudes, don't use radio buttons. Cards sorts are a better alternative. Whether card sorts are better than visual grids is an open questionÛ_ With the verbal grid, we saw a tendency toward greater use of the mid-point with radio buttons. Don't use them if you don't have to.
A sample of 4027 people age 18+, approximately 1000 each in US and Canada and 2000 in the UK. A randomly selected half of each sample saw the card sort questions and the other half saw the radio buttons. The survey was fielded in October 2009.