Research

Best Practice for Visual Questions

Best Practice for Visual Questions

In our continuing quest for intelligence about different types of survey formats and suitability for different question types, our latest Research on Research compares card sorts and radio buttons.

When respondents were asked their level of agreement with 24 statements presented as a card sort or series of radio buttons we found significantly different responses depending on the layout of the question used. Their options were shown as a verbally labeled 5 point scale.

Attitude Statements ‰ – Don’t Knows

With attitude statements there are fewer “don’t knows” with card sorts than with radio buttons. Radio button responders are considerably more likely to straight-line and choose don’t know for all of the statements than card sorter responders. The card sorters also had a higher number of responders who didn’t choose “don’t know” for any of their questions.

1 Best practice for visual questions2 best practice for visual questions

Attitude Statements ‰ – Mid-point
Using a verbally labeled five point scale and a question about “trustworthiness”, radio buttons result in a slightly greater use of the midpoint and less use of the topbox than responses with the card sort.
In previous Research on Research we have seen even more pronounced mid-point effects and topbox drop-off with radio buttons compared to visual grids and sliders.

3 best practice for visual questions4 best practice for visual questions

Behavioral Questions ‰ – Mid-point
Using a verbally labeled scale and question about media consumption patterns there is no discernable difference in scale usage.

5 best practice for visual questions

Why Should This Be?
Card sorts collect better data with attitude statements, compared to radio buttons. The hypothesis is that, with a card sort, people are more engaged and encouraged to actually read and think about the statement.
This makes them less likely to just blow off the exercise, dash off a run of radio buttons, and straight-line with “don’t know”.

We Recommend
If you are asking about attitudes, don’t use radio buttons. Cards sorts are a better alternative. Whether card sorts are better than visual grids is an open question‰Û_ With the verbal grid, we saw a tendency toward greater use of the mid-point with radio buttons. Don’t use them if you don’t have to.

Methodology
A sample of 4027 people age 18+, approximately 1000 each in US and Canada and 2000 in the UK. A randomly selected half of each sample saw the card sort questions and the other half saw the radio buttons. The survey was fielded in October 2009.



Subscribe to the Vision Critical blog

Get free customer intelligence tips and resources delivered weekly to your inbox.

By completing this form you consent to receive emails from Vision Critical. You can unsubscribe at any time. Learn more in our privacy policy.