Research

Determining how many questions to ask each respondent

Determining how many questions to ask each respondent

The last time I mentioned beer in the blog title it went over well, so I thought I‰’d try it again. With the holidays coming up, perhaps I will do some empirical research and answer this question definitively. However, you won‰’t be surprised to hear, when I googled the question, the answer seems to be ‰”it depends‰”.

‰”It depends‰” is also the answer to ‰”how many questions should you ask in a Choice Based Conjoint (CBC) study?‰” My previous blog on this topic emphasized the fatigue and loss of engagement that results if you ask respondents too many conjoint questions. However, it didn‰’t directly answer the question ‰”how many questions are too many?‰”

In our follow up paper, ‰”How Low Can You Go?: Toward a better understanding of the number of choice tasks required for reliable input to market segmentation,‰” my colleague Andrew Grenville and I examined the relationship between the number of tasks respondents were asked to complete and the quality of the model we could derive from their responses. In particular, we wanted to ensure that we asked enough questions to understand individual respondent‰’s preferences and tradeoffs. The exact limit for each individual respondent varies and is dependent on a number of factors. However, we found that under normal conditions a conjoint exercise involving ten tasks was sufficient for modeling purposes. So while we don‰’t know how many is too many for any individual respondent, we shouldn‰’t need more than ten to be able to build solid models that allow us to understand their individual purchase behaviors.

As to the original question ‰ – how many beers are too many ‰ – please let me know the results of any pertinent research results you may obtain.

On second thought ‰ – don‰’t tell me. Really, I don‰’t want to know.

References:

Johnson, R. and Orme, B. (1996), ‰”How Many Questions Should You Ask In Choice-Based Conjoint Studies?‰” ART Forum Proceedings.

Rathod S and LaBruna A (2005), ‰”Questionnaire length and fatigue effects ‰ – does size really matter?‰” ESOMAR, Conference on Panel Research, Budapest.

Tang, J. and Grenville, A. (2010), ‰”How Many Questions Should You Ask in CBC Studies? ‰ – Revisited Again‰” Sawtooth Software Conference Proceedings.

Tang, J. and Grenville, A. (2012), ‰”How Low Can You Go?: Toward a better understanding of the number of choice tasks required for reliable input to market segmentation‰” Sawtooth Software Conference Proceedings.



Subscribe to the Vision Critical blog

Get free customer intelligence tips and resources delivered weekly to your inbox.

By completing this form you consent to receive emails from Vision Critical. You can unsubscribe at any time. Learn more in our privacy policy.