500? 1000? 20,000? Is bigger better or less more? Many access panels like to brag about how many members they have, citing that size is important to ensure better representation. Whilst even this is spurious without factoring in response rates, comparing the size of your community panel to access panels is irrelevant since it’s like comparing apples and oranges. In fact, comparing one community panel’s size against another does not work either, as you will learn as you read on. Often the decision around panel size takes place far too soon, when a company is still shopping around and putting out tenders for a community panel. Typically, not enough is known at this point to make this call – rather, the decision about how big your community panel needs to be to meet the needs of your research program is a complex question and should be part of a more sophisticated panel strategy conversation.
The following are just two of the key the questions you need to ask in when determining how big to grow your community panel. Your answers to these questions need to be factored in to your calculation for community panel size:
A first consideration is that consumer respondents will tend to be more willing to answer than business respondents since for the latter you may well have their business email and be contacting at work, where you will be competing for their time against their workplace priorities.
Understanding the interest amongst your customers for your particular category (think Sport versus Finance) will also provide a picture of how responsive you can expect potential community panel members to be. Beyond the category in general, how strong your brand resonates with its customers is perhaps an even bigger factor as to what your response rate will be: Apple claims to have around 80% response for their customer satisfaction studies, but don’t expect the same if you are the tax office!
- 1. Who do you think you are? – What category or industry are you in? Are you B2B or B2C? How well does your brand resonate? Back in the days when everyone used access panels, this was not something that you really had to concern yourself with. The access panel company worked out how strongly your category would resonate and take this into consideration and charge you accordingly. However, when managing your own community panel, you need to be able to make some estimates around the likely response rate to your survey invitations in order to know how big a panel you’ll need: the higher the response rate you expect, the fewer community panel members you need to recruit to make sure that your surveys receive (and continue to receive) enough completed responses to give you the sample size you need for valid conclusions.
Another reason that a rough research roadmap for your community panel is key circles back to your answers to the first question about who you think you are. If your research requires you run several surveys a month, you may decide that this is just too much to ask any one of your community panel members to do. Say you decide that, based on brand affinity, and who your members are likely to be, it’s reasonable to ask member to take two per month, but you need to conduct four studies in total with the same sample sizes, you are going to need twice the panel size.
- 2. Know your research schedule – Unless you know what your research needs are, you are going to struggle to decipher the correct number of members to recruit. Are you going to be testing the occasional advert on a monthly basis against a national representation or undertaking your entire brand’s research program on the community panel with specific cells? You do not need to know the exact projects you will be running, but having a good idea as to the type of research and frequency will help with this particular numbers game.
Based on these two elements, here is a quick formulae to follow to work out how many customers you should start with for your community panel. I have broken it into two for simplicity. The first part deals with how the percentage of people who are likely to respond to each study invitation, and the second part factors in the number of studies you need to undertake, versus the number you are willing for any individual to be invited to.
Required Sample Size / Estimated Response Rate = Required Members per study
(e.g. 500 completes / 40% response rate = 1250)
(Number of Studies per month / Max invites per member per month) X Required Members per study = Required CP Members
(e.g. 4 studies per month / 2 invites per member per month) X 1250) = 2500
There are several more factors that may be taken into account when calculating the size your Community Panel should be. These include:
- Û¢ Whether your surveys will be at 100% screening incidence
Û¢ An estimate for churn (number of people that will drop out of the panel) before re-recruiting
Û¢ If you have more than one segment in the panel that each needs to achieve a certain minimum response size per survey, which may have different response or churn rates, or even need to be surveys with a different frequency.
Fully taking into consideration all these elements often requires more of a spreadsheet than a formula but the above still provides a good initial working estimate.
Finally, of course, your budget will also help to determine how big your community panel can be: especially if you have no or limited internal resources for recruitment like sample lists your marketing or customer departments already own. However, if you invest to recruit more members to your own community panel, you will see a return on your investment as you push more erstwhile ad hoc through at little incremental cost.