Vision Critical Podcasts - Transcripts

How Banana Republic Created a Successful Customer Advisory Panel

Announcer

Welcome to Around the Block, a series of podcasts from Vision Critical, the world leader in custom online panels and interactive research. In each episode our host, Caroline Hickton, interviews experts from the world of online market research to share best practices, learned lessons, and emerging trends. Are you ready?

Caroline 

Hi, I’m Caroline Hickton and welcome to a special edition of Around the Block. Direct from Chicago at the IIR Excellence in Market Research Conference we have a presentation by Banana Republic on how they’ve created a healthy and vibrant proprietary panel with consumers from New York to Tokyo. Hear from both Tamra Krefman, Consumer Strategist with Banana Republic, and Jen Reid, SVP Panel Services at Vision Critical,as they outline best practices to recruit, survey, engage, and manage a proprietary panel.

Jen 

What we are going to do this morning is Tamra is going to take you through a case study of what Banana Republic is doing but first just a couple of little background pieces. So, “how do we define a propriety panel” I guess is a good question. In general I like to say that it’s a group of people that a client will recruit, that they use from their exclusive purposes. It’s often costumers but sometimes it’s looking at their competitive marketplace. And in the case of Banana Republic it is their loyalists, it their good shoppers.

Banana Republic, in case anyone here doesn’t know, has about four hundred stores in US, in Canada, the Middle East, Japan, and entering the UK soon. So, right now the panel is only in the US but they do share the results up into the various countries and we have talked a little bit about, um, extending the panel to more a global stage as time goes on.

Vision Critical is a software company that provides software for the running and management of online panels. Our software looks after everything from the ability to program the surveys, to managing the panel, understanding what the health of your panel looks like, deployment and results. This is a list of some of the clients that we have; we have really a wide variety across a lot of different industries, so later if you have questions we can talk about how to apply the Banana Republic case study that Tamra’s going to talk about into other, uh, different kinds of industries.

I guess it’s important when you set off in the building of a panel to understand how the roles work, so that you understand what we each bring to the table. The client in this case, Banana Republic, is responsible for the definition of the research topics, the crafting and programming of the surveys. I know that Banana Republic often uses a third party vendor; we have a lot of clients that will come in and use other market research companies to design the research for them. Um, getting it integrated with the organization, I think this is something that Banana Republic has done a really nice job of and, um, we are going to see some examples of that. Uh, working with – with a researcher, if the case may be. We have some clients that run all of their own research internally; they do all the questionnaire design and others that, um, bring somebody else in to do that. And then also providing for surveys, Banana Republic uses a lot of photos and advertising, or imagery in their- in their studies. So, they’re also responsible, obviously, for providing that.

Vision Critical supplies the software. It’s an ASP, um, that clients purchase monthly licensing for, but we also found that while you could provide software for running a panel that we also need to be able to provide the expertise in terms of how you plan it, how you recruit it, and how you manage it on-going. So, that’s another piece, another service that we offer and can bring to the table. And that’s design, branding, understanding health, and then finally making recommendations as time goes by. “When is time to purge? When is it time to, um, refresh your data?” and those sorts of questions are things that we can bring to the table.

Good, so I’m going to pass it over to Tamra, who’s gonna [sic] give you a really great case study.

Tamra

I hope that what I can share with you today is a lot of the things that we thought about from when we launched this panel, until what our consumers experience each month as they receive a survey today.

Um, our panel – we launched in January for 2006. So, it’s been around for a year an half, we have about ten thousands panelists today which is less than one percent of our total consumer base and I’ll speak a little bit about how we decided how many people to invite, who we invited, um, just a lot of questions we had in the design and planning considerations. But, to date, we’ve had over twenty-two studies, so we aim for about one study a month. Sometimes we’ll divide the panel into two groups and men could, for example, receive one survey while women receive another survey. We’ve also used the panel for multiple photo exercises that I will be sharing with and recruiting for qualitative research, which has been phenomenal and the panelists have all been extremely articulate and very representative of who we want for some of our best costumers.

So, back in November of 2005 was when we started really saying “We want to launch this panel, how can we get it up and running pretty quickly?” and we got it up and running; all of the planning, all of the work through legal; within two months and we had our first survey out the door the first week of January of 2006. So, it was less difficult than I anticipated going into it, just thinking about how much lead time you would really need to do – to get something really up and running. That said, one of our luxuries is that we did use our credit card list and selected some of our best costumers to invite to the panel. So, I recognize that is a huge luxury, but for our purposes we wanted the panel to be loyal consumers. So, we selected some of our highest spenders with the brand and some of the people who transact most frequently. So there are people who are in our doors regularly and we were really able to trust them that way to give us feedback on the product, on their store experience, on the advertising. They love the brand and many of them will tell you that Banana Republic is one of their favourite stores.

That does lead to somewhat of a challenge when we message internally, in term of managing this loyal bias, and I’ll speak about this a little bit more later, but just acknowledging that these are some of our best costumers, so when we are disappointing them we’re most likely disappointing many other costumers as well. But when we’re pleasing them we have to realize that there is some lift above the general consumer population and overall attitudes.

So, as we were designing this panel we thought, “What ways do we wanna [sic] be able to look at our costumers?” And this all went into the profiling questionnaire. So, we had some very traditional things: such as income, such as age, what region do you live in? And then we had some more unique things that we asked in our profiling questionnaire to make sure that we had a good mix of costumers.

“What’s your favourite store besides Banana Republic that you shop at?”

“How often do you shop online?”

“What size are you? Are you petit, are you tall?”

So we could really get a good mix of costumers, and as we invited them to join the panel and reviewed the initial applicants we were able to try and aim for a representative mix of our costumer population. So, we did look “do we have a good regional dispersion?” “do we have a good age mix and it is representative of what we know about our cons- our costumers form other tracking studies that we do?” And we try and integrate the panel results with results from other research as we message within the organization. So, for example, we have a Point of Sale survey where with your receipt, you can take a survey, go online. We look at those results compared to the panel results in our loyal population and try and match the two to see where are there disconnects and where are there synergies.

We’ve also overlaid the transactional spend data with the panel, which has been huge. And like I said, the panelists are some of our best costumers. Now, we looked at them and said “Who was best in 2005 and 2006?” We looked at another group “Who was great in 2005 and then their spend levels trailed off a little bit in 2006”, “who’s spend increased over those two years?” and we backfilled this data into the panel portal. So, now, I can go and cross-tab against those different groups, we can call them, like, “Cream of the Crop”, “High Spenders for two consecutive years”, look at their attitudes compared to people who’s spend has recently trailed off. We can tailor questions accordingly if we want to send questions to just one group of those people and hopefully, in future evolutions, we would like to get it even more granular level: looking at their transactional spend, who bought items at regular price versus on sale and what we know from the credit card side of things and then map it back to asking the questions about their attitudes.

Um, so in terms of determining the size of our panel we worked really closely with Vision Critical to think about what can we anticipate for response rates to be, how many people do we think we need to invite, and initially we ended up mailing to one-hundred thousand costumers and it got a panel of about thirteen thousand that we started with. So, it was a pretty strong response rate and at that point, about six months later we did our first purge. And we went through and looked at people who hadn’t been responding to most of the surveys and sent them a “Last Chance Survey”; said “this is your last chance to remain an Insider, do you want to stay an Insider or not?” The ones who didn’t respond to that survey we did purge. So, we lost about… three thousand costumers at that point, which was 20% to 25% and that was after six months. Since then the retention rate has been amazing and we are going to do our second purge, probably in a few months and I am only anticipating that we will lose one or two thousand additional costumers. So we saw some of that attrition at the very beginning but since then they’ve been excellent about continuing to respond to surveys.

Who are the Insiders? So, as I said they were selected because they were really high level Banana Republic spenders. We tried to make them representative of our target costumers, so 84% are between 25 and 44, they are nationally dispersed, they are crossed [sic] size ranges, their representative of our target segments with heavier focused [sic] on our target segments that are more fashion engaged. So, our segmentation was part of the profiling questionnaire and we were able to heavily weight towards those that were fashion engaged since we are asking them about fashion and trend. As some other fun facts that we’ve learned throughout the year about these people: 73% of them work more than a forty hour work week, of all celebrities that they can most envision wearing Banana Republic would be Jennifer Aniston and Jake Gyllenhaal, and of our men twice as many wear boxers than wear briefs. (Laughter)

So, those are just some examples of questions that we’ve been able to gain throughout the year, and the types of questions we ask them really vary. So, each survey we try and concentrate on a particular topic and I’ll speak more about this shortly, but one time it might be all about for-benefit product and cause marketing and we try to keep the entire survey geared on that one topic. The next month they may get a survey about handbags, jewelry, scarves, socks, boxers, non-apparel and that would all be one survey. So, we try and keep the focused on what we’re asking them about, it’s not questions out of the blue about ten different things but throughout the year they are answering about a wide variety of topics.

How does Banana Republic Insiders work? This is our login screen, and it’s… lightly visible right now, but in the background, we change this out about four times a year to really be consistent and be able to represent our brand image and personality. So we put whatever the current advertising campaign is. It’s like a watermark in the background on the login page and that’s something we work with Vision Critical on to be able to execute. It’s really nice because it does keep the panel portal feeling fresh to the costumers as they login each quarter. Many of them won’t even see this page, because what happens they receive an email in their inbox, they can click on the link directly and go to the survey online. But, if they choose to go to the Banana Republic Insiders homepage this is what they would see and then they can log on with their email address and see if they have any live studies that they are able to respond to.     

So, like I said, costumers are contacted about every three weeks to fours weeks with a new invitation, a new survey invitation. The topics are pretty much unlimited so we can ask them anything. We – once a quarter we do a seasonal shopping survey and we keep a standard battery of questions for those surveys. So, we’ll ask “In our Fall Collection, what did you most notice?” So, it will be an open ended question just to try and get some reactions to cut though messaging; what are they noticing in our stores, or in our campaign. Then we’ll ask a standard battery on questions that include things like “How appropriate were the clothing – was the clothing for work?” “How seasonally appropriate was the clothing?” So they are a few statements that we keep and ask consistently each quarter so we are able to look at them almost as a mini tracking study. We use it more for directional reads than for truly statistically significant movement. So we do try and message that within the organization that this is directional reads but we try and keep some of those questions consistent. Um, and then, like I said, we’ve used panel for invitations to qualitative research which I’ll show you some photos of shortly and for photo studies.

And, on the next page, this is what the panel portal looks like to us, as the client. And we can go in and we see what are out pending studies, what are out live studies and what are our closed studies. And as Jen was mentioning we do work with a third party vendor to program most of the surveys, so I’m not in the tool doing most of the programming, but I do use this page frequently to check things like the participation statistics. So, we invited people two days ago to a new survey, how many of them have responded so far? The topline, where I can look really quickly and see, you know, this is real time, so I can launch a study and go in six hours later see what are the most commonly… chosen… selections at that point. I also look – sometimes do crosstabs. And you can crosstab from a current study to a past study. So, let’s say… a few months ago we asked someone about how they liked the Spring Collection and then this time we are asking them some questions about summer. So, we may take the people who said they didn’t like spring and look at what are they saying about summer. So, you can cross tab from one study to another, which is all very intuitive, very user friendly. I had pretty minimal training on this but was able to go in and navigate the technology very easily. You can also export verbatim responses, so we typically include a few open ended question within each survey and easily are able to export those responses and look at some of those things that I was telling you about. Look at their demographics from past studies, look at where these people live, so you can choose any field that you want, export that data, and get really nice profile and able to add quotes and enhance any other presentations that we are giving with more of a picture about these people are.

This is a quick example of what the email invitation may look like that the costumers receive, this one was a little but long, but it usually just makes reference of what the topic is, that we are asking them about, and tells them what the incentive is for that particular month which I will talk about out incentives shortly. It’s all sweepstakes based.

And these are some of examples of a few different questions we’ve used in past studies. So, we always have an advertising study each quarter and we’ll show them the campaigns like you see at the top and again we ask a standard battery of questions about our advertising. We’ll usually ask a few open ended questions and then some particular, maybe trying to assess how well we are delivering on what the creative brief was, or we have the flexibility to change that each time. But, this was an example of our summer advertising where we showed them the imagery we’ve done different. When we’ve done some pre-advertising we showed them multiple images and asked which ones they’ve liked best. And although advertising has been shot at that point in can impact rotations or where we’re gonna [sic] place most heavily in terms of media. So, it can have some pretty quick implications, that way.

This is another examples, this was just a basic yes or no question that qualified them for another part of the survey but asking about particular t-shirts within the store: have they purchased them? If they said yes, we would ask them some of the most compelling attributes of why? If they said – if they said no, we would ask them what was barriers to purchase for that particular product.

These are just some examples of how we incorporate imagery. We just did something where we incorporated the Flash technology into a study we launched last week. And we are able to be pretty creative, and knowing that these people are so engaged and loyal we really able to dig deeper into the product with them. They’re more familiar with our stores, they are in our doors usually at least once a month, so they are very familiar and are able to articulate specific details about the product that your everyday costumer, who comes in once or twice a year, isn’t able to share with you as much.

These are some examples for incentives, so the way they we run our incentives is we say “by completing the survey you get an additional entry to the sweepstakes for the month”. Once a month we do a sweepstakes, it’s usually valued at five hundred dollars, sometimes we do it pretty straight forward – “you can win a $500 Banana Republic Appreciation Card this month”. We try to change the incentives every single month and keep them exciting. So, you’ll, see we had five winners and they all could win their selections of our new personal care fragrance, and this was before it was in stores. It was like a preview for them as Banana Republic Insiders as something that they could win. During the holiday time we had three people who won cashmere throws. We had these bags at the bottom – we had two men’s bags and one women’s bag, that was one month’s prize. So we had three winners that particular month. Coming up we are doing something were you can win a two hundred dollar appreciation card and an appointment with a personal shopper in one of our stores. So, we try and give them access things that regular costumers don’t necessarily have access to, to make them feel a little bit more exclusive or a little bit more elite. Whether it’s limited edition product, whether its product that they receive a month before it actually on the sales floor. So, just try to keep things exciting for them and wanting them to stay as Insiders. We have tried doing American Express gift cards and the response rate was lower than when we used our own product and we actually asked them a question in one of surveys of “What types of products would you most like to win?” and we had them rank the types of products that they would like and they said that they would like Banana Republic Limited Edition merchandise. So, that’s what we try and provide.

In terms of… this is an example of a panel health report and this is a generic one, not specific to our brand, but this is something that Vision Critical prepares for us on a monthly basis and gives us what does our panel health look like. How many people responded, how many people are considered ideal respondents at this time. They compare it to norms across all their panels and then provide recommendations of when we should be doing a purge and when we should be doing a refresh. So, it’s pretty helpful to be able to have that snapshot on a monthly basis.

How to engage panelists. So, we’ve done a few different photo upload activities. We did one last holiday timeframe where we had them take pictures of gifts that they were giving to people. Tell us where they bought them, how much they spent on them, who they were for and, uh, why did they purchase this particular product. And we were able to share that with the merchandizing and design teams. Some other really helpful things internally have been these books which I will be able to pass around for you guys to take a look at. We did this set last year: The Banana Republic Woman and The Banana Republic Man: A Look into His Closet where we asked them to take photos of themselves for different occasions. So, take pictures of yourself when you are going to work, take pictures of yourself when you are dressed casual and tell us about where the clothing is from, why they purchased it, and able to give us a more holistic picture of who are these people and what are they really wearing.

And then a few months ago we did a new activity around versatility and this reminds me of something that was brought up this morning around having costumers helping you define a word. So, we continually hear from costumers “I buy product because it is versatile.” This is something we hear over and over again in focus groups, in other studies that we’ve done. So, we then incorporated this word “versatile” into our product filter, so all the merchants are aligned with “we are trying to make our product versatile” but they really didn’t have a strong idea of what that meant. So, we reached out to our panelists and did a photo study on the topic of versatility and what does that really mean? Take a picture of yourself with one item and wear it three different ways. Take pictures of yourself in each of those items. They’d upload those photos. Other people: take pictures of yourself that you wear to work and after work but you don’t change in between. What do those outfits look like? So we were able to put these books together and share them with our design team and our merchandising team who respond much more favourably to visual imagery that to giving them a PowerPoint deck to read though. And now these books sit on their desks and they take them to meetings and they open them up and it just gives them such a stronger sense of who is out costumer, who are we trying to design for, who are we trying to sort the store for and it was a really, we thought, it was a really innovative way to use our panel. So, enjoy. I brought them with, so feel free to take a look and if I could just have them back before the end of the presentation.

Read and see. So, we did learn that it was somewhat different from men versus women. Where, for men is was more being versatile across their wardrobe and can they really envisioned themselves wearing that one shirt with a variety of different pants. Where for women it was more about occasion and can I wear this product for when I am dressing up and when I am dressing more casually. So, the two genders certainly had slightly different interpretations. More than anything it was showing our internal teams photo imagery of the panelists that really got them excited and to understand who are these costumers that they are designing for. I don’t know how many of you work with creative audiences but there is a, uh, heavy resistance to research in general. And also when it comes to fashion and trend being able to have them trust what costumers are telling you. They’re trying to be inspired and come up with a vision and share that with costumers and having it work the other way, we have faced some challenges. (Laughter) So, one – one thing I’ve really learned it the importance of gaining their buy in the beginning. So before we design any survey with our panelist we meeting with the relevant teams and try to understand “what would help you most?” You’re the - they’re the designer of sweaters, what would help you most to know about sweaters? What do you want to know about your costumer? And we’ll sit down with the merchandiser and the designer ahead of time and gain their buy in about what the survey should even look like. Then when we get the results back they are much more receptive to hearing the information. The other thing is learning of alternative vehicles of sharing information. Like I said, that it’s not always through a PowerPoint deck, but how can we do photo imagery, as an example, how can we use Flash cards, how can we use other tools that make them more excited and more receptive to the information. And honestly, when they saw the books for the first time, the first series of books, they were a little bit, like, cringed and “oh, this is who I am designing for?” and “oh, this is what you want my product to look like?” and we had to really be straight-forward that the goal of this was not to be prescriptive, but it was more to provide some food for thought and some context around who is your costumer truly, and who were some of your best costumers actually. What do they really look like? And don’t particularly design this way, but realize this is the way the clothing is being worn. So, it’s given them a much richer idea of who the costumer is.

These are some photos of some in-person research that we did. We knew that we wanted to do some research in New York. We actually had our designers and our merchants in the rooms for this research with the costumers and so we were able to use the panel and look and say, “we want just women, we want them between these sizes and we want them in Manhattan. Let’s send an invitation to those women.” And then the invitation to those women would say something like “as a valued Banana Republic Insider you are invited to meet some of our merchants and design team. Come to our headquarters in New York and share with us your opinions about your work wardrobe and pants.” And of course, we added some incentive for them to be able to do this activity. So then -  online, this was all online ahead of time, they would fill out the questionnaire – “are you available these dates and times?” a few additional screening questions, but we didn’t have to pay for any external recruit. We were able to recruit fully through our panel. So, that was a huge benefit and like I said, they were very articulate and it was really fun for our teams who have worked on developing the panel to actually get to meet them in person which has been really rewarding.

We did some, also, we did some fit testing a the Michigan Ave store last year, and we sent out to and email to panelist, they didn’t even have to respond to this one, we just sent it out to women in Chicago and we said, “we will be at the Michigan Ave store between this time and this time on this day. There may be too many people to participate but feel free to swing by, we’d love to meet you and if you qualify you can do the fit testing.” So, it’s been a really easy way to try and build additional traffic and addition recruiting methods for our women. We also did some usability testing for the website in San Francisco. Similar approach where we did all the recruiting online with the panelists.

So another way that we engage with the panelists is once a quarter we send them an electronic newsletter. And this is how we really try and treat them like “You are a Banana Republic Insider.” So, for example we have a Meet the Designers sections where we’ll tell them fun facts about out designers and things that they otherwise wouldn’t know as an everyday costumer. We have a What We Learned From You section, where we tell them that “this year you told us that our Spring Collection was far too neutral and didn’t have enough colour, we’re taking this in to account as we plan next year’s line. This year in  January you told us that everything we had in store was sleeveless and not seasonally appropriate, we are taking that into consideration as we plan for next January.” It’s trying to give them some actions steps of what – of how are we using the information and what should they expect to see in stores and a result of it. A couple other things from the newsletter that we do, is we always have Words from Our Winners of who won the sweepstakes prizes from that particular month. Uh, we also have some upcoming events for them. So, again, things as everyday costumer wouldn’t know. “The next new product is going to be in stores on X day.” “This promotion is coming up in two weeks and here’s what it’s going to be for Mother’s day if you spend over a hundred and fifty dollars in our store, you get a twenty-five dollar gift card with your purchase.” So, things that you wouldn’t know unless you were a Banana Republic Insider have been pretty enticing for them to stay involved and stay engaged.

We tend to stay away from product pre-testing because of a lot of the resistance that I was sharing with you earlier from more creative audiences. But we did use it in this sense. We tested three different handbag collections that were not yet in stores. They were already landed on in terms of the design and in terms of the fact that there were going to be in the collection but there was still some question in terms of depth, how heavily should we be buying each one and more in terms, for the online team, how should we be messaging behind each collection, which one should we featuring most prominently, which one should we expect the most challenges with? So, this is an example of something we did where Collection Two was the most positively received, it also ended up selling the best which was refreshing to see. But we learned about some challenges where Collection Three was perceived as less high quality, so when online went to message that on their site they were really able to beef up the elements of quality as an attribute that costumers were looking for but didn’t sense from these photos.

So, like I said, we do those quarterly surveys just on seasonal shopping perceptions and behaviours of our store. We include open ended questions in those surveys. We do quarterly advertising testing, and then we’ve done a bunch of topical studies. So, around brand stretch and brand extensions: where do we freedom to go and permission to go in our costumers eyes?  We’ve done charitable and for-benefit activities surveys. Survey about Mother’s and Father’s Day gifting. A survey about Holiday gifts. In general, we’ve done some category specific research, so around sweaters (specifically cashmere), around men’s woven shirts, around handbags and shoes, so the panel has allowed us to go in a lot of different ways, but like I said, through just deciding to launch an online survey we wouldn’t be able to get as much information with such quick turn around at as deep of a product level.

So, overall the benefits for us have been extremely quick turnaround and rapid research. We have, like I said, we have access to the results real-time, typically put some kind of a report or a recap with in two weeks of when we launch the survey for our internal teams, to be able to know what were the results. Being cost effective, category specific, and very proprietary. We use it for hypothesis generation and directional validation, sometimes we’ll use it as a precursor for other work that we are going to be doing, particularly qualitative work. We’ll use this to generate some initial hypotheses and then take them a level further when we go into other research. And just allowing us to be creating with things like the photo activities for example, that otherwise it would have been much more difficult to achieve.

So, challenges that we faced along the way that if you were to do the same thing I would caution you about are: One, not over communicating. So, we were speaking about this earlier but just being mindful of the internal calendar that you have where you are already communicating with your panelist and also there will be this desire and flood of request from the various business partners of “can you ask the panelist this? can you ask the panelist this?” We have had to be pretty diligent about creating a calendar and saying, “this month’s topic is X, next month Y, if it’s an urgent business need we’ll try and fit it in and rearrange things” but not over-communicating with the panelists. We have worked around that with things like splitting the panel and having two surveys going concurrently within one month to two different populations if there are additional requests from business partners.

[Two] Um. Keeping the panel fresh both with topics and incentives. I think we’ve been able to do this. We’ve, like I said, we’ve run twenty-two surveys it gets a little bit more difficult to come up with incentives so we’ve actually had internal brainstorms around “how can we come up with additional incentives?” because there are only so many handbags that you can give away.

[Three] Managing the loyal bias and with our internal partners, in particular, make them realize that these are some of your best costumers but you have to take what they are responding to and what they are reacting to with that filter in mind and then just communicating internally. So, I’m going to turn it back over to Jen…

Jen

What I wanted to do is really show you what are sort of the six basic principles when you’re planning a panel you should keep in mind and these are all things that I think Banana Republic in their own case but there are different answers to some of the questions depending on where you come from.

The top of the hierarchy, I think, when you are planning a panel is understanding purpose and tone. I think what, what Tamra has done a really good job of today is really talking about how they have a lot of focus about what their panel is; they understand what it is and what it’s for. And so if you have a panel that you want to run a lot of really traditional research on, let’s say, than you may need to be thinking about a blind panel, it needs to be bigger, or you have to go through a lot more rigueur in how you recruit it. Whereas if it’s a panel where you want to do really fast turnaround; we have some panels that do - all they want it for in 24-hour, gut checks. Um, well, that’s a different kind of panel that needs to be recruited in a little bit of a different kind of way. So, you want to first start by understanding what your purpose is.

The other thing that I think is so important in tone. I think as researchers we tend to treat respondents a little bit like lab rats sometimes. I mean, it’s – their  sort of the-the respondent… n, n=100 as opposed to it being a hundred people that you are talking to. In a panel situation you really need to actually reach beyond that and be thinking about the kinds of the people that you are recruiting and again, I think that with all that photo work that Tamra showed you can see how Banana Republic has done a really good job of that. But tone is important, and a lot of questions that I ask clients in the early days on panel planning is, is this corporate, is it fun, is it casual? Is it girly, is it stoic? You know, whatever the combination is, if you say it’s fun then it needs to be fun throughout. So it is what it looks like, what it’s called. The kind of ways that you are going to write the surveys, we have a panel of women that’s run by, actually, a man but he pretends to be, I don’t remember actually, Susan somebody or other, and he does a really good job trying to write in this, in a tone that’s appropriate, that has a certain level of casualness and conversation and in the kinds of research he’s doing that’s appropriate for. So, unlike an access panel really where you are used to just going and buying sample, when you have your own panel you really do have to think about the people that are on it and the relationship that you want to have with them.

The next piece is understanding your distribution. Tamra talked a lot about how they, about how this is a panel of loyalists and they understand what that is. One of the exercises we go through at the beginning of every panel is this panel distribution – understanding what an Optimal Panel Distribution would look like. I call that the OPD because “Optimal Panel Distribution” is a mouthful. But in the OPD we are looking at, not only these things which are “I wanna [sic] have a certain represenation of…west vs. east, or age or gender” or these pieces but also how I define my costumers becomes really important. Um, are they loyalist? Am I looking for people who, let’s say, have gone away from my brand, or shop in a competitive situation. You really need to understand who it is you are trying to get and that will effect the sources that you are going about to find them. I mean, obviously if you are looking for Purchase Intenders you are not going to get that off your credit card. We have to go and do something else for that. So, understanding distribution is real [sic] important.

The recruitment offer: I think offer, we can look at the bullet points there but really it boils down to two things. When you’re doing you recruitment you have to tell them a.) what’s in it for them and b.) you should tell them what’s in it for you. Because really it is not only about what they are going to get in term of incentives or information or feedback on data and those pieces but it’s also going to be making it really clear that, I’m going to be try and test products, I’m going to be wanting to test advertising, I want your input on this, that, or the other thing and the more… I was at the ESOMAR conference in Barcelona last year, and they talked, we had a great speaker, who talked about, research traditionally being an adult to child relationship. These are the questions just answer them don’t ask me why. Versus, really trying to move toward an adult to adult relationship where you are saying this is why I need to have this information, this is what I am going to do with about. I think it’s very important.

That leads you down the funnel to sources. Of course, um, Banana Republic was very lucky. They have a panel of loyalists; they’ve got a great response rate, by the way, 13% on a recruit off a database is about double what we see as average, it’s usually about seven. Um, and uh, there’s a tonne of sources that we can, you can use. You can use your websites. We’ve done postcards, you can do in-store, end of receipt. And we can also look at recruiting; I spend a lot of time personally, doing recruiting for blind panels where we have to go externally from a client’s assets to do recruitment. And while that can be a little expensive you can often get some very interesting people that way and start a very interesting dialogue that often a little less branded then who you would get from your own, internal lists.

Retention is, um, really important. I will say, I’ve got a couple slides on retention but the most important thing you can do to retain panelists to ask them smart, intelligent surveys, really. I mean, a lot of times we have clients that come and want to talk about newsletters and incentive programs and all those pieces, and all of that is important. But really, if you enjoyed taking the survey, “That was fun, that was, you know, didn’t take too much of my time, it seemed intelligent, I was happy to answer those questions…” then the next time I see that invite in my inbox I’m going to want to go and do it. If instead, it’s grid after grid about, you know, where id - we have soda pop company that have a panel and they wanted to ask fifteen grids about where was the last place you bought water, and where did you buy it, and was that at a 7-Eleven or at a gas station, and what size was it… you know, down a huge list of things and you have to figure out a different way to ask those questions when you’ve got a panel because that actually is the biggest danger to attrition.

Here we have an example of the fusion technology that Vision Critical offers and really, we are starting to work towards using a visual question as opposed to, so this would replace a grid, for example. So instead here you’re reading a statements and we are asking you to put them inside of a, um, of the boxes and drag and drop. And we recently did a validation study on this which we are going to be presenting at an ESOMAR conference in September and we found that by moving questionnaires into this way we were able to get a 20% lift on enjoyment of the survey and we see in some of our panels that have been doing this a lift that in ongoing response rates that are upwards of 30%. So, this really works as questionnaire design.

And then, of course, the other piece of it, which I think Tamra really should some nice stuff, is the other parts around that. Good, smart incentive programs that don’t have to be too complicated but a little bit of, have a little bit of, you’ve put a little bit of thought into what it is. As well as, newsletters and feedback and these pieces.

The last thing is just understanding the rules of engagement. Uh, how any times are you going to contact them, when are you going to purge them, minimal acceptable response rates, aging issues. So, if you asked a question to, Banana Republic doesn’t really have a lot of aging things but, if you were talking to pregnant women, right? Well, obviously you are going to have to figure out how you are going to monitor where they are in their pregnancy, for example. And then understanding on going recruitment.

So those are just sort of six basic principles that I think are important to consider if you are thinking about doing an online panel.

Caroline

We hope you’ve enjoyed hearing how our client at Banana Republic is using a propriety panel and the successes they’ve had. Any listeners that have comments or questions, feel free to send us an email. Our email is podcast@visioncritical.com. I’m Caroline Hickton and thanks for listening.