Center for Strategy Research

• 
• Vol 2 Issue 1, January 2006

Happy New Year and Welcome!

You don’t need to be Emily Post to know that too many guests at an event can detract from even the best setting.

And yet, many companies put the success of their research projects at risk when they also attempt to overpopulate them — with questions! Read on for our take on why this is problematic.

As always, your thoughts and comments are appreciated.

Julie Brown President

Mark Palmerino Executive Vice President

Don’t Crowd the Pool

One of our company’s largest clients is based in Miami. Of course it’s January here in New England, and my icy toes and I are well aware of our good fortune whenever we visit this client — sometimes several times — during the winter.

As if that isn’t upsetting enough to those of you who also hail from snow-bound climates, I would also like to mention that on my last trip to Miami, I was invited to a beautiful pool party!

Interestingly, what made it so beautiful was that in addition to the glorious weather, the number of guests was limited. My experienced hosts knew that even an Olympic-sized swimming pool gets less and less inviting as more hot, thirsty people crowd around it.

If you’re still with me (and believe me, I know that all this talk of pool parties is enough to drive any driveway- shoveling citizen to reach for the delete key), you’re probably wondering what any of this has to do with research. Good question.

It’s simply this: Loading up a research study with too many objectives, or, the frequent result, too many questions, is the research manager’s equivalent of an overcrowded pool party.

I lost count long ago of the number of times a research manager has introduced a question pattern with little correlation to the objectives of a given study with: “Well, we thought, since you have these people on the phone, you could ask this too.”

Because while this may seem to make sense at first blush, in practice, it’s just the opposite. The temptation to take advantage of the presence of a willing survey participant to ask as many questions as the respondent has time and patience for adds little useful data, often adds cost, and may even reduce the value of the questions that are on topic.

Here’s a good example that came up in a recent study we undertook for a leading media conglomerate. This conglomerate, whose sponsorship of the survey was not disclosed, structured the questionnaire to start by asking about diversity initiatives within the Fortune 1000 companies to which this survey was directed.

After about 10 minutes however, the question pattern changed completely — driven by the motivation to “take advantage of the fact you have these folks on the line” — to a series of questions about media use and newspaper/newsmagazine readership patterns.

It didn’t go well. Three reasons why: Transition, Target, and Transparency.

1. Transition. When the subject changed abruptly, it was difficult to manage the transition from one topic to another within the interview. Even the most well-crafted transition verbiage requires time to deliver and a concurrent increase in respondent patience (i.e. decreased completion rates, resulting in increased costs).
2. Target. It turned out that much of the target audience to whom the media use questions (part two of the survey) were directed, were entirely clueless about diversity initiatives within their corporation (part one of the survey). Therefore, their level of engagement for the first 10 minutes of the interview suffered and the reliability of the answers was suspect. Others simply told us to go talk to the HR or Diversity officer about the questions, a detour which resulted in two interviews rather than one as planned
3. Transparency. Some number of executives terminated as soon as we got to the media questions, expressing frustration that two entirely unrelated areas were being merged into one survey. As intelligent and sophisticated buyers and users of research themselves, it was transparent that the client was trying to load multiple objectives into one conversation.

In summary, I can’t emphasize enough the importance of not adding questions to an interview that don’t relate directly to a study’s main objectives. Like a poorly planned pool party, too much overcrowding prevents anybody from getting the results they’re hoping to achieve.

Thanks for reading. Now, I’m headed back to Miami and away from New England winters!

— Julie