Center for Strategy Research

Boston

Center for Strategy Research
Skip to content
  • Home
  • What We Do
    • Research Approaches
    • Industry Experience
  • Benefits of Working with Us
    • Insights
    • High Value Audiences
    • Our Twist on Research
    • More Cost-Effective Research
    • Hands-On Senior Researchers
  • Who We Are
    • Team Members
    • Achievements
    • CSR in Print
    • Join Our Twisted Team
  • Examples of Our Work
  • Our Newsletter
  • Contact Us
  • In This Issue

    Don’t Crowd the Pool
    Mixology (Putting Research into Practice)
    Twist and Shout
    About Us

    Last week we e-mailed each of you an invitation to share your feedback about this e-newsletter. If you have not yet done so, please click the link below to participate in a very short survey we are using to solicit feedback from “Research With A Twist” recipients.

    As a “thank you” for a few moments of your time, we will donate $5.00 to the American Red Cross. We very much appreciate your opinions of our newsletter and how we can improve it.

    Just click here to share your ideas with us.

    “Not everything that can be counted counts and not everything that counts can be counted.”

    — Albert Einstein

    Questions? Click here to send us an email with your request.
    csrinfo@csr-bos.com
    www.csr-bos.com
    (617) 451-9500
    • Vol 2 Issue 1, January 2006

    In This Issue

    Don’t Crowd the Pool
    Mixology (Putting Research into Practice)
    Twist and Shout
    About Us

    Happy New Year and Welcome!

    You don’t need to be Emily Post to know that too many guests at an event can detract from even the best setting.

    And yet, many companies put the success of their research projects at risk when they also attempt to overpopulate them — with questions! Read on for our take on why this is problematic.

    As always, your thoughts and comments are appreciated.

    Julie Brown
    President
    Mark Palmerino
    Executive Vice President

    Don’t Crowd the Pool

    One of our company’s largest clients is based in Miami. Of course it’s January here in New England, and my icy toes and I are well aware of our good fortune whenever we visit this client — sometimes several times — during the winter.

    As if that isn’t upsetting enough to those of you who also hail from snow-bound climates, I would also like to mention that on my last trip to Miami, I was invited to a beautiful pool party!

    Interestingly, what made it so beautiful was that in addition to the glorious weather, the number of guests was limited. My experienced hosts knew that even an Olympic-sized swimming pool gets less and less inviting as more hot, thirsty people crowd around it.

    If you’re still with me (and believe me, I know that all this talk of pool parties is enough to drive any driveway- shoveling citizen to reach for the delete key), you’re probably wondering what any of this has to do with research. Good question.

    It’s simply this: Loading up a research study with too many objectives, or, the frequent result, too many questions, is the research manager’s equivalent of an overcrowded pool party.

    I lost count long ago of the number of times a research manager has introduced a question pattern with little correlation to the objectives of a given study with: “Well, we thought, since you have these people on the phone, you could ask this too.”

    Because while this may seem to make sense at first blush, in practice, it’s just the opposite. The temptation to take advantage of the presence of a willing survey participant to ask as many questions as the respondent has time and patience for adds little useful data, often adds cost, and may even reduce the value of the questions that are on topic.

    Here’s a good example that came up in a recent study we undertook for a leading media conglomerate. This conglomerate, whose sponsorship of the survey was not disclosed, structured the questionnaire to start by asking about diversity initiatives within the Fortune 1000 companies to which this survey was directed.

    After about 10 minutes however, the question pattern changed completely — driven by the motivation to “take advantage of the fact you have these folks on the line” — to a series of questions about media use and newspaper/newsmagazine readership patterns.

    It didn’t go well. Three reasons why: Transition, Target, and Transparency.

    1. Transition. When the subject changed abruptly, it was difficult to manage the transition from one topic to another within the interview. Even the most well-crafted transition verbiage requires time to deliver and a concurrent increase in respondent patience (i.e. decreased completion rates, resulting in increased costs).
    2. Target. It turned out that much of the target audience to whom the media use questions (part two of the survey) were directed, were entirely clueless about diversity initiatives within their corporation (part one of the survey). Therefore, their level of engagement for the first 10 minutes of the interview suffered and the reliability of the answers was suspect. Others simply told us to go talk to the HR or Diversity officer about the questions, a detour which resulted in two interviews rather than one as planned
    3. Transparency. Some number of executives terminated as soon as we got to the media questions, expressing frustration that two entirely unrelated areas were being merged into one survey. As intelligent and sophisticated buyers and users of research themselves, it was transparent that the client was trying to load multiple objectives into one conversation.

    In summary, I can’t emphasize enough the importance of not adding questions to an interview that don’t relate directly to a study’s main objectives. Like a poorly planned pool party, too much overcrowding prevents anybody from getting the results they’re hoping to achieve.

    Thanks for reading. Now, I’m headed back to Miami and away from New England winters!

    — Julie

  • Mixology (Putting Research into Practice)

When an Answer, Isn’t!

“There is no greater impediment to the advancement of knowledge than the ambiguity of words.”

— Thomas Reid, Scottish Philosopher

In a previous Mixology, we discussed the ambiguity that can arise from a poorly worded question. Another area where ambiguity occurs is in the answers that respondents give.

In general, allowing people to respond in their own words to the researcher’s questions of interest leads to a much better understanding of what they think, and why they think it. This insight is critical to making the right decision with regard to the product or service being researched.

However, there are times when a respondent’s “answer” isn’t as full of useful information as we’d like. It is prudent, therefore, for interviewers to be sensitive to the ambiguity inherent in many words and be prepared with probes designed to help the respondent clarify what he means.

In the context of good open-ended interviews, many words can be ambiguous. Examples include: good, bad, service, convenient and so on. The following are actual quotes from four different respondents, all of whom are describing the exact same bank. These illustrate both how ambiguous a concept like “convenience” can be and how judicious probing can clarify what the respondent really means. (Note: Probes by the interviewer are in square brackets, respondent definitions of “convenient” are in italics.)

“They’re the largest in our area. They’ve gobbled up every other bank around. They’re convenient, lots of branches.”

“Convenience.” [Meaning what?] “Close to our office. If they weren’t just across the street, I’d switch to the closest one.”

“Convenience.” [Meaning?] “They’re a smaller bank, which makes it more personalized. Yet, they’re big enough to allow me the convenience of getting everything I need from one bank.” [How close are they?] “Oh, not very.”

“I would say convenient because they don’t charge us a fee for our Escrow accounts, which most banks do. So that was another major decision.” [How close are you?] “Oh, we do our banking by phone. I don’t even know where they are.”

While it does take some training, interviewers can and should become sensitive to ambiguous words. When such a word is detected, a simple probe like “What do you mean by (ambiguous word)?” is often all it takes to encourage a respondent to explain what he means. With this clarification, deeper insights are obtained and better business decisions made.

— Mark

Last week we e-mailed each of you an invitation to share your feedback about this e-newsletter. If you have not yet done so, please click the link below to participate in a very short survey we are using to solicit feedback from “Research With A Twist” recipients.

As a “thank you” for a few moments of your time, we will donate $5.00 to the American Red Cross. We very much appreciate your opinions of our newsletter and how we can improve it.

Just click here to share your ideas with us.

“Not everything that can be counted counts and not everything that counts can be counted.”

— Albert Einstein

Questions? Click here to send us an email with your request.
csrinfo@csr-bos.com
www.csr-bos.com
(617) 451-9500

About Us

The Center for Strategy Research, Inc. (CSR) is a research firm. The “Twist” to what we offer is this: We combine open-ended questioning with our proprietary technology to create quantifiable data. As a result our clients gain more actionable and valuable insights from their research efforts.

Post navigation

← Have You Hugged A Survey Participant Lately? In Research, As In Football, Watching Doesn’t Necessarily Mean Understanding →

    RESEARCH WITH A TWIST

    “Research with a Twist” is our monthly newsletter featuring creative and practical insights for research professionals.

    Learn more and read back issues. Or subscribe below.


    By submitting this form, you are consenting to receive marketing emails from: The Center for Strategy Research, Inc., 101 Federal Street, Suite 1900, Boston, MA, 02110, http://www.csr-bos.com. You can revoke your consent to receive emails at any time by using the SafeUnsubscribe® link, found at the bottom of every email. Emails are serviced by Constant Contact

    From our current issue:

  • I love all holidays, and celebrate them to the full extent of the law. Yes, in fact, the law does come into play…" Read on…

Latest newsletters:

  • Thanks-Market Giving-Research
  • Boo!
  • September! Do you remember?
  • What’s in the top of your funnel?
  • Let my research participants go!
  • The Center for Strategy Research, Inc.
  • 101 Federal Street
  • Suite 1900
  • Boston, MA 02110
  • (617) 451-9500
  • csrinfo@csr-bos.com
  • Privacy Policy
  • Site Map