Center for Strategy Research

• 
• Vol. 14, Issue 6, September 2019

Hello!

Happy September to everyone. Yes, Summer is pretty much over, but at least it’s not Winter yet! With that cheerful thought, we hope that you enjoy this month’s installment of Research with a Twist, September! Do you remember?

—Jennifer

September! Do you remember?

Earlier this month, I suddenly had the Earth, Wind & Fire song, “September” in my head as a constant companion. And though I love the song and sing certain parts with absolute conviction, I have no idea what the lyrics are. “DO YOU REMEMBER? The something something… SEPTEMBER.” The song could be about slaying dragons, I literally have no idea.

When conducting research, we frequently ask research participants to draw from their memories. How do we know we are getting an accurate answer to our question, and not the “something something” that has fallen between their proverbial mental cracks? Well, we can’t really know for sure that anyone’s recall is 100% reliable, but we can put some simple guardrails in place to get as close to accurate as we can, including:

1. Recency

When asking a research participant about a purchase decision or about an interaction with your company, it’s important to ask about it as soon as possible after the event has happened. If you had asked me the lyrics to “September” back in 1978, when the song first came out, I would have had a better idea of what the song is actually about, even though I was just a young child at the time.

In this newsletter, we have talked quite a bit about creating and maintaining customer panels, and recency is a key reason for that. With a customer panel, your organization can tap into customers very quickly after events take place that are important to you (you can even track those events!), which will increase the likelihood of accurate recall.

2. Retain interviewers to ask the questions

Another way to make sure that memories are accurate is to have a human being ask the questions. For example, if I took an online survey and one of the questions was, “Do you remember the lyrics to the Earth, Wind & Fire song September?” I would select “yes”, particularly if I knew I could remember some; the “yes” isn’t intended to be misleading, I just don’t want to come across as dumb for forgetting such a popular song.

However, if a person asked me that question, I might say “yes,” but then I’d actually run through the lyrics a bit (maybe even sing them out loud, much to the interviewer’s likely chagrin), then suggest to the interviewer that I would need to temper or change my response.

We see that pattern in our phone interviews all the time. Research participants often answer impulsively one way, but when they say it out loud, they remember that the answer is more nuanced. As we talk more and probe, they tell us about complications, and eventually we get the more accurate response.

3. Make sure the questioner is neutral and unbiased

Whomever interviews or moderates interactions with your customers or prospects should be careful not to ask leading questions. For example, if someone had said to me, “What was the day in September that Earth, Wind & Fire sang about?” I MIGHT have remembered “the 21st day”, but I might have said “25th” or even “Tuesday or Wednesday, I’m not sure.”

But if the interviewer had said, “Is the 21st of September an important date in that song?” I’d have said, “yes, definitely,” even though I wasn’t 100% sure. In the latter case, the research would have gotten the “correct” answer, but it wasn’t MY answer. No matter what, even if it’s muddy and complicated, our marketing should be based on the unbiased truth, or it’s not going to be very effective.

Here’s the Twist: So, apparently, the answer to the classic Earth, Wind & Fire question, “Do you remember?” is complicated! The answer is “no” I don’t remember the lyrics, but “yes” I do remember that people’s memories are fallible (especially among those of us approaching the “September” of our lives!), and market research must account for that!