I plan on reading this report at least a few more times to develop an understanding of the role Next Step Consulting plays throughout this evolution. Overall, I am confident that companies will always need to understand the needs and desires of their customers while conceiving new products, as well as testing their “user-friendliness” while they are being developed. I believe that big data will help to identify general needs and directions for new products but expect that qualitative research will continue to be the best tool to flesh out the details indicated by the statistical trends.

I estimate that currently at least 80 percent of my projects relate to new product development and initial marketing preparations. But once a product is established in the market, is there a role for qualitative market research throughout the process of “on-demand” marketing as envisioned by McKinsey & Company? From my perspective I see many opportunities related to gauging the nature of each touch point. I know that I receive far more survey requests than I used to after I make a purchase or interact with customer support personnel. My guess is that this type of immediate, ongoing and relatively inexpensive type of testing will be the norm of the future. I would also guess that online communities and panels might become increasingly important as companies follow the mood, needs and desires of their “tribes”.

Of course, time will tell. The report also emphasizes the need for consumers to trust and cooperate with companies in terms of providing and allowing the use of personal data. No doubt there will be a direct link between the need for qualitative research and the quality and reliability of the information gleaned from big data.

In this first clip the interviewer asks questions related to the presidential inauguration and in this second clip she asks questions related to the Oscars. People “on-the-street” answer the interviewer’s questions about these events, decisively respond to her probes and also add elaborate detail. However, the questions were asked days before these two events actually happened. In other words, the people in these videos are completely making it up.

The ridiculous nature of the interviewer’s probes makes for good entertainment but leaves me somewhat incredulous regarding the outrageous lies these folks are telling. I wonder how many times a participant in one of my studies was acting similarly. I feel horrified while witnessing the power of a well-executed leading question. In the extreme I wonder about the validity of the whole model of interviewing respondents or even worse, conducting groups with participants and relying on what they say.

Now… to be fair, there is a lot I don’t know about how these interviews are conducted. One would suspect that folks are selected based on their likelihood to  respond inappropriately and encouraged to behave outrageously since they are generating data for entertainment, not market research. I also wonder about the incidence of folks who respond by lying and agreeing with everything the interviewer says. How many interviews do you think they have to conduct to get the 5-10 clips that they include in a single segment of the show? If, like me, you basically trust human beings to be truthful, you would hope that the incidence of outrageous clips  is lower, rather than higher. (I asked a question about this through the shows website, and if I get an answer I will report it.)

I chose the above two clips because they do show the flip side of this entertainment stunt. In the first clip a woman admits that she is lying and just going along with everything the interviewer says. In the second someone responds accurately that the Oscars have not yet happened and despite being drunk, sticks to her story.

Ultimately, I am not really worried about the participants in my studies behaving similar to those in these two videos. However, watching Lie Witness News makes me even more aware of the importance of following basic market research principles such as properly setting up a group or interview, avoiding direct and leading questions, and examining data in a discriminating manner that draws inferences more than relying only on what people say.

To start with the positive, the automobile manufacturing industry is pretty much a testament to the ability of car makers to fathom and respond effectively to the desires of drivers with ever increasing levels of refinement. I just brought my older Prius to my local dealer for a recall repair. In the spirit of turning lemons into lemonade all was handled beautifully making the chore of delivering and being without my car for a day as easy as possible. Plus, I enjoyed learning about the new features of the newer Prius models during the short time I spent in the shop while drinking their excellent coffee and waiting for my shuttle ride back to my office. In a surprisingly no pressure manner a salesman briefed me on the differences, leaving me with the impression that there was a model for just about every type of car owner as well as some really cool new features I would enjoy using.

In contrast, the ending of the last episode of season three of Downton Abbey is an example of an obvious lack of market research. (Spoiler alert… if you do not want to know what happens in the final moments of this last season’s episode do not read further.)

If you are not among the millions familiar with the story you need to know that in the last few minutes of the show one of the key characters dies in an automobile crash. For many reasons this plot turn seems random, illogical, cruel and redundant to the existing story line, not to mention counterproductive to developing the story in coming seasons. It also totally altered the mood of the story they had built to that point from sweet to tragic.

A Google search shows a long list of articles about the outraged reaction from viewers with headlines such as this one “Is it time Downton died and went to TV heaven?”, which pretty much reflects my feelings immediately after the show ended. Apparently the negative reaction was strong enough that the same search also shows explanations from the producers and from the key actor involved. These explanations tell me that the character Matthew had to die because the actor, Dan Stevens, was not willing to sign a contract for the next season. If only they had put a similar level of creativity into addressing this situation as they did to the many other aspects of the show.

The Huffington Post’s one question survey asking:  “Is Matthew’s death good for “Downton Abbey”? is currently running at about 83% agreeing that it is not a good thing. Surely, a proper market research effort would have uncovered the likelihood of this outcome and identified better alternatives prior to alienating a large part of the audience.

The moral of this post is that when business circumstances surprise you test your response options so that you don’t disappoint your “viewers” or customers or clients with your own unstudied and unpleasant “surprise”.

• Ideal
• Average
• Outlier

These are useful categories to consider when designing a market research project.

Most qualitative market research projects strive to recruit average participants assuming that their opinions and experiences represent the largest number of users of a product or service. I believe that most of the time this is an effective approach.

However, at certain times I have sought out more ideal participants. For example, interviewing thought leaders sometimes helps to predict future trends. This post describes some important things to consider when conducting this type of research. Also, recruiting a small segment of ideal users in usability testing allows me to compare their usability experiences to those of more average users.

I do not recall a time that I intentionally recruited outliers for a study. However, occasionally they show up accidentally, due to the vagaries of the recruitment process. Most of the time I dismiss these participants when it becomes clear that they are not representative of the average. However, I recall completing some interviews with outliers that provided interesting contrast to the practices of the other participants and helped my clients to understand the perspective of their fringe customers.

Bottom line, there are no rigid rules governing how to populate a study except to select the mix of participants that best fulfill the research objectives.

Recent conversations with some of my more experienced and brilliant clients highlight several mistakes that they see occurring on an increasing basis that illustrate the dangers of conducting research that is poorly conceived and executed.

I believe that one of the most common causes of these mistakes is a lack of “band-width” on the part of my corporate clients. It is important to note that this is not their fault. Almost all of my active clients find themselves in a situation where downsizing and budget cuts have increased their workloads to the extent that it seems impossible to give any one aspect of their jobs the attention it deserves. Add budget cuts and tighter development timelines to the mix and you greatly increase the likelihood of making one of the cited mistakes and generating “dangerous” market research findings.

Unfortunately, some respond to this dilemma by completely eliminating market research and developing new products without the guidance of ongoing customer feedback. I believe that this is a risky proposition on many levels.

My proposed solution is to hire a market research professional who can ensure that each study gets the attention it deserves. This previous post entitled, Is your market research firm SMALL enough to “DO IT ALL”? speaks to this issue.

“I think that it is important to say that many projects mistakenly begin with a consideration of research methods rather than an adequate analysis of the research objectives. I believe that this often results in bad research where you get the answers that are possible with the predetermined methodology but do not get to the heart of the matter, meaning what the client needed to know to take action. For this reason I strongly encourage my clients to come to me prepared to talk about what they want to learn, rather than how they want to get the information.”

That said… upon consideration of my list of “methodologies” I have come to the realization that qualitative market research really comes down to how many people you communicate with at a time. Working upwards you have:

Once you have determined the ideal number of people to speak with at a time, you have multiple choices about how to actually engage with them. This brings into play many new online tools, which is what I believe people often mean when they reference new qualitative market research methodologies.

Here some of the options and tools that I am aware of

So, bottom line, I believe that the methodologies of qualitative research currently are very much the same as they have always been but that the number of tools that potentially help to execute these methodologies provide an increasingly richer assortment of choices when considering how to best achieve the objectives of a given study.

The posts that follow, all originating from the beginning of this blog, still provide what I consider to be useful insight into how to begin a study. I hope that they serve you well.

• Defining Deep & Detailed Study Objectives
• Comparing the Primary Qualitative Market Research Methodologies
• Selecting the Best Qualitative Methodologies
• “Real-Time” Qualitative in Three Not-So-Simple Steps

Technology is now providing many ways to measure people’s actual actions in ways that are changing the nature of market research. “Big data”, the tracking of our actions, such as where we click, the things we purchase, where we go and what we watch on television can be analyzed directly to produce insights, or can be used as a tool to facilitate an inquiry.

I would presume that the increased ability to accurately quantify the actual actions of a segment of individuals has reduced the amount of certain types of traditional qualitative market research that is being conducted. Why ask people about their practices when you can view data showing what they actually do?

I also see ways that qualitative market research can take advantage of “big data” and expose a deeper level of insight related to why individuals are taking specific actions. Most critically I think that big data can assist to identify the most important individuals to participate in a qualitative study.

Here are some ideas:

• Use big data to help identify and then recruit different types of customers. Most obviously contrast the opinions of loyal customers to those who recently abandoned your brand and purchased the product of a competitor.
• Compare  the experiences of first time customers who purchased a product from your website, or who “liked” your company, with those who visited your website and did not make a purchase or interact in any way.
• Interview those who have commented on your company’s services through social media, perhaps comparing those with positive, neutral and negative reviews.

According to a report from SAS entitled Big Data: Lessons from the Leaders, many companies are collecting big data but are challenged to isolate what is meaningful and to gain insight from it. I believe that qualitative market research can help by finding the right folks and interacting with them in strategic ways to answer the most relevant questions related to improving marketing results.

The sequential nature of participant’s responses in online bulletin boards is always problematic. The first participant to answer a discussion question has no previous posts to respond to. The second can only respond to the posts of the first. In fact, the key issues that are relevant to the online discussion do not really become evident until about half the participants have weighed in on a question. This is a little late for moderators to intervene and focus the discussion because only the remaining half of the participants in the conversation have the opportunity to really discuss these issues.

To address this problem I structured a recent bulletin board in two phases.

1. In the first phase, participants answered roughly 20 questions in a “blind” manner. Essentially, completing an online journal, without seeing the posts of their peers or interacting in any way.
2. In the second phase they “discussed” about five of these questions.

The online hosting service I use (Focus Forums) enables me to design very detailed discussion guides and to control the level of “discussion” on each question.  Some of the phase one questions gathered data with ratings or addressed the specific practices of participants that were unique to their market and not universal.  Realistically, many of these questions would not benefit from any type of “discussion”.

The phase two questions were only those that were truly suitable for discussion. Every participant had already made a “blind” response to these questions before Phase 2 began. As the moderator, I was able to read all of these responses and insert specific probes to focus the discussion before it began. The first Phase 2 participant had the opportunity to read and reply to the initial posts of every other respondent in the study, as well as my additional queries.

The study I am referencing was a small one due to the uniqueness of the 22 participants. The final transcript included roughly 1,200 entries. On average, each participant posted over 50 times! More importantly, the depth and quality of the discussion was the best of any of the online bulletin boards I have conducted. I believe that the described two-phase structure is the reason.

There are three main reasons for conducting a dynamic debrief:

• Time pressure – A dynamic debrief is usually conducted on the day following the completion of the field component of the research and results in a list of the key findings and agreed upon next action steps.
• Need for consensus – The dynamic debrief process generates a deeper level of interactivity among team members than may be typical. If a study is complex, then the interactivity this process demands often helps to get to the bottom line.
• Desire for an energizing “jump start” for a project – Participants usually leave these sessions with a clear list of next steps that they are excited to begin.

There are three main steps to a dynamic debrief. Here is the shortcut formula:

• What? – Identify the key research findings
• So What? – Generate recommendations to address the findings
• Now What? – Prioritize and refine the recommendations into “next steps”

Here is a brief overview of each step:

Step 1 – What?

Unless it is patently obvious, it is smart to begin with an overview of the study objectives. If the team leader outlines the history of the project and the desired outcome of the session the team members tend to pay attention and hold the focus as the process advances.

Participants begin working individually by posting their “findings”. Imagine a team of people circling a large conference room, stopping at each of 5 to 10 “stations”, writing short statements about “what they heard” on large post-its and placing them on flipcharts labeled with the main research questions. Eventually, everyone has posted their findings to every flipchart or station. These findings are then organized by small groups into a few key themes.

Step 2 – So What?

Participants divide into teams and generate recommendations to address the research findings at each of the stations. After posting their recommendations, the teams rotate through each of the stations and expand on the recommendations made so far by adding their ideas or comments.

Step 3 – Now What?

Participants are each provided with a limited number of dots and instructed to “vote” for the recommendations they regard as most critical and important. To conclude the process, the whole group assembles at each station and discusses the highest ranked recommendations, defining and agreeing on the next steps.

Output

Participants receive a table outlining the key findings and the recommended next step actions in order of importance.

Some Final Thoughts

• This post outlines the basics of the dynamic debrief process but leaves out many important details.
• There are many possible variations to finely tune the process to match the idiosyncrasies of a given study or team.
• With simple studies the process can be completed in a half-day. More complex studies may benefit from a full day.
• Running an effective dynamic debrief requires careful planning and is definitely more complicated and demanding than running a typical focus group.
• Recent developments in online group brainstorming tools inspire me to attempt to run this process in a paperless and perhaps a remote fashion.

I believe that conducting a dynamic debrief merits the investment if time is tight, group interaction is critical to effective decision-making and the team will benefit from an energizing “jump start”. As you might imagine, participants leave a dynamic debrief with a level of clarity about the study results and what they are going to do about it, that is much deeper and more practical than the “usual” experience of reading a report a couple of weeks after completion. Most teams also find the process to be fun and engaging, and the outcome to be effective.