Internal Market Research – Leveraging the Wisdom of Your Company Employees

In the search for market research information companies sometimes overlook an obvious, knowledgeable and easy to recruit resource… their own employees.

Sometimes the opportunity is obvious. For example, to assess the effectiveness of a sales rollout for a new flagship product I conducted internal telephone focus groups with the company’s sales representatives. Once confidentiality was established (this is one study where the “client”, meaning management, did not listen in), the participants were quite forthcoming about what was working and what was problematic in terms of the support provided by the company. The group format also helped to solidify the commitment of the sales team to work together for a higher level of success.

The sales representatives at another firm were interviewed individually to increase understanding regarding the reasons behind the outcome of certain large sales campaigns in varying states. This type of “win-loss” research compares factors contributing to both outcomes and helps to set priorities for future campaigns.

At other times the potential advantage of surveying company employees may be less obvious. For example, a belief in the value of the “cross-pollination” of ideas led to a series of informal brainstorming focus groups during the early stages of the development of a new product. Employees from the same company who were creating different products for the same market met in a series of groups. Beneficial ideas were generated for the new product that were also applied those already under development. An increase in the level of consistency in the company’s product line was an unexpected benefit of the research.

Here are several important things to consider when conducting market research with employees:

1. Assume Bias – Be aware that the findings from internal market research are biased by an above average level of knowledge and personal investment in certain ideas. For this reason, internal research should be balanced by inquiries with other audiences to help paint a more accurate picture.
2. Ensure Confidentiality – Select the best methodology to encourage disclosure. Ask yourself if your participants are best served by groups, interviews or some anonymous methodology.  Also, determine who can observe and how the findings will be represented in the report to help participants to feel safe.
3. Be Constructive – Establish an objective to make things better for all concerned. Guide the inquiry in ways that uncover challenges, identify solutions and avoid blame.

The next time you are brainstorming about the details of a new market research project consider whether or not the people at your company can help. You may find that formalizing an internal information gathering process can yield valuable insights.

60,000 emails yields (Guess… how many?) respondents

Two recent online studies needed to be completed quickly. Because they required larger numbers of participants than usual and time was extremely limited, they were recruited via email blasts. The two projects involved similar participants but were very different in methodology. One was a 15-minute online survey and the other a 90-minute online journal/bulletin board. Each used its own database of 12,000 potential participants and each offered what was considered a “reasonable” cash incentive.

The goal for each study was roughly 50 participants. Do you think that multiple email blasts to 12,000 potential participants for each study fulfilled either goal?

Whether you voted yes or no you are right. The short survey met the goal but the more involved online bulletin board study only resulted in 30 completions. The differences between the two studies were instructive. Here is some data to consider.

	Study 1	Study 2
Total recruited	50-participants	30-participants
Number of blasts	3 blasts to 12,000 addresses	2 blasts to 12,000 addresses
Length of research process	15-minutes	90-minutes
Amount of initial incentive	$20	$75
Amount of increased incentive	$35	$100

Note that both studies required multiple email blasts. Each blast generated activity for about 24 hours, after which the number of participants clicking the invitation slowed dramatically.

Also note that the incentive for both studies was increased when it became clear that the original amount offered was not compelling enough to generate the desired result.

What did I learn? Well, this is what I will do differently next time:

1. Incentivise generously rather than reasonably. Each email blast costs over $1,000. Increasing the incentive from a reasonable to a generous rate costs about half this amount. If the higher incentive results in a full study with fewer email blasts, then I have saved money and I have happier and more motivated participants.
2. Blast more participants the first time. I did not choose to use the entire available universe of participants because I thought that 12,000 would be more than sufficient. I assumed that at least one-percent would click the invitation and that about half of them would qualify. In reality a much higher number “clicked in” but a majority of them never actually began answering the qualifying questions or abandoned the process after answering the first few. Based on results, a more realistic assumption might be that .05% will be motivated enough to complete the qualifying questions.  Of course the incidence of those who answer all the questions and actually qualify will depend on the nature of the questions.
3. Conduct a test blast. A message sent to a small number of participants would help to gauge the likely incidence when the same offer is blasted to the full sample. This type of test, which would take roughly 24 hours, would enable me to more accurately gauge the right number of potentials to include and the amount of incentive to offer to fulfill my goal with a single blast.

Online Learning – Pros and Cons

Many of my clients are textbook publishers and all of them are increasingly including technology as part of their programs. Digital versions of the textbooks along with supplementary websites for students and teachers are now expected. Most of the work I am currently doing relates to developing the technology side of their programs.

This NY Times article focuses on the Khan Academy and does a good job presenting both the pros and cons of their particular online approach toward teaching math. It also points out that all of the Khan Academy videos are free, which is something that is problematic for my clients to emulate with their for profit business model.

The article predicts that there will be many more imitators of the Khan Academy in the immediate future. It will be important for my clients to monitor what schools can get for free and to ensure that their  products deliver sufficient value to be purchased.

Looks like “interesting” times are ahead.

Beware of Smartphone Interface Rage!

I laughed hard after reading today’s Dilbert cartoon. I have “been there and done that” (or at least my own version of the final panel) more times than I care to admit.

The cartoon speaks for itself… unfortunately.

Smart Phone Research with Addicts has Market Research Implications

This NPR story has striking implications for market research and provides credibility regarding the role that smart phones might play.

The story describes how a methadone research program in Baltimore issues smart phones to the addicts they serve in order to gain insight into the causes of relapse. The participants are instructed to call-in and complete a survey when they are craving drugs. Sometimes this occurs prior to a relapse and other times afterwards. Regardless, participants login on their smart phones and answer some critical questions related to how they are feeling and behaving. Additionally, their geographic locations are being tracked via GPS loggers that are part of the phones.

Additionally, randomly, three times a day, participants are contacted on their smartphones and they respond to several questions.

In this way the study is gathering data about what triggers each individual participant and cumulatively what geographic parts of the city are potentially most toxic to addicts.

Several elements of the story stand out in my mind as relevant to market research.

1. The story explains that the study was conceived as a way to gather real-time data to surmount the limitations of “recall-bias” which is something that anyone familiar with market research can relate to and which applies to humans universally, not just addicts.
2. The methodology used two-way communication which increased the likelihood of receiving “real time” data. At a minimum, participants responded to the message sent to them three times a day. Additionally, if they were conscientious enough to call in at the time that cravings began, this data was even more relevant and timely.
3. Participants were rewarded for participating (they stayed in the methadone program) and not punished for relapses. Although, they were held accountable for keeping their smart phones and participating in the research.
4. The study generates “behavioral geography” data. It suggests that if a map can be created to chart the geographical danger zones for addicts in a given city that other “maps” might effectively predict how certain types of people will behave in specific environments.

I continue to be fascinated by the possibilities of smart phone research. Here are links to a couple of other related blog posts: Real-Time” Qualitative in Three Not-So-Simple Steps and Using Smart Phones for Market Research.

Exploratory Research – Using Four-Hour Focus Groups

I recently conducted a series of four-hour long, live focus groups. It has been some time since one of my studies required such a lengthy investigation and I have both the benefits and challenges fresh in my mind.

Each night during introductions I joked with the participants by asking “what were you thinking when you signed up for a four-hour group?”

If the question were turned back to me, this is how I would reply:

“I have a wide range of materials I want to explore with you, including a few that will require some time for you to properly evaluate. Fully benefiting from this detailed review will require an in-depth discussion of your perceptions, including speculation about your future needs. Four-hours may seem like a long time but to complete our agenda we will actually need to stay focused and work efficiently.”

This “reply” to my own question points toward the approach taken to “explore” a host of relevant issues to ensure that my client’s new product is launched on an accurate trajectory.

To further increase the likelihood of gathering deep and accurate data I recruited “veterans”, meaning participants with more years of experience in their profession than I usually require.

Despite the length of the group the evening felt fast-paced and participants commented that it seemed to go by quickly. In the end the data was rich and deep and according to my client, well worth the investment of time and money.

The challenges of such lengthy groups are mostly logistical, such as gathering the needed materials, planning the presentations, orchestrating the many transitions and incorporating time for breaks and food efficiently.

Coordinating the creation of a discussion guide, PowerPoint presentation and a variety of note-taking and survey “worksheets” took a lot longer than a typical focus group. However, the time investment paid off in the form of smoothly running groups that covered all of the material without feeling rushed.

I think that four hours is about the limit for a focus group format. I don’t think that anyone; participants, observers or the moderator can effectively sustain a useful “focus” for much longer. Plus, when your total time frame requires multiple breaks and more than one meal it seems wise to move to a half or full-day meeting format.

Twitter and Market Research

I loved this cartoon from the most recent issue of the New Yorker! If you are not familiar enough with Twitter to know; a hashtag allows all of the folks using it to tweet with each other in a sort of private texting conversation. The cartoon is suggesting that the funeral attendees may want to have a background chat on their smart phones during the service. Sounds like fun, don’t you think

To check out a business use of the hashtag, search Twitter for the hashtag #brandchat  where a hosted “discussion” is conducted on a weekly basis with anyone who cares to participate. Alternatively, you can see a description of brandchat on this website.

I have explored running market research discussions on Twitter by using hashtags or by using some of the associated third-party applications that allow a person to moderate groups via Twitter in a more private “setting”. To date, I have always chosen to revert to the hosted bulletin board type services due to security concerns and the hassle to participants related to establishing a Twitter identity.

However, should a study arise that might benefit from public exposure or a wide base of participants that do not require careful screening I believe that Twitter could be a great tool.

I hope that we don’t get to the point that people feel the need to background chat at a funeral. However, there are many events where background chat via Twitter or some of the new market research hosting services might provide useful market research information.

I often think about these possibilities… for related information:

• this post discusses the advantages of using smartphones for market research and
• this one speculates on the possibilities of gathering information from captive audiences

If you have a need for market research information that might be fulfilled through the creative use of social media send me a Tweet… or better yet, let’s talk on the phone. Did you know that you can make phone calls with that texting tool that you carry around with you? It’s true, voice to voice in real-time. Its revolutionary!

Market Research’s Role During an Early Drug Shortage Crisis

Recently we have seen a plethora of news stories about drug shortages. This Storify provides a collection of reports, tweets and videos related to a workshop on drug shortages that the Food and Drug Administration convened last week. (A previous blog post discussed how Storify enables reporting by linking all types of items from various media.)

Some blame the shortages on drug companies that have stopped making certain drugs because they are unprofitable. But, according to this NPR report, officials at the Food and Drug Administration say only 11 percent of shortages happen because a company decides to stop making an unprofitable drug.  Most shortages, they say, occur because something goes wrong in the manufacturing process that halts production.

One of the first manufacturing process related drug shortages occurred in 2002 to one of my clients. It was a Murphy’s Law type of situation where failed batches of a hard to produce new biotech drug limited supply during a time when its use was increasing exponentially.

There was no quick fix for this dilemma, so in addition to fast-tracking solutions for their manufacturing problems, the company tested the responses of patients and physicians to a range of communications describing several possible programs to help manage the fair distribution of the drug until adequate supplies were reliably available.

Due to the newness of this issue at that time and the potentially profound impact of the drug shortage on both established and prospective patients, the interviews were both sensitive and interesting to conduct. I would not be surprised if the circumstances of this shortage were studied in business schools or if the lessons learned were instructive to the next generation of biotech companies, who mostly have done a better job of ramping up their manufacturing capability for the blockbuster drugs that followed.

Gathering Ongoing Feedback from Small Expert “Panels”

My clients are demonstrating an increasing interest in gathering feedback from customers in between the usual phases of exploratory and prototype testing research. They explain that they have a wide range of issues about which they desire quick feedback to help guide their decisions.  I believe that small panels comprised of “expert” customers are a great way to fulfill this need.

The small expert panels proposed in this article are totally different from the massive online panels gaining prominence in the market research industry. The ability to broadcast surveys and other types of market research instruments to millions of “panelists” have proven useful to me for some very specific needs. However, I believe that a panel of up to a dozen specially recruited “experts” is a much more targeted and useful tool for product development.

Throughout the 90’s while online methodologies were still unsophisticated I frequently conducted live panel discussions. Expert participants typically gathered at a hotel conference room for a full day meeting. Participants and clients sat in the same room and at times interacted directly with each other. A typical day involved discussing current practices, reviewing materials and/or using the prototype and then brainstorming on solutions to guide the next steps of the development team. This description is pretty generic because I have successfully used this model with both educational publishers and pharmaceutical companies. It truly has wide-ranging application that is relevant to this day.

Additionally, clients sometimes maintained small panels of expert participants who resided near their home office. Some conducted a standing biweekly or monthly meeting with their panelists while others called meetings as needed when the input of the panel would aid their development process.

The availability of online videoconferencing and the ubiquitous nature of high-speed internet access opened many more possibilities for gathering information from expert panelists either in groups or through individual interviews. Online methods also transcend geography and enable the creation of multiple regional panels for products that are customized for different markets, such as textbook programs.

Next Step Consulting has conducted online panels using the following online methods, which are listed in order according to their frequency of use:

• Application or document sharing technologies (i.e. Webex) to support in-depth interviews with individuals or groups of up to six participants
• Online bulletin boards with groups or online journals with individuals
• “Talking head” technologies (real-time video head shots of each participant) to support in-depth interviews with individuals or with small groups
• Asynchronous video technologies (panelists make and then submit videos as feedback)
• Document editing and annotation technologies (participants add comments and annotations as they respond to documents, concepts, images etc… see this earlier post)

Regardless of the specific methodology chosen to interact with the panel, here are two very important caveats about the ongoing use of small expert panels to aid product development:

1. Panels do NOT reliably replace other forms of market research. It is wise to regularly double-check information generated from panels using your usual market research techniques.
• Recognize that panels, as they are defined in this post, are comprised of a small sample of opinionated and exceptional participants. They provide valuable information but it MAY NOT necessarily align exactly to the perceptions of the universe of “regular” customers.
2. Panelists lose their ability to critically evaluate a prototype that becomes familiar. After a few rounds it is wise to replace “veteran” panelists with promising new recruits.
• Repeated use of a prototype product tends to generate loyalty on the part of participants and acceptance of its idiosyncrasies. Additionally, once panelists have an emotional or intellectual investment in the prototype (it reflects their personal feedback in some manner) they begin demonstrating a level of attachment and bias that makes their feedback less useful.

There is much more to know than can be covered in one post regarding the best ways to establish, manage and run these types of panels. Hopefully, this information explains the basics in terms of the forms they may take and the role they may play in product development.

End Malaria – Buy this Book (Unless I Worked for You This Year!)

I just found my holiday present for my clients who were active this year. To some I am known as the chocolate covered cherry guy. However, when I am inspired I offer gifts that do good for others.

Seth Godin has inspired me again with this offer. A compilation of ideas on “doing good work” from roughly 60 authors of business-related titles.

FOR EACH BOOK PURCHASED $20 IS DONATED TO MALARIA NO MORE!

Love kids? You can help save them by purchasing this book. I am purchasing a few copies today because it is officially End Malaria Day.

Please join me.