Qualitative analysis: where do the humans bring value?

Robots replacing humans is a fear that has been increasingly dominant in the media in recent years. The increasing presence of technology in everyday life and the decreasing need for humans in a variety of tasks has led to the fear that humanity will one day be rendered obsolete. Although technology has been present in quantitative data analysis, especially big data analysis, for a number of years, it is now increasingly being used in qualitative data analysis too, because of developments that allow robots to process natural language. Technology can help qualitative research by providing textual analysis, sentiment analysis, image and video analysis and voice recognition, leading to interview analysis, but to what extent can technology take over the role of qualitative researchers? Ultimately, what value do humans bring to the process?

A major strength of technology based qualitative research is the speed. Although humans are fully capable of analyzing qualitative data, they can’t compete in terms of pace. Analysis completed by robots is therefore appealing when clients are looking for quick insights. Moreover, although the initial investment in the required technology is expensive, spending fewer hours on a project will ultimately make projects cheaper, which is always appealing for clients and will give the researcher that uses it an edge over competitors.

However, while robots can now understand language, they are certainly still unable to empathize.  As empathy is crucial for successful market research, technology’s role is therefore limited. In fact, using fast-paced machines to analyze the data is actually helpful, because the less time researchers spend analyzing data themselves, the more time they have to empathize with the respondents’ feelings. It is this “system 2” thinking that ultimately delivers high-quality results to clients because it allows researchers to see the world through the eyes of the consumer. Humans should utilize their strengths as great thinkers and collaborate with machine analysis to achieve the best results.

Moreover, although researchers aim to be objective, using machines reduces the risk of unconscious bias, which is crucial in providing accurate and actionable insights for clients. However, although machines may not have prejudices, their general accuracy can still be questioned. Although their understanding of natural language has improved, it still does not meet human standards and their ability to understand language traits such as humor and sarcasm, while contextualizing meaning, is still limited. Likewise, machines can be over enthusiastic to categorize data when it may not fit under a specific label, a problem that is specifically relevant during sentiment analysis (see our recent blog on this here). Humans on the other hand are more flexible, and can make judgement based on specific situations, rather than making routine decisions based on programming. Admittedly, the sometimes tedious process of data analysis can lead to errors from humans too. But, humans are definitely required in the analysis process, even if merely as quality control.

Although technology has a clear place within qualitative research analysis, we ultimately complete market research, both quantitative and qualitative, to understand people so that clients can meet their requirements with products and services. It’s therefore important that humans remain present in the process and are not distanced from dealing with the data. This is especially crucial because researchers’ knowledge of language, their flexible judgement and their ability to think, feel and empathize adds a value that machines cannot currently provide. When humans and robots work alongside each other, utilizing each other’s strengths, the results will be the most successful and deep, actionable insights will be delivered to clients.

6 Steps to Perfect Qual Series: Step 4 – Qualitative Analysis | New White Paper from L&E

6 Steps to Perfect Qual

White Paper Download

Step 4: Qualitative Analysis

To get to the heart of qualitative analysis, you need to dig deep, as well as scope the bigger picture, to provide actionable insight for the client to address the problem.

This next L&E trust guide sets the scene for the current scale of qualitative data, and offers research consultants a way to grasp the large amount of information by using the latest thinking and technology.

This paper will guide research consultants through the process of qualitative analysis to capitalize on scale, reach, cost, depth and speed. In this paper you will find:

  • Practical tips in categorizing and structuring data
  • The technology used to speed up the analysis stage
  • Support from L&E and technology partners

To continue reading, download our latest white paper by clicking here (no form to fill out).

Webinar | 6 Steps to Perfect Qual- Steps 4 & 5: Qualitative Analysis & Insights Communication

6 Steps to Perfect Qual – Free L&E Webinar Series (available via on-demand)

On June 6, 2018, L&E hosted our third webinar in our “Six Steps to Perfect Qual” webinar series for 2018 with Joan M. Lewis, Shaun Stripling and Walt Barron as our panelists. The focus was on steps 4 & 5: qualitative analysis and insights communications.

Guest Speakers: 

  • Joan M. Lewis, Former P&G Insights Director & Independent Advisor and Consultant 
  • Shaun Stripling, SVP, Chief Strategy Officer and Global Director of Frank About Women at Mullen
  • Walt Barron, Chief Strategy Officer, McKinney

Putting together qualitative analysis and communicating qualitative insight can seem a daunting process, so we wanted to offer you the latest techniques and best practices for telling strong stories and engaging clients with impactful deliverables.

Our third webinar will guide you through the best methods to ask questions and get actionable next steps to lead your clients to true insight. We will discuss the impact of powerful communications through a combination of the latest technology and practical tips for storytelling and creating actionable deliverables.

In this webinar you’ll learn how to:

  • Build impactful communications through storytelling (with practical advice on finding the golden thread in the information)
  • Different ways of communicating insight using technology

Is machine learning getting to grip with sarcasm? Sentiment and Semantics

It’s no secret that qualitative data is harder to analyze than quantitative. The replacement of ticked boxes for a plethora of data from verbatim, videos, group conversations and observations means that participant answers are vastly more time consuming to categorize. Not only this, but the context and motivations of the individuals need to be considered in addition to their responses and behaviors. But the effort is worth it; you can work to really understand consumer behavior and get to the ‘why’ behind the ‘what’.

Of the many techniques used to analyze qualitative information, sentiment analysis and semantic analysis are two we often come across. These provide insight into the data by looking at attitudes and themes. And as new technology continues to make its way into qualitative analysis, we need to understand how it can impact these two methods of analysis, and how it can help develop more meaningful insight.

As a reminder, sentiment analysis categorizes language based on opinions, often checking for a positive or negative viewpoint. This can include, for example, the views of consumers expressed on social media or online communities toward a specific product, idea or service. Although this deep analysis can be done by humans, it is quicker using technology – most often machine learning. Machine learning is helpful in analyzing large quantities of information and making judgements based on language that would be far too time consuming for researchers.

However, it’s important that sentiment analysis is not entirely reliant on technology. We are often asked to analyze data in a binary way, but the expressed sentiment is rarely that black and white. Technology often makes assumptions about the sentiment of a response to make it fit a binary format, while humans would be more cautious to force content into a category. Moreover, while humans can contextualize written language and understand colloquialisms and contradictions, as well as turns of phrase and humor such as sarcasm, machines struggle with this. Machines simply cannot understand all the nuances of language and its grammatical patterns to the same accuracy that a human can. In spite of the claims of some of the technology providers, we are far off being in a place where text can be analyzed by a machine as well as by a human. It is therefore important to use a joint approach to sentiment analysis; machines may make things quicker, but they’re also more likely to create inaccurate data through generalization, lack of contextual knowledge and a lack of natural language skills.

Semantic analysis, on the other hand, generates meaning by grouping together the themes in qualitative data. It is often referred to as topic or keyword analysis. This is an important approach when trying to understand the important subjects in the field of study, regardless of how a participant feels about them. This is often useful in the early stages of a research project, when exploring a broad idea and looking into other relevant and more specific ideas that need to be examined further. As with sentiment analysis, this can easily be done by a computer, and indeed, it is far more efficient to do so, leaving the researcher with more time for deeper analytical thinking. Furthermore, a machine learning process can take place with no prior knowledge of the documents – again saving time. However, there are limitations, as the grouping of themes needs to go beyond basic classification to give a meaningful representation of the qualitative information.

So machine learning speeds up sentiment and semantic analysis; for text based analyses this can open up a whole new world of information about consumers from social media, communities, and other online resources. The tech guys are working hard to solve the nuances of language such as sarcasm – machine learning’s biggest barrier in sentiment analysis. And then with semantic analysis comes the challenge of context – such as the ability to distinguish Mars the chocolate bar from Mars the planet. Technology is certainly getting closer to solving these problems, but in the meantime, it’s nice to keep the humans around.

For more on qualitative analysis, download our white paper here.

Renee’s Takeaways – Get the scoop on L&E’s Qual Research Recruiting Webinar!

6 Steps to Perfect Qual Webinar Series Summary

Step 2 – Qualitative Research Recruiting Webinar – Webinar held on April 11, 2018

Let’s add another webinar filled with great information to the books!  In this webinar in our series, we focused on Qualitative Recruitment and got to spend some time with three experts in our industry.  First up, Teressa Johnnson, Director, Sensory Science and Technical Support at The Wendy’s Company.  Next, Heather Hall, Vice President of Project Management at Illumination Research.  And last but not least, our very own Brett Watkins, CEO and President of L&E Research.

In this webinar, we focused on Q&A based around Qualitative Recruitment, tips to help your recruiting go as smoothly as possible, and also help ward off any issues before they become issues starting with the design process.

Here’s what our panelists had to say!

1. We started by asking about kicking off a new project with a recruiting partner.  What might our panelists be able to share with an amateur researcher in regards to getting recruiting started off on the right foot?

  • Partner, partner, partner! You want to be sure to work with recruiting companies who not only care about helping YOU be successful, but who also align with your core values. In any good relationship, personal or business, communication and honesty are key elements to be able to have a true partnership.   In the end, you’ll want to be sure that you can trust your partner to do what is best for your project.
  • Be clear about your expectations for the recruit right from the start! Have a discussion about who your ideal respondent is. Ask your recruiting partner if they have any past experience with the population you are trying to reach, and if they can offer and advice on the best ways to reach them.
  • Ask your recruiting partner what their plan is to find your population. If plan A does not work, what’s plan B?  Having this mapped out at the beginning, will help keep your recruiting from getting hung up, and keep the process moving along.

2. Next we discussed what might be at the top of our panelist’s checklist when kicking off a new project.

  • Be sure that your recruiting partner is fully aligned with you! Make sure you are crystal clear on who you really want for your research.  This will help eliminate issues down the line.
  • Have a kick-off call! Extremely important to have a kick-off call with your recruiting partner before getting started.  In the age of email, some things may be uncovered in a phone call, that was not communicated in the email thread.  This is a good time for both parties to ask questions, gain clarification, brainstorm, etc.

3. We also touched a little bit on screener development.  What do our panelists always work into their screeners?

  • False, or catch questions. Ask about product usage, and include a few items in a list that do not actually exist.  If a potential respondent answers yes to using any of the false products, TERMINATE!
  • Include your plan B right in the screener! If you anticipate that a question may not work, or may be a sticking point, make your plan B known from the start as to not hold up recruiting.
  • Be careful of the way your questions are worded. It’s easy for a question to be interpreted by different people in different ways.  It’s extremely important to be sure that your questions do not have multiple ways they can be interpreted.  Let someone else on your team put their eyes on it, even take it for a test run if time allows.

4. Touching on recruiting partnerships again, we asked our panelists what some of their must haves are in a recruiting partner.

  • Communication is key. Over communication is even better!  Keeping the client in the loop, and always informed about what is going on with their recruit.  If they are anticipating issues, let the client know, and even come to the table with some suggestions on what can be done to keep the recruit moving right along.
  • Timely updates. A partner who understands that no news makes the client anxious!  All ties in with the above, and if there is an issue, knowing about is sooner than later is best to be able to stay on top of it.

5. Along the same vein, we asked our panelists what the best service is that their recruiting partner can provide.

  • Once a project is kicked off, get started quickly! If recruiting gets started quickly, you can have a gauge early on how the recruiting is going, and be able to identify any issues sooner rather than later.
  • Be honest about feasibility! Work with a partner who does their homework up front, and can let you know, with certainty, if they can recruit the population you are after. If they cannot, they should come to the table with what they CAN do.
  • A solid database. It’s helpful if a recruiting partner’s database holds a lot of preliminary information about potential respondents.  This way, you can focus on the meat of the recruit, and get the demographics and such out of the way up front.

6. One question I think we all ask now and then is how we avoid some of the common issues that may arise with recruiting.  Is there anything that can be done during the design process in order to get the most authentic participants?

  • Blind recruit. Set up your screener to hide the topic, but as mentioned earlier on, include the red herring questions.
  • Re-screen on-site. If someone was not truthful during the screening process, they will likely not remember how they answered if screened again.
  • Work with a recruiting partner who uses technology to track things such as past participation, participant behavior, etc., so you are able to weed out non-qualifying participants early on in the process. Also a partner who is always working to grow their database to ensure fresh participants.

7. New technology and recruiting.  What do our panelists think about it, what do they like, and what are some things that need to be overcome when working with it?

  • Online research. You can reach a lot of people who you may not have been able to reach before, expand your reach. One of the issues with this are bots, and how they can mimic human behavior to answer questions.  Better technology could be put in place to help with that.
  • Webcams can give us some great insights into consumer behavior, and that can work alone as the sole methodology. It’s even better when you can use it prior to groups or interviews to get the conversation started before the respondents even arrive on-site.

We hope you found this summary to be helpful! If you didn’t register for this webinar you can listen to this webinar in its entirety, or download a free, transcribed version by clicking here.

Be on the lookout for our next webinar, which will be this summer. If you can’t wait until then, you can always view our on-demand webinars and learn about the latest technology solutions. Don’t forget to subscribe to our blog so you can keep up with what is happening at L&E!

Until next time!

Your Research Design Engineer at L&E,

Renee Wyckoff

Webinar Recording | 6 Steps to Perfect Qual- Step 2: Qualitative Research Recruitment

6 Steps to Perfect Qual – Free L&E Webinar Series

On April 11, 2018, L&E hosted our second webinar in our “Six Steps to Perfect Qual” webinar series for 2018 with Teressa Johnson, Heather Hall and Brett Watkins as our panelists. The second step in our series focused on qualitative research recruiting and its importance to great qualitative research.

Miss out on this webinar? Not to fear, you can find the on-demand webinar recording here.

And if you haven’t read our next 2 white papers in the 6 step series, you can find those here.

B2B Qual – going beyond in-depth interviews

For the deepest, most comprehensive insights and personal opinions, it is difficult to beat the one-on-one interview.  However, there are other options. Many people assume that B2B qualitative research cannot use the same variety of methods as B2C qual, or even B2B quant for that matter. While it’s true that the in-depth interview is a great way of gaining deep insights into a participant’s thoughts, the vast potential of the internet means there are more convenient ways to reach professionals with tight schedules across varying locations. Moreover, observational methods such as ethnography provide objective viewpoints of businesses that personal interviews may miss.

Admittedly, focus groups are less frequently used in B2B research than they are in B2C because of the time and geographical restrictions of participants. B2B projects often require researchers to reach senior staff members to provide insights on specific issues and these are, of course, the people with the busiest schedules who have little spare time to provide feedback. Moreover, geographical issues mean that even if spare time could be aligned between participants, there would be no convenient place for everyone to meet. This issue can be solved by holding online focus groups so participants can log in to a virtual research space wherever they are. And, to make sure participants join, remember to send a meeting request to participants’ work calendars so the event is firmly scheduled.

Even better, for those with extremely tight schedules, is holding an online discussion group, allowing participants from any location to add to the conversation at a time that suits them. This flexibility is important when respondents have little time to spare. It will allow them to give more thoughtful, meaningful responses without the pressures of time and what is more it means they can talk with each other in a secure and controlled environment. In an increasingly technological world, respondents often feel more comfortable giving their opinions to a faceless researcher online than they would to a real person who they may feel judged by. Another benefit to researchers is that participants who do not engage satisfactorily in the discussion can be replaced, ensuring that productive discussion takes place between the moderators and participants (and the participants and participants) so authentic insight can be gathered. This is also a great method for categorizing participants, as researchers can target specific group members with different questions to develop insight further.

Depending on your research problem, B2B ethnography is also a great method to consider. Although there can be a process involved in recruiting participants, a research technique that simply requires researchers to watch and learn is a great solution for participants with limited time as no changes to their normal habits are required. This technique is excellent for research problems that look to assess behavior in the workplace and to observe the intricate happenings within a business, without asking employees for their own personal opinions which may be wound up in internal politics. Ethnography is therefore a good way to get an objective B2B viewpoint.

So, although interviews are important in qualitative research, there are other, sometimes overlooked, methods which are a great fit for the B2B sector. We all know that working life is busy, especially if you’re the CEO, so resources like online tools are useful for bringing people together regardless of location – with online focus groups providing a universal research location, while online discussion groups allow research on demand for those whose unpredictable schedules mean they can’t commit to a specific time slot. And for objective, unopinionated research, why not try ethnography, to see the background of a business without working around the time restraints of employees.

To access more information on qualitative data collection tools, download our white paper on fieldwork and data collection in qualitative research here.

Renee’s Takeaways – Quarterly Roundup

  1. Methodologies – There is no one-size fits all when it comes to methodology for your research. While focus groups definitely have their place and are necessary, we’re seeing more and more companies using behavioral methodologies such as in-home interviews, or video diaries, etc. to be able to capture those “in the moment” elements that you may not catch otherwise. People tend to act and respond more naturally in their own environment.
  1. Qualitative recruiting – There are steps YOU can take as a researcher to improve the quality of recruiting coming from your partners! It all starts with project design.  Clearly defined research questions, and a clear expectation of what you are hoping to find out from the data that is collected.  This will lead to clearly defined specifications as far as “who” you would like to speak to, and will provide the basis of the questions you will ask in your screener. The more heavy lifting done on the front end, the more it will help ensure truly qualified respondents on the back end.
  1. The future of MR data collection/analysis – Is the future of MR data collection and analysis all about quick and dirty? Yes, to an extent. While it seems that clients are looking to do the research faster, and get the results faster, when it comes to the analysis of the data collected, the human element is still very important in order to be most accurate.  There are ranges of emotion and detail that software platforms are not able to detect and accurately identify.   If that is understood, and taken into consideration, then a fully automated process could work. Otherwise, people power is still needed!
  1. Screener development – As we already know, a streamlined, well written screener is going to produce better respondents! A few things I’ve learned to be effective along the way are the following –
    • Keep it short and sweet! The main goal here is to be sure that the person qualifies for the upcoming research, not to find out all of the information you need to know in advance of the research. Use the screener for the important qualifying questions, dig deeper during your research.
    • Try to ask easy questions that are likely to disqualify early on before getting to the meat. This will help the recruiting process go faster.
    • If there is a typing tool/algorithm that goes along with your screener, suggest to your recruiting partner that they run potential respondents through the tool first. If they are not the desired segment, or one of the desired segments, why continue? Just another way to expedite the process.
  1. A subject not related to MR at all! – Here at L&E, we have an internal learning system which is meant for both professional and personal enrichment alike. When I come to a module that I particularly enjoy, I like to share it! This particular module was about happiness.  It is largely thought that success will lead to happiness, when it has been studied that it is actually the other way around.  Happy people are more productive, more creative, and able to have richer, more meaningful relationships! These things are successes in and of themselves, and there are several small things one can do to inspire happiness in themselves and in others.  Just a few of my favorites include the following – 1) Express gratitude. Take a few minutes each day to think about a few things you are grateful for.  2) Practice random acts of kindness.  Do something nice for someone, it makes them feel good, and you feel even better! 3) Appreciate the little things. Don’t forget to stop and smell the roses!

Until next time!

Your Research Design Engineer at L&E,

Renee Wyckoff

Video open-ends – think outside the box

Consumers have a lot of opinions and valuable advice to offer, but how can we ensure they’re heard?

Video puts data collection tools right into the hands of consumers. It does a great job in engaging research participants so that they know they are really being listened to. It puts them center stage, making them feel their worth and their value. With video open ends, participants aren’t constrained by text box responses, or even worse, a pre-written answer key. They are able to elaborate on their thoughts and opinions, while conveying their true emotions and feelings. And this is all done much quicker and easier, and on a more personal level, than typing in a box.

For researchers, video analysis means accessing deep insight from responses that are much richer than reading numbers or text alone. In fact, text box open-ends on average only get three or four word responses. While video gives researchers access to six times more content.

But although video brings a lot to the table, there’s still some confusion around how researchers can fully harness the value of video open ends.

Open-ended video questions can seem out of reach, especially for independent research consultants. But as a powerful source of insight, they’re well worth it. After all, it’s our jobs as researchers to take advantage of every technology and every resource available so as to access the raw emotions and the complex opinions of our participants, to elicit deeper insight to share with our clients.

So, how can you make video work for your research project?

Add video to current methodologies. Video works very well with in the world of qual, but it also applies to quant projects. Video helps to marry the two for richer insight, to let consumers tell the story behind the data. Video can be added to online surveys, NPS studies, online communities, brand trackers, ad testing, focus groups, ethnographies and more.

Take advantage of mobile. Mobile phones and laptops can easily capture video from anywhere in the world, at any time. Mobile is giving consumers 24/7 access to sharing their views with brands. Researchers and brands can reach out to consumers at any time with ‘push’ content to solicit video feedback in the moment.

Show and tell. Video is a great way to put products in front of participants. It can help the participants understand how a new product is used in a certain setting, and how it will be positioned, in order to find out how the messaging and language will resonate with target consumers.

Put it into context. Video should be used to go into greater depth, to get the fuller story, and to put everything into context for the researcher. A video can show the researcher how the participant uses the product in their home, or what products fill the shelves at the store.

Feelings. The facial expressions and the pure emotion that is picked up in a video within ten seconds can say much more than the typical text box response of 50 characters. Make sure the questions are designed to explore the participant’s feelings and emotions, and to probe their opinions.

Don’t settle on quality. The quality of the video software used is important as it can affect both the participant’s experience and the insight collected. Poor quality video can negatively impact the quality of the insight, so if you can control the video quality, so if you are using the software in a survey for example, the video must be of a standard that doesn’t distract or negatively influence the participants.

It’s easy to dismiss the video trend in research as a tool that is out of reach and too complex to integrate into current methodologies, but there are a lot of partners and tools out there to help. If you’re looking for more on video and other data collection tools, download our white paper on fieldwork and data collection in qualitative research here.

6 Steps to Perfect Qual Series: Step 3 – Fieldwork & Data Collection | New White Paper from L&E

6 Steps to Perfect Qual

White Paper Download

Step 3: Fieldwork & Data Collection

When you back your fieldwork and data collection with the latest technology, you can’t go wrong.

Download the latest L&E trust guide to access new (and go-to) qualitative research techniques, along with the in-the-moment tools, video analytics, and other technology they run on. In this paper we’ll also share practical tips in choosing the right facility and creating an optimal research environment for participants.

This paper includes:

  • Emerging fieldwork techniques (and the technology they run on)
  • Practical tips on making your participants and clients comfortable on site
  • How to deliver an environment that will optimize data collection
  • Strategy for choosing the right facility

To continue reading, download our latest white paper by clicking here (no form to fill out).