Latest / The Insurance Marketer / Navigating Research in Insurance w/ Debbie Maas & Amy Rey
Transcript
- 0:00SPEAKER_00: From the OneFire studio, welcome to the IMCA Peer-to-Peer Podcast, where we dive deep into technology, brand, social media, and other topics with an insurance industry expert.
- 0:13SPEAKER_00: Let's welcome today's guest.
- 0:15SPEAKER_01: Hi there, this is Jim Flynn from Onefire, along with my co-host, Leslie Castillo from American Equity Underwriters.
- 0:22SPEAKER_01: And it's our pleasure to welcome you to today's episode of the IMCA Peer-to-Peer Podcast, where today we'll be talking about the key differences between qualitative and quantitative research methodologies, a favorite of mine.
- 0:38SPEAKER_01: And with uh and when each of those methodologies is appropriate.
- 0:46SPEAKER_01: And to help us better understand that journey, our duo of guests today are Debbie Moss and Amy Ray.
- 0:52SPEAKER_01: Debbie is the senior vice president of communications and client engagement at Fusion.
- 0:57SPEAKER_01: As a member of Fusion's senior team, she focuses on communications, business development, marketing, customer service, and strategic planning.
- 1:06SPEAKER_01: She is committed to making a positive impact on carrier clients and their policyholders.
- 1:12SPEAKER_01: Debbie has used various research tools to obtain input from policyholders and industry leaders on various topics.
- 1:18SPEAKER_01: She is on the board of the Long Term Care International Forum, is a member of the International Claims Association, Life and Long-Term Care Committee, and serves as a chair committee on leadership for women's organization.
- 1:34SPEAKER_01: Amy Ray is the managing director of growth and strategy at Zeltis Research.
- 1:39SPEAKER_01: She is a seasoned market research professional whose primary focus lies on financial planning, investments, and insurance, from retirement plans and annuities to life insurance, long-term care, and everything in between.
- 1:52SPEAKER_01: That is the real estate that she's involved in.
- 1:55SPEAKER_01: She has expertise in a wide variety of research engagements, including loyalty and satisfaction, customer journey, message and concept testing, market segmentation and assessment.
- 2:05SPEAKER_01: She has experience with qualitative and quantitative methodologies with consumers and B2B audiences.
- 2:12SPEAKER_01: She's an experienced moderator of in-depth interviews and a focus group moderator as well with consumers, advisory groups, agents, and other audiences.
- 2:22SPEAKER_01: Welcome to the show, ladies.
- 2:24SPEAKER_01: We're excited to get started and dive into some deep research stuff.
- 2:28SPEAKER_04: In summary, they are very smart people with us today.
- 2:33SPEAKER_04: Yeah.
- 2:34SPEAKER_04: So no pressure on us to ask smart questions.
- 2:37SPEAKER_04: We will definitely get smart answers.
- 2:40SPEAKER_04: Okay.
- 2:41SPEAKER_04: So before we get started, uh, where do you both hail from today?
- 2:45SPEAKER_04: And where is home originally?
- 2:47SPEAKER_05: Um this is Amy.
- 2:49SPEAKER_05: I am a Jersey girl born and bred.
- 2:51SPEAKER_05: So I have lived born in New Jersey, lived in New Jersey my whole life besides four years of undergraduate.
- 2:57SPEAKER_05: Um, and I'm in the Princeton area here.
- 2:59unknown: Great.
- 3:00SPEAKER_03: Oh, cool.
- 3:01SPEAKER_03: Yeah, this is Debbie.
- 3:02SPEAKER_03: Thanks for having us today.
- 3:04SPEAKER_03: Um, I'm calling in today from um Indianapolis, Indiana.
- 3:09SPEAKER_03: And probably you might know that I'm not from here.
- 3:12SPEAKER_03: I'm not from the Midwest.
- 3:14SPEAKER_03: I'm actually from the beautiful state of West Virginia.
- 3:17SPEAKER_03: All my my birth family is all there, and I get to go back frequently, but love living here in Indianapolis with my family now.
- 3:26SPEAKER_01: Okay, I did detect a southern accent.
- 3:29SPEAKER_01: Definitely not an indie uh drawl.
- 3:33SPEAKER_03: No, no.
- 3:34SPEAKER_03: People when people ask me where I'm from, I'll say, Do you want to know where I live or why I talk this way?
- 3:43SPEAKER_05: Well, I get the opposite.
- 3:44SPEAKER_05: I get you don't sound like you're from New Jersey.
- 3:48SPEAKER_04: Well, I do have to I have to ask Debbie, so um your accent is West Virginian.
- 3:54SPEAKER_04: Um, so do you consider yourself a Southerner?
- 3:56SPEAKER_04: Because you do have a Southern accent.
- 3:58SPEAKER_04: Is it below the Basin Dixon line?
- 4:00SPEAKER_03: Yes, yes.
- 4:02SPEAKER_03: It is.
- 4:03SPEAKER_03: It's it's it is right on the line, though.
- 4:05SPEAKER_03: But but yes, I I do feel um I have a lot of southern tendencies.
- 4:10SPEAKER_03: Okay, good.
- 4:11SPEAKER_04: Well, you're in good company with me.
- 4:13SPEAKER_04: I'm in I'm in Mobile, Alabama.
- 4:15SPEAKER_04: So okay, we can get we've got that out of the way.
- 4:17SPEAKER_04: Let's talk research.
- 4:19SPEAKER_01: Let's start out with some real basics here, you know, in outline for us just in a top line level, the difference between qualitative and quantitative research and and and what the primary options for each of those are, at least in the context of how it might be used in in the insurance industry for research.
- 4:38SPEAKER_05: Sure.
- 4:39SPEAKER_05: So, in terms of just market research in general and and how it's used in the insurance industry, um, you know, there's a wide variety of things we do with our insurance clients.
- 4:50SPEAKER_05: And that can be customer journey, customer satisfaction work.
- 4:54SPEAKER_05: It can be um looking at needs assessment, product development research, um, testing messaging, value props, exploring the market.
- 5:03SPEAKER_05: I mean, just any number of things that I think are that are important in the space.
- 5:07SPEAKER_05: And then the two kind of paths you can take, um, although you can also take both, are qualitative and quantitative research, right?
- 5:15SPEAKER_05: So, qualitative research, when we talk about qualitative, we tend to be talking about uh smaller numbers of people, um in-depth, deep conversations with kind of open-ended type things.
- 5:28SPEAKER_05: So we're talking about the whys and the hows and really how people feel about things.
- 5:33SPEAKER_05: Um, and and you might do things like focus groups.
- 5:36SPEAKER_05: Most people have a, you know, television or movie reference to focus groups, people sitting around a table, right?
- 5:41SPEAKER_05: One-way mirror.
- 5:42SPEAKER_05: Um, or you might do one-on-one discussions we call in-depth interviews.
- 5:46SPEAKER_05: You might do online bulletin board discussions, ethnographies.
- 5:50SPEAKER_05: Uh, on the quantitative side, you're talking about talking to large number of people, right, to get a broader um range of feedback on more closed-ended questions, scale type questions, right?
- 6:02SPEAKER_05: X percent of people said this.
- 6:05SPEAKER_05: Um, and that's generally done by surveys, which these days are mostly done online.
- 6:09SPEAKER_05: Uh, occasionally we still do a phone survey.
- 6:12SPEAKER_05: Once in a while you might see a mail survey, but for the most part, they're done uh through online research.
- 6:17SPEAKER_05: And when you're doing something quantitatively, you can look at subgroups, you can look at statistical differences.
- 6:22SPEAKER_05: Um, that's a little bit different than what you'd see in qualitative.
- 6:26SPEAKER_01: I just got a mail survey.
- 6:28SPEAKER_01: Oh, within the last few months with a dollar attached to it, the old dollar, you know, guilt you into, you know, to doing the survey.
- 6:36SPEAKER_01: And it's like, man, you're not guilting me much for a buck.
- 6:40SPEAKER_01: No.
- 6:42SPEAKER_04: I know.
- 6:42SPEAKER_04: I just chalk it up to that was a bad move on their part.
- 6:46SPEAKER_04: I'd just take the dollar and throw away the crap.
- 6:49SPEAKER_04: Like, you got the wrong person, your list wasn't very good.
- 6:52SPEAKER_04: Um man.
- 6:54SPEAKER_04: Okay, so that's that's kind of the lay of the land.
- 6:56SPEAKER_04: Two different types, you know, two different buckets of of research.
- 7:00SPEAKER_04: So if I am a marketer and I have a project or some initiative that I know I need to do, some kind of measurement, some kind of research, how do I go about determining which method is best suited for my business goal?
- 7:16SPEAKER_04: Sure, sure.
- 7:17SPEAKER_03: So um we do a lot of research, um, both internally led research and through um expert vendors in this space.
- 7:26SPEAKER_03: And so when we're embarking on a research need, I'm gonna um really ask myself a series of questions.
- 7:33SPEAKER_03: Um, the first thing I'm gonna ask is this um research need project based?
- 7:38SPEAKER_03: Is this something that's really just a one-time need, like for a particular program or a service?
- 7:46SPEAKER_03: It could be a new program or service, or you might be wanting to do an assessment on an existing program or service, or it could be a new product launch.
- 7:56SPEAKER_03: So those are all things that are are one point in time type projects.
- 8:02SPEAKER_03: And then I'm also going um to want to see am I um wanting to stay aware of satisfaction levels or improvement needs for something that is ongoing in nature?
- 8:14SPEAKER_03: So those are some of the things that I'm going to assess when figuring out which type of research.
- 8:21SPEAKER_03: I also want to know the number of respondents that I really need to pull from.
- 8:27SPEAKER_03: Is it really important to have a wide variety of respondents at a higher level, or do I want to seek deeper insight from fewer industry leaders or fewer policy holders for this area that I need to know information on?
- 8:44SPEAKER_01: That's a nice um kind of background on how you think about that.
- 8:49SPEAKER_01: Um, you you you spoke a little before about the different methodologies and you know, as you look at in-depth interviews and and you look at focus groups, you know, so both qualitative um methodologies, you know, how do they differ in you know in terms of output and and the reasons why you might do them?
- 9:15SPEAKER_03: Sure.
- 9:16SPEAKER_03: So we just wrapped up a research project with in-depth interviews, and we have commissioned two other in-depth interviews in the past four years.
- 9:27SPEAKER_03: So we have a lot of experience in that area at Fusion.
- 9:30SPEAKER_03: Um, why we chose that method, we needed comprehensive and extensive input in each of these research projects.
- 9:38SPEAKER_03: And we worked with our research vendor to develop a very detailed discussion guide.
- 9:44SPEAKER_03: And I'll just pause there for a second.
- 9:46SPEAKER_03: You know, when you're choosing which way you want to go with research, there may be some type of research that's on an ongoing basis, and you feel that you have the experience in that area that you can manage the research with your own team.
- 10:01SPEAKER_03: Or it might be something that's very specialized that you don't have the experience in.
- 10:07SPEAKER_03: And there really is not a reason to recreate the wheel.
- 10:10SPEAKER_03: There are firms out there that this is they breathe, eat, and sleep research.
- 10:16SPEAKER_03: This is what they do.
- 10:17SPEAKER_03: So, in my opinion, we kind of segue that between what things do we want to manage in-house because we feel it's in our area of expertise, or which things do we want to seek help from the outside to make sure that we're getting the best research possible.
- 10:35SPEAKER_03: So when we were developing with the research firm this discussion guide, we realized um it was going to need to go really deep and be very extensive in the way it was developed to get the input out of the research that we were hoping for.
- 10:50SPEAKER_03: And when we looked at that discussion guide, if we wanted to try to do that in a focus group, that focus group would have been way too long.
- 11:00SPEAKER_03: We really, you know, it would have gone on for four or five hours.
- 11:03SPEAKER_01: And put on put on another pot of coffee.
- 11:06SPEAKER_01: We have more questions here.
- 11:08SPEAKER_03: You're gonna have to give a lot of snacks in that focus group.
- 11:12SPEAKER_03: You you don't really want to take that much time out of today's busy professionals or even your policy holder group that you might be seeking input from.
- 11:22SPEAKER_03: So that would be a reason why you may want to um do an in-depth interview to protect the time.
- 11:29SPEAKER_03: Um, the other thing is to really think about if you're wanting your research participants to try to feed off of each other or not.
- 11:37SPEAKER_03: Um, sometimes you want to get them um bouncing ideas off of each other and you'll get a better quality output that way.
- 11:45SPEAKER_03: But in other ways, you might not want to have people swayed by someone else's opinion, and you really want authentic feedback from that one individual that's not impacted by what they're hearing other people say.
- 12:00SPEAKER_05: Yeah, and I'll I'll just add on to that.
- 12:02SPEAKER_05: So I think, you know, that brings up a really good point.
- 12:04SPEAKER_05: Focus groups are great for ideation.
- 12:06SPEAKER_05: They're great when you want people to bounce ideas off each other and you want them to come up with new ideas, exploratory work, uh, needs assessment, those are all great uh ways to use focus groups.
- 12:17SPEAKER_05: As you might imagine, not great for sensitive topics, right?
- 12:20SPEAKER_05: Not great where you want people to share real personal information or business professionals to share information about their business.
- 12:27SPEAKER_05: They don't want other business, you know, competitors to know that.
- 12:31SPEAKER_05: Um, whereas in individually, as as Debbie referenced, you can go a little bit deeper.
- 12:36SPEAKER_05: Um, it's great for something like a like journey mapping to understand one person's experience because everybody's experience is different.
- 12:42SPEAKER_05: So that's hard to do in a group of people, right?
- 12:44SPEAKER_05: It's easier to do one-on-one.
- 12:46SPEAKER_05: Um, so I think you know, you look at each situation and you kind of assess.
- 12:51SPEAKER_05: And sometimes either would be fine.
- 12:53SPEAKER_05: Uh and in that case, I I'd say the tie goes to a focus group because it's more efficient because you're collecting data from more people at the same time.
- 13:01SPEAKER_01: You know, I I've even used uh one-on-ones as a precursor to focus groups to help to narrow down the discussion guide.
- 13:10SPEAKER_01: You know, if you if you're really not quite sure where where you should focus that thing.
- 13:14SPEAKER_01: So yeah, no, that's this is these are great examples.
- 13:17SPEAKER_01: Uh thanks so much.
- 13:18SPEAKER_04: Yeah, research on research.
- 13:20SPEAKER_04: So um, and you're a naturally inquisitive guy, Jim.
- 13:25SPEAKER_04: So I feel like you're just like you're just a researcher.
- 13:28SPEAKER_04: You're just researching all through life.
- 13:30SPEAKER_04: Um, so we often see projects come to life as the result of anecdotal feedback, which I I suppose is some form of qualitative research, right?
- 13:41SPEAKER_04: Even though maybe we're not seeking it out, we get feedback and suddenly, oh, we should do that thing.
- 13:46SPEAKER_04: So I imagine there's a number of pitfalls that would be an example of one that marketers need to look out for when it comes to qualitative research and their reliance on it.
- 13:54SPEAKER_04: Um, and in your opinion, Debbie or Amy, what are some of the more common pitfalls when it comes to qualitative?
- 14:01SPEAKER_03: Yeah, there's I think one of the biggest things that you have to look at with qualitative research is the subjectiveness that can be there, especially in interpreting the results.
- 14:14SPEAKER_03: And that doesn't really exist in quantitative research.
- 14:18SPEAKER_03: In that situation, the data is the data, the facts are the facts.
- 14:23SPEAKER_03: Now you can start to develop some type of um action items out of quantitative research that could be seen as subjective, but I think that I get probably more concerned on the qualitative side about that subjectiveness.
- 14:38SPEAKER_03: So, and these IDIs that we recently um worked with our vendor on, um, one of the ways that we helped with the subjectiveness from the research perspective was being able to attend as many live sessions as possible.
- 14:54SPEAKER_03: Um, I'll just say plus, that's just really super fun to be able to hear the research live and to be able to interact with the research professional in the background and add in things where you want more deeper dive questions to occur.
- 15:10SPEAKER_03: But then if you can't attend live, we're all pretty busy.
- 15:13SPEAKER_03: Um, even being able to watch recordings of the interviews allowed us to gain that firsthand experience.
- 15:21SPEAKER_03: We also were looking for ways to get more of our leadership and marketing and communication professionals involved.
- 15:28SPEAKER_03: So we um looked at a few to say, like, which of these you know little clips would we want others to look at?
- 15:35SPEAKER_03: So when you do go to do any type of actionable activities that would come out of the research, you're gonna have some of that managerial associate buy-in built in from them being involved up front.
- 15:53SPEAKER_03: And I'd also just go on to say that I think it's um all about getting to the right answers by asking the right questions.
- 16:01SPEAKER_03: So it goes back to the development of the discussion guide.
- 16:05SPEAKER_03: Any research that we've been involved in, we've been very collaborative with our research professional to be involved in the discussion guide, exactly how the questions are asked, what questions are going to be asked, because you want to make sure that we're not leading a participant to the answer that we want to see.
- 16:27SPEAKER_03: We we want their authentic, informative feedback so we will um get get information, even if it's not the path we think we should be going down, it'll be the right path because it's an informed path from our participants.
- 16:44SPEAKER_01: Absolutely.
- 16:45SPEAKER_01: Yeah, that that makes sense.
- 16:46SPEAKER_01: Yeah, and and focus group guides can discussion guides can be leading.
- 16:52SPEAKER_01: You're you know, I mean, if you're not careful, you for sure.
- 16:55SPEAKER_01: Yeah.
- 16:56SPEAKER_01: Shouldn't be, no, but but they no, no, but but but it can happen is what I'm what's uh yeah, yeah, yeah.
- 17:04SPEAKER_04: I feel I feel like um marketers uh are exceptionally good at uh taking like projects or initiatives that maybe we don't have a lot of uh quantitative feedback on to say that it was successful or you know hadn't meant we met the objective, and they'll take uh anecdotal feedback.
- 17:23SPEAKER_04: That's kind of what I was referring to earlier, and there's that is their indication that it was good or successful.
- 17:28SPEAKER_04: And I just feel like it's really dangerous.
- 17:29SPEAKER_04: And as marketers, because we're always like twisting things, I just feel like we're exceptionally good at that and it's a slippery slope.
- 17:38SPEAKER_05: That's one of the advantages of using a third party, right, to help you assess the results of whatever qualitative research you did.
- 17:45SPEAKER_05: Because I always I always say that qualitative research is like the double-edged sword, right?
- 17:50SPEAKER_05: I on one hand, having your stakeholders, having marketers, having product people observe qualitative research is so great.
- 17:58SPEAKER_05: They become so engaged and they have so much buy-in.
- 18:01SPEAKER_05: On the flip side, you get somebody who hears one person say one thing and they latch onto it and they want to make a decision based on it.
- 18:08SPEAKER_05: And so you do need somebody, whether that's an internal research team or an external research partner, to say, whoa, whoa, whoa.
- 18:16SPEAKER_05: Here's what we saw across, right?
- 18:18SPEAKER_05: That is beyond what just one person said.
- 18:21SPEAKER_05: Um, so I do think you can get deep insights.
- 18:24SPEAKER_05: I mean, I think it's more important to talk to more than one person, but I think, of course, you always have to realize it's not quantitative, right?
- 18:31SPEAKER_05: This isn't statistically predictable if you're doing qualitative, right?
- 18:34SPEAKER_01: Yeah, you you always get that one person who came to the one focus group, and it's like, well, don't you remember Joe in Evansville said, well, yeah, but we did 10 other cities.
- 18:48SPEAKER_01: We'll take a quick break and come right back.
- 18:51SPEAKER_02: Are you looking to take your company's growth to the next level?
- 18:54SPEAKER_02: Look no further than One Fire, the RevOps focused marketing agency that specializes in helping insurance companies grow.
- 19:02SPEAKER_02: Our expert team of strategists, marketers, and HubSpot specialists will help you optimize your marketing efforts and maximize your ROI.
- 19:10SPEAKER_02: From lead generation to conversion and retention, OneFire has you covered with our deep insurance industry expertise and modern approach to growth.
- 19:18SPEAKER_02: Don't let your competitors leave you in the dust.
- 19:21SPEAKER_02: Ignite your growth with Onefire.
- 19:24SPEAKER_02: Visit us today at onefire.com.
- 19:28SPEAKER_01: Can you share either of you a real-world example of, you know, where you know quality qualitative research really led to a breakthrough for a campaign or for uh you know a new product for that uh that matter?
- 19:42SPEAKER_05: I mean, I would say a lot of times that happens.
- 19:45SPEAKER_05: I mean, that's kind of the goal, but I will I'll use actually kind of a counterintuitive example.
- 19:50SPEAKER_05: Um uh couple years ago, um, working with one of our clients, carrier clients, who uh had a very passionate, a new project they were very passionate about, senior leadership, especially.
- 20:02SPEAKER_05: This was a passion project, right?
- 20:03SPEAKER_05: This is a new product, it's gonna change the market, it's gonna be amazing.
- 20:07SPEAKER_05: Um, so we tested it with uh, in this case, advisors and agents.
- 20:13SPEAKER_05: Um and we did, I think, bulletin boards.
- 20:16SPEAKER_05: So, you know, you get more, you get 20, 30 people evaluating it.
- 20:20SPEAKER_05: Um and the feedback was so universally and unequivocally negative.
- 20:28SPEAKER_05: It was so you do not need this.
- 20:30SPEAKER_05: We have this other thing that does exactly this, and I would never use this.
- 20:34SPEAKER_05: And it was so clear because we were going to do qualitative and then quantitative, that they were like, nope, we don't even need to do the quantitative.
- 20:41SPEAKER_05: Everybody hates this, and it is just terrible.
- 20:44SPEAKER_05: And I say that because obviously that's not the news that people want to hear, but it demonstrates that market research can be just as important as a risk management tool as a way to prevent yourself from spending a lot of money on something that's not going to be successful.
- 20:59SPEAKER_05: I actually think that's at its core, that's what it is.
- 21:02SPEAKER_05: Um, so that's a great example of something where the money spent on that qualitative research prevented that company from making a very costly error.
- 21:10SPEAKER_01: You bet.
- 21:11SPEAKER_03: Yes, I have a I have a unique example, and it's kind of funny because you all were talking about um surveys that were in the mail.
- 21:19SPEAKER_03: So this is not going to end up with putting a survey in the mail.
- 21:22SPEAKER_03: Um, but it was an interesting case that we worked actually with Zelda's Research, and it was a two-phase research project for a long-term care insurance client of Fusions.
- 21:35SPEAKER_03: And this insurance carrier needed to implement a very complex premium and benefit modification offering to its policyholders, where the average age of the policyholder is over in their upper 80s.
- 21:51SPEAKER_03: So you can see that's that's a pretty um senior base there.
- 21:56SPEAKER_03: So the first phase of the research project had several.
- 22:00SPEAKER_03: Goals, but one of the most important goals was to kind of put the educational materials and the response form together to provide the best education possible and what their choices would be and what those choices would mean for them financially, and also to provide a very clearly designed response form that would help the carrier receive the most responses possible.
- 22:32SPEAKER_03: And with that first phase of the survey, there's actually a strong preference for having these educational materials and the response form come to them in the US mail.
- 22:45SPEAKER_03: They didn't want to have to log on to an online portal or a website or receive a link through an email.
- 22:53SPEAKER_03: They wanted to be able to take the materials, lay everything out on the kitchen table, and go through it possibly with a family member or a trusted advisor and be able to really understand because this is a lot of dollars that were at stake and a lot of impact that this would have on their family.
- 23:11SPEAKER_03: So the other thing that we did, though, that's kind of on the opposite side of that, we explored the opportunity to put together an educational video series that would supplement the mailed materials and give them, you know, an oral explanation of what was in this material and how they needed to go about making their decision.
- 23:36SPEAKER_03: And seven out of the ten research participants said that they would get onto a website and look at a video.
- 23:44SPEAKER_03: So we went through the second phase of the research.
- 23:47SPEAKER_03: We made improvements on the materials that we did.
- 23:50SPEAKER_03: We now provided this client with a seven series video for them to use with their policy holders and actually even submitted it for an IMCA award and received an IMCA showcase award.
- 24:05SPEAKER_03: I was proud of that.
- 24:08SPEAKER_03: So it was just a very interesting thing to really dig into a unique target audience and figure out how we could help those policy holders the best.
- 24:18SPEAKER_04: Well, and I that's a I think that's a really good example of using research to um make sure that the marketing department and the company isn't wasting time on the wrong format, right?
- 24:29SPEAKER_04: I mean, videos are expensive, right?
- 24:31SPEAKER_04: So if they assume to make a video without any kind of research, and let's say they didn't want to see a video, that could have been a a big, a big miss.
- 24:40SPEAKER_04: So that's a great example.
- 24:42SPEAKER_04: Uh switching gears to quantitative.
- 24:45SPEAKER_04: So when it comes to shaping insurance product development or coming up with the marketing strategy, what are some of the reasons that a company might lean more on a quantitative approach, like a survey?
- 24:59SPEAKER_04: And then if you wouldn't mind just also going into maybe the pros and cons of different types of surveys, say online versus phone, you know, that kind of thing.
- 25:08SPEAKER_05: So I think when we use quantitative in product development or communication strategies, marketing strategies, it's it's usually down the road, right?
- 25:17SPEAKER_05: Where it's the the product has been developed, the messaging has been developed, um, the executions have been at least somewhat developed, right?
- 25:26SPEAKER_05: Maybe using qualitative research, maybe not, maybe just internally or using experience.
- 25:30SPEAKER_05: But when you're doing quant, it's you have something to test already, right?
- 25:33SPEAKER_05: It's quant isn't best used to try to figure out what to do because you you have limited kind of questions you're asking, you're not going as deep.
- 25:41SPEAKER_05: But once you have, for example, a product, or maybe you have a couple executions of a marketing campaign and you want to find out which one's best, which one's gonna work, or is this product gonna resonate?
- 25:53SPEAKER_05: What's the potential uptake for this, right?
- 25:56SPEAKER_05: Can we project out uh how we're gonna do with this new product?
- 26:00SPEAKER_05: Then quantitative research is great for that, right?
- 26:02SPEAKER_05: It's gonna help you prioritize, it's gonna help you understand the quote winner, you know, among different executions.
- 26:09SPEAKER_05: It's gonna help you know whether your product's gonna be successful and project out um what the buy might look like by subgroup, who to target.
- 26:18SPEAKER_05: Um, maybe you want to target different groups with different messages.
- 26:20SPEAKER_05: It's gonna help you do that.
- 26:22SPEAKER_05: So it's gonna give you more quantitative feedback, as you know, in the name.
- 26:27SPEAKER_05: It's gonna give you more data that's gonna help you project statistically into the population, as opposed to be able to develop.
- 26:34SPEAKER_05: It might tell you, like, we're doing, for example, it's something right now where we're looking at a value proposition for a new insurance product, and people are gonna highlight the messaging, like what words, what phrases are gonna work best, which ones are really gonna turn people off.
- 26:47SPEAKER_05: So it's gonna give you that level of detail.
- 26:49SPEAKER_05: Um, in terms of the type of quantitative, right now, probably more than 90% of quantitative research is done through online surveys.
- 26:57SPEAKER_05: Um, but we are reaching uh some challenges.
- 27:02SPEAKER_05: If you're doing quantitative research where you need people, right?
- 27:06SPEAKER_05: You're not providing a list of customers, you're not providing your own sample.
- 27:09SPEAKER_05: You have to go out into the world and find people.
- 27:12SPEAKER_05: Um, there's been some major issues with data quality in panels that are out there to go out and find people.
- 27:19SPEAKER_05: AI, unfortunately, bots, things have caused problems with fraud.
- 27:23SPEAKER_05: Um, so that's one of the reasons that some people are going back to phone surveys.
- 27:27SPEAKER_05: So I've got someone on the line with me, I can kind of verify that they are who they're supposed to be.
- 27:32SPEAKER_05: So, from a data quality perspective, a phone can be advantageous.
- 27:35SPEAKER_05: The other bit is represent representation, right?
- 27:38SPEAKER_05: Um, Debbie mentioned talking to folks who are in their 80s.
- 27:43SPEAKER_05: Maybe an online survey isn't the best approach with that population, right?
- 27:46SPEAKER_05: Maybe you're more likely to have success with a phone survey.
- 27:50SPEAKER_05: So you have to look at who you're looking for and whether it'll be represent representative to do something online versus phone.
- 27:56SPEAKER_04: I have a follow-up question related to um, I don't know if bias is the it would be the technical term here, but my phone rings and and nine times out of ten these days, I don't know the phone number that is calling me, right?
- 28:09SPEAKER_04: And so I just don't pick up.
- 28:11SPEAKER_04: And I feel like we're probably all of us are in that same boat.
- 28:14SPEAKER_04: You bet.
- 28:16SPEAKER_04: The same is true for emails, right?
- 28:18SPEAKER_04: We get inundated with emails, they go to junk, they go to spam, the ones that make it through, they get deleted really quickly.
- 28:24SPEAKER_04: Is there some form of, again, I don't know if bias is the right word, but is there some kind of something?
- 28:30SPEAKER_04: Like there's a certain type of person that is gonna pick up the phone or gonna answer that survey.
- 28:35SPEAKER_04: And does that, is there anything about that that makes the sample set not representative of the whole?
- 28:42SPEAKER_05: Yes.
- 28:42SPEAKER_05: I mean, you have hit on many, many issues in our country very succinctly, right?
- 28:47SPEAKER_05: Like that.
- 28:47SPEAKER_05: So, first of all, yes, it's it is bias.
- 28:49SPEAKER_05: It's called response bias, right?
- 28:51SPEAKER_05: So who's gonna respond and who isn't?
- 28:53SPEAKER_05: And are those two populations inherently different?
- 28:56SPEAKER_05: And so when is your people responding look very different than the people you're actually targeting as a whole?
- 29:03SPEAKER_05: All of those things are things we deal with all of the time.
- 29:06SPEAKER_05: And there's lots of different solutions.
- 29:08SPEAKER_05: I mean, I could spend all day talking about there's probability-based sampling.
- 29:11SPEAKER_05: I mean, if you, you know, know anything about political polling, there's a lot of talk of these kinds of things, right?
- 29:16SPEAKER_05: How do we reach the real audience?
- 29:18SPEAKER_05: So there's companies out there that make sure they're creating a sample that's representative, they're going out to households that wouldn't normally participate and making sure they're participating.
- 29:26SPEAKER_05: So that's one way.
- 29:28SPEAKER_05: Um, you know, that's part of the discussion of phone versus online.
- 29:31SPEAKER_05: That's part of the discussion of do you sell phones or not?
- 29:34SPEAKER_05: Do you try to reach people via text?
- 29:36SPEAKER_05: So the industry as a whole is always looking at those things and trying to find ways to make sure that the folks that you're hearing from represent the overall population you're trying to reach.
- 29:47SPEAKER_05: Um, but it's a challenge.
- 29:49SPEAKER_05: I mean, it is, you know, we're always looking for new ways to do that.
- 29:52unknown: Yeah.
- 29:52SPEAKER_01: Heading down the or continuing down the quantitative uh path, and I'm sure the the answer to this is different based on the audience, you know, for example, independent agents versus business customers versus consumers, but how large does a sample size need to be, you know, to get statistically reliable results?
- 30:15SPEAKER_05: Oh, this is like my favorite question.
- 30:17SPEAKER_05: Oh, well.
- 30:18SPEAKER_05: How big a sample do I need to have statistically valid results?
- 30:22SPEAKER_05: And the answer is there's no answer to that question.
- 30:24SPEAKER_05: The answer is there's no such thing, right?
- 30:26SPEAKER_05: There's no, there's no number, specific number.
- 30:30SPEAKER_05: There's no specific definition of statistically reliable or valid results.
- 30:34SPEAKER_05: But that said, right?
- 30:35SPEAKER_05: So there's no, I can't tell you what is the number you have to hit to be statistically reliable.
- 30:40SPEAKER_05: But there are tolerance tables.
- 30:41SPEAKER_05: There's things out there, there are statistics that tell you, depending on the sample size you have, the number of completes you have, what is your plus or minus error, right?
- 30:50SPEAKER_05: So as you go up, that plus or minus gets smaller.
- 30:53SPEAKER_05: As you go down, that plus or minus gets wider, meaning that it you're less likely to be hitting the actual number that you're trying to hit.
- 31:01SPEAKER_05: So the decision on how many to talk to is a mixture of how much error are you willing to have, how much plus or minus, how much money are you willing to spend?
- 31:11SPEAKER_05: Because right at the end of the day, you're making decisions around budget.
- 31:14SPEAKER_05: So you can look at that and say, you know, I really want, I don't want to be more than plus or minus three, right?
- 31:19SPEAKER_05: In terms of what I'm that my statistic is going to match the population statistic.
- 31:23SPEAKER_05: And then you're gonna look at that and you're gonna say, okay, I need X number, I need 500, or I need 700.
- 31:28SPEAKER_05: Um, I will say, just as a rule of thumb, we don't like to look at any group quantitatively, ideally below 100.
- 31:38SPEAKER_05: And sometimes when you really need to look at a subgroup or a tough to maybe below 50, right?
- 31:43SPEAKER_05: Maybe you'll use 50, but you'll use it with some some caution, right?
- 31:47SPEAKER_05: Um, but a hundred is where you start to see a little bit more stability.
- 31:51SPEAKER_05: And beyond that, again, it really depends on your threshold for willingness to have some some error in the data.
- 31:58SPEAKER_04: Could that be translated to um percentages?
- 32:01SPEAKER_04: So for example, um if if if we did a survey, if we had an event and we wanted to do a post-event survey about, you know, how how did you feel about the event?
- 32:11SPEAKER_04: Was it a good event?
- 32:12SPEAKER_04: And we only had 50 people attend, right?
- 32:15SPEAKER_04: Like we aren't gonna get 100%.
- 32:17SPEAKER_04: So would a 50% like where's the threshold of percentage?
- 32:22SPEAKER_04: Like, oh, well, 50% of the people responded.
- 32:24SPEAKER_04: So that feels like we can take these results and they're statistically significant.
- 32:29SPEAKER_05: Yeah, I mean, I don't, you know, we can there's power in statistics, so it does depend on the size of the population and the size of the relative sample that you're drawing.
- 32:37SPEAKER_05: I don't want to get I'm not a statistician.
- 32:39SPEAKER_05: I will technically I studied statistics, but I'm not a statistician.
- 32:42SPEAKER_05: Um, but yes, I short answer is it does matter how big your population is and how big of a sample you're drawing from that population.
- 32:51SPEAKER_05: But again, no magic number.
- 32:53SPEAKER_05: Not nobody can say, okay, as long as you get 50% of the population, then it's statistically valid.
- 32:59SPEAKER_05: Yeah.
- 33:00SPEAKER_05: There's just a range.
- 33:01SPEAKER_03: Yeah, there's a type of survey that we do quarterly for a client, and it's based on um call center activity and their um experience that they had in talking to a call center representative to get information.
- 33:18SPEAKER_03: And so we do have a um a sample size that we usually do of 500.
- 33:23SPEAKER_03: It's a core, it's a quarterly rolling type of thing.
- 33:26SPEAKER_03: But we get to a point where if calls are down and we have less than 500 and you'd have to go back for more than two months to get more um participants, well, that's that's gonna hurt you because that's too long.
- 33:43SPEAKER_03: You know, even two months is a time for someone to remember a conversation.
- 33:47SPEAKER_03: But so we like are looking at a balance between time and quantity to get us to that right, right thing.
- 33:55SPEAKER_03: So I think there sometimes are other variables that weigh in, like you'd like to have this certain quantity, but there's other things that might trump that if it um doesn't allow it to happen.
- 34:07SPEAKER_05: Yeah, that makes sense.
- 34:08SPEAKER_05: I also I just want to throw out there when we're talking about sample sizes, and I know we're talking about quant now, but I just want to go back for one second to qual because one of the things I see very often is folks who think they need way more than they need for qual.
- 34:21SPEAKER_05: Like they're trying to like somehow make qual into quant, but getting 30 in-depth interviews, you don't need 30 internet in-depth interviews, and it doesn't make it quant, and it's just wasting a lot of time and energy.
- 34:31SPEAKER_05: So I will just put it out there that you know, with qualitative, you know, don't try to force it to be much higher than it needs to be.
- 34:39SPEAKER_04: That's interesting.
- 34:40SPEAKER_04: I didn't think about that.
- 34:41SPEAKER_04: People probably think it's like super research, right?
- 34:43SPEAKER_04: So they're getting a lot of a lot of words.
- 34:46SPEAKER_04: Super qual.
- 34:48SPEAKER_04: Super qual.
- 34:49SPEAKER_01: Now, and Leslie, if you need to you think about bending it a little bit, I'm looking over at my uh my bookcase there, and I have a book all of uh how to lie with statistics.
- 34:59SPEAKER_04: Yeah, I mean, as long as as long as it's a little bit more than half, it's most, right?
- 35:03SPEAKER_04: Most workers.
- 35:07SPEAKER_04: Markers are dangerous people.
- 35:10SPEAKER_04: Um okay, so super qual, that's a that's a new word.
- 35:14SPEAKER_04: I like that.
- 35:15SPEAKER_04: Uh another word that I learned um prior to this conversation is synchronous and asynchronous.
- 35:23SPEAKER_04: And I have no idea what those words mean.
- 35:25SPEAKER_04: And so, but I know that they're very important in the research space.
- 35:28SPEAKER_04: So I'm hoping that you both can tell us a little bit about the use of synchronous versus asynchronous in qualitative research, what it means and how it works.
- 35:40SPEAKER_05: So all we mean when we say synchronous versus asynchronous, when we're talking about qualitative research, is it is it live in the present?
- 35:47SPEAKER_05: We're having this conversation, or is it something where someone can go back whenever they want to to answer the questions or to answer follow-up questions?
- 35:55SPEAKER_05: So is it happening live point in time, or is it happening at the respondent's convenience over a longer time period?
- 36:02SPEAKER_05: So in qualitative research, when I do a focus group or an in-depth interview, that's synchronous.
- 36:07SPEAKER_05: It's happening in real time.
- 36:08SPEAKER_05: I'm talking to them.
- 36:09SPEAKER_05: If they say something, I can probe them in real time to get more information or to go deeper.
- 36:14SPEAKER_05: If I do an online bulletin board discussion where I'm putting up a series of questions, I'm posting them, and I'm telling the respondents, hey, you can go on whenever you want.
- 36:23SPEAKER_05: We'd like you to go on, you know, twice or three times in a day, answer the questions, check back to see if there's any follow-ups, answer those follow-ups.
- 36:30SPEAKER_05: That's asynchronous, right?
- 36:32SPEAKER_05: They're doing it on their own clock.
- 36:34SPEAKER_05: I'm not there to be like, but what did you mean by that?
- 36:37SPEAKER_05: I can put it in a probe, but they're they may or not may not come back and answer that.
- 36:42SPEAKER_05: But asynchronous research works very well for folks who are very busy or very hard to schedule because it can happen whenever they want it to happen.
- 36:50SPEAKER_03: Yeah, I was thinking the very same thing with Amy.
- 36:53SPEAKER_03: It's it's hard sometimes to um get the people that you want to participate in research due to their busy schedules.
- 37:01SPEAKER_03: Um, and the other thing that I would um add on the synchronous research is you really gain momentum when you have people doing this type of research live.
- 37:12SPEAKER_03: So you're getting people to add into each other's thoughts, um, dig deeper on certain topics and layer things on.
- 37:22SPEAKER_03: And I just don't think that you can get the quality of that momentum going with an asynchronous asynchronous qualitative research.
- 37:31SPEAKER_01: Makes sense.
- 37:32SPEAKER_01: So so kind of to put a bow around a lot of what we've talked about so far, what's a what's a smart way to combine you know multiple forms of research in you know into a kind of a phased approach?
- 37:49SPEAKER_01: And can you give us some examples of where that uh that that might make sense?
- 37:54SPEAKER_03: Sure.
- 37:55SPEAKER_03: I think that starting with a quantitative research with a large group of participants can help you get initial data.
- 38:05SPEAKER_03: And then you can add a few qualitative questions at the end, might be a written survey or an online survey that's check the box, rate it one through five, and then add just a can you give us more information about this, or is there any other feedback you'd like to offer?
- 38:21SPEAKER_03: Some type of open-ended question at the end of your quantitative survey.
- 38:26SPEAKER_03: And then based on those responses, you can choose respondents to participate in a qualitative research to get that more in-depth information from a smaller group of people.
- 38:41SPEAKER_03: And we've been able to do that with some of our satisfaction surveys where we can ask the quantitative information, we're being able to develop the trend lines, we can measure it against the baseline, seeing what some of the activities that we're implementing are seeing the improvements that we want to improve.
- 39:00SPEAKER_03: But then there's still those open-ended questions that may require or would encourage a phone call to reach out on and do a more in-depth interview about a certain situation that can lead to even greater intel on that program or that satisfaction of the service that you're providing that can give you more information to make improvements for the future.
- 39:26SPEAKER_05: And I'll give the opposite example because that's a common thing that we do.
- 39:30SPEAKER_05: But on the flip side, with concept development, product development, messaging, we often will do qual to sort of get the ideas, to maybe test some high-level thoughts and get some sort of suggestions for tweaking and improvement and have those qualitative conversations.
- 39:47SPEAKER_05: Use those to have the marketing team develop final executions, versions, and then test those in a quantitative to get the numbers, to get the you know, statistical validity around it.
- 39:57SPEAKER_05: Um, so that's another common sort of multi-phase qual quant, kind of the other way, right?
- 40:02SPEAKER_05: Qual for before quant.
- 40:03SPEAKER_01: Those are great examples of of um different ways to use the methodologies together.
- 40:10SPEAKER_01: Thank you.
- 40:10SPEAKER_04: Yeah, definitely.
- 40:11SPEAKER_04: And and it doesn't involve uh interviewing a lot of people, right?
- 40:16SPEAKER_04: The super, the super qual.
- 40:17SPEAKER_04: Super qual.
- 40:18SPEAKER_06: Yeah.
- 40:19SPEAKER_06: Super qual.
- 40:20SPEAKER_04: Which is music to my ears because that sounds like a lot of time.
- 40:23SPEAKER_04: Yeah.
- 40:23SPEAKER_04: Um well Debbie and Amy, thank you so much.
- 40:26SPEAKER_04: You guys have given us a lot to think about when it comes to using research in our marketing efforts.
- 40:31SPEAKER_04: And if if any of our listeners want to reach out to you, if they have more questions uh following this conversation, what's the what's the best way for them to get in touch with you?
- 40:40SPEAKER_05: You can reach me at Amy, just A M Y at Zeldisresearch.com.
- 40:45SPEAKER_05: We are such a small company, we all just use our first names in our email.
- 40:50SPEAKER_03: Love it.
- 40:51SPEAKER_03: Yeah, you can reach me at DMoss.
- 40:54SPEAKER_03: That's D-M-A-A-S at Fusion with a Z, fusion services.com.
- 41:02SPEAKER_03: Um, you're also welcome to connect with me on LinkedIn.
- 41:05SPEAKER_03: And my LinkedIn URL is Debbie F Moss.
- 41:10SPEAKER_01: Awesome.
- 41:11SPEAKER_01: That is a wrap on today's episode of the IMCA Peer-to-Peer Podcast.
- 41:16SPEAKER_01: We hope you gained some valuable insights into the key differences between qualitative and quantitative and super qual is we have a new word out there and how each of those is uh appropriate and when to use them.
- 41:31SPEAKER_01: And uh it's really been interesting to see that and to talk about the different options here.
- 41:37SPEAKER_01: If you've enjoyed the episode, please don't forget to subscribe and give us a five-star rating and leave us a review.
- 41:44SPEAKER_01: Leaving us a review helps us reach more listeners like you.
- 41:48SPEAKER_01: And if you would like to share some thoughts on topics that you'd like us to cover in future episodes, that would be great.
- 41:56SPEAKER_01: Shoot us over an email at ideas at imca peer, the number two peer podcast.com.
- 42:06SPEAKER_01: Once again, this is Jim Flynn from Onefire, along with my co-host Leslie Castillo from America Equity Underwriters.
- 42:14SPEAKER_01: And it has been our honor and a pleasure to spend this time together along with our guest, Debbie and Amy.
- 42:20SPEAKER_01: Thank you so much again for being part of today's episode of the IMCA Peer to Peer Podcast.
- 42:25SPEAKER_03: Thank you.
- 42:26SPEAKER_03: Thank you, Jim.
- 42:27SPEAKER_00: The Peer to Peer Podcast is a presentation of IMCA.
- 42:30SPEAKER_00: To learn more about today's episode and listen to other episodes, visit IMCANet.com.
- 42:36SPEAKER_00: This has been a production of One Fire.