Latest / Shiny New Object - a Marketing Podcast / Episode 276 / Boaz Sobrado / Capital.com / Director of Marketing Analytics
Transcript
- 0:00Boaz Sobrado: If you develop relationship with the data, if
- 0:02you look at it every day, if you interrogate it every day, if you
- 0:05look at how different variables move together, you will start
- 0:08developing instincts around how things move together and what
- 0:13happens when all things happen.
- 0:21Speaker: Are you a fitness loving foodie? Then defy snacks
- 0:24is for you. Defy offers real high quality chocolate you love,
- 0:27combined with functional ingredients, giving you as much
- 0:29as 26 grams of protein per bag. So you can be both a little bit
- 0:33bad and a little bit good at the same time as a female owned
- 0:36brand, defy dedicates a portion of its profits to other women
- 0:40owned businesses currently sold in the US. You can find us on
- 0:43defy with an eye snacks.com and follow us on social at defy
- 0:48snacks.
- 0:54Tom Ollerton: Hello and welcome to the shiny new object podcast.
- 0:58My name is Tom Ollerton. I'm the founder of automated creative
- 1:01and this is a podcast about the future of data driven marketing,
- 1:06and I have the privileged position of being able to
- 1:08interview some amazing people from across the industry about
- 1:12their vision for the future of this industry. And this week is
- 1:15absolutely no different. I'm on a call with Boaz Sobrado, who is
- 1:21director of marketing analytics at Capital.com. So Boaz, for
- 1:25anyone who doesn't know who you are and what you do, can you
- 1:28give them a quick background?
- 1:30Boaz Sobrado: Hi Tom. It's absolutely great to be on the
- 1:32podcast. I have listened to a good few episodes of your
- 1:35podcast, and I've really enjoyed it. I've learned a lot. So thank
- 1:38you for inviting me on. Yes. So in terms of my role and myself
- 1:45as an individual and as a professional, I work for
- 1:48capital.com and I'm responsible for marketing analytics. The way
- 1:54I usually explain what Marketing Analytics does is I use that old
- 2:01phrase that was attributed to a retailer over 100 years ago,
- 2:05which is, I know that half of my marketing budget is wasted. I
- 2:10just don't know which half. The way I explain what Marketing
- 2:14Analytics does is trying to figure out which half of the
- 2:19marketing budget effectively is wasted, and this is a much
- 2:25tougher task than you might think, because it's very much
- 2:30not obvious what exactly drives performance and what doesn't. So
- 2:34that is the sort of thing that me and my team try to help with
- 2:37here at Capital.com.
- 2:39Tom Ollerton: So are you a marketing book reader? Or are
- 2:44you a... Do you watch videos or conferences? Or how do you get
- 2:48your knowledge about how to get better at your data driven
- 2:51marketing job?
- 2:53Boaz Sobrado: There's... there's a wide range of sources. So I do
- 2:56very much enjoy listening to podcasts. I also spend probably
- 3:02more time than I should on Twitter, engaging in the
- 3:06conversations that are there around marketing, around
- 3:09technology, etc. But I am also a very big fan of books. And I'm a
- 3:15fan of books that are actually perhaps slightly different to
- 3:18the ones that most people would read simply because, I think the
- 3:24difficulty with marketing with books is that, particularly
- 3:28digital marketing, things move so quickly, so fast, that by the
- 3:31time that someone has the time to write a book and publish it,
- 3:35and then someone actually gets to the point of buying it, a lot
- 3:39of what has been written is kind of no longer actual or out of
- 3:44date, etc. So one thing that I've particularly liked about
- 3:48books is looking at books that have been published a long time
- 3:52ago. So you can very easily go to your local library, look up
- 3:58the marketing section and try to look at books that are 5060,
- 4:027080, even 100 years or more old. And the very interesting
- 4:06thing about them is that, obviously there's a lot of stuff
- 4:12there that's out of date, but it's very obvious what's out of
- 4:16date. So it's actually quite easy to get signal and the
- 4:19signal to noise ratio is a lot better. And there's a lot of
- 4:22things that if they were valid 100 years ago, they're very much
- 4:25valid still today, then they're particularly true.
- 4:28Tom Ollerton: So how did you come up with that idea? And
- 4:30that's such a kind of odd, odd approach. Did you just stumble
- 4:35across that as a source of insight, or did someone suggest
- 4:38it to you?
- 4:39Boaz Sobrado: So there's so this is something that I've heard
- 4:45recommended by other people in other fields as well. I mean,
- 4:49it's no coincidence that people to this date like reading the
- 4:53classics, so to speak, both in literature, but in other fields.
- 4:58But the way I've heard. Actually came across it is I was
- 5:01wandering around in the library, and I came across a very old
- 5:07direct response handbook from over 100 years ago. And as I
- 5:13started reading through it, I realized, Oh, actually, a lot of
- 5:16the stuff that is mentioned here, if it's not very clearly,
- 5:20obviously, out of date, then it's actually very useful. So
- 5:26that's how I stumbled into this little niche.
- 5:29Tom Ollerton: That's interesting. I'm writing a book
- 5:30myself at the minute about the relationship between data,
- 5:33creativity and marketing, and it's really frustrating, because
- 5:36everyone wants to talk about generative AI, which I do as
- 5:39well, but by the time it gets published next year, then that
- 5:43that will be a whole different beast. You know, may have faded
- 5:46or morphed to, who knows? So, yes, you have to think like,
- 5:50okay, how can I put stuff in this book? This hopefully will
- 5:53be useful in 100 years time? Oh gosh, the thought of anyone
- 5:56doing that would be remarkable. But anyway, so moving on. So
- 6:00what is there a specific book that you would recommend?
- 6:03Boaz Sobrado: So one of the books I recommend to all of the
- 6:06new joiners here at Capital is a book that's not actually related
- 6:12directly to marketing. It's called reminisces of a stock
- 6:16market operator, and it was written almost exactly 100 years
- 6:19ago. But although it's not directly about marketing, I
- 6:24think it touches a lot of marketing topics very well. It
- 6:27touches the importance of word of mouth, of how different
- 6:32narratives get constructed and how different narratives get
- 6:35disseminated. But also, and it's very, very powerful on this, it
- 6:39touches a lot on the instincts that drive people to make a
- 6:45purchase. And although the absolute details of the
- 6:51companies and markets and products that this book talks
- 6:54about are entirely out of date, you know, it's talking about
- 6:57companies that existed 100 years ago and no longer exist.
- 7:02Nevertheless, a lot of the sort of mechanisms and the procedures
- 7:06and the way this information spreads is very, very much
- 7:09topical, and I recommend it to everyone that joins the company.
- 7:19Tom Ollerton: So what advice would you give to today's data
- 7:23driven marketer to be better at the discipline?
- 7:26Boaz Sobrado: I think one of the big advantages we have today is
- 7:30that we're about whereas 100 years ago, I mean, this is
- 7:34something that's mentioned in the book. 100 years ago, you had
- 7:36to pay someone to make you a chart about whatever you wanted
- 7:40to see. And it took a couple of days, or maybe even weeks to get
- 7:44your chart. Today, you can sit down at your laptop and look at
- 7:48your chart as many times as you want. And I think that starting
- 7:52that your day like that, starting your day looking at
- 7:55dashboards, interrogating them, digging through them, seeing how
- 8:00different things move together, is basically the fastest and
- 8:03easiest way you can have to becoming a good data driven
- 8:07marketer, because if you develop a relationship with the data, if
- 8:11you look at it every day, if you interrogate it every day, if you
- 8:14look at how different variables move together, you will start
- 8:17developing instincts around how things move together and what
- 8:23happens when odd things happen. You can look at a dashboard one
- 8:27day you can see, oh, that that's kind of odd. Usually when that
- 8:31line goes up, this other one also goes up. But today it
- 8:34hasn't gone up, you know. And then you have to interrogate it
- 8:37a little bit, dig through it a little bit, and then it turns
- 8:39out that, for some reason, there was an issue with the Android
- 8:43version of your app, or there is some language that has been
- 8:46mistranslated to, and there's been performance issues in some
- 8:49countries, or something like that. The point being that if
- 8:53you develop a good, instinctive familiarity with the data, and
- 8:57you can only do this by looking at it every day, you will start
- 9:00spotting things that other people wouldn't spot, and that
- 9:03will be an advantage.
- 9:04Tom Ollerton: So how do you do that specifically? Could you
- 9:07give me an example, maybe, in a previous role where you you've
- 9:13daily, had that kind of daily view of the data, and you spot
- 9:17the anomaly you've digged in. You dug into the anomaly, and
- 9:21you found an insight that's been beneficial to the growth of the
- 9:24business. Could you? Could you give me an anecdote or a story
- 9:27to articulate that?
- 9:28Boaz Sobrado: Yeah, I'm happy to share in general terms, the the
- 9:33way I like to approach it is looking at the top line business
- 9:37metrics that the top business leaders would be interested in
- 9:41looking at the revenue numbers, the spend numbers, all of the
- 9:47important things that happen between all of the onboarding
- 9:51flows split across different channels, regions, etc, etc. So
- 9:57just getting an instinctive feel for the heartbeat of the
- 10:01business. You know, our Monday is usually stronger than
- 10:04Tuesdays or Fridays. The strongest day. Is the last
- 10:07Friday of the month unusually strong? Or is it the first
- 10:11Monday of the month that's unusually strong? Just sort of
- 10:15getting an intuition around, you know, the data, heartbeat of the
- 10:19business. And once you've done that, once you've acquired that
- 10:23instinctive feeling for it, you can immediately spot trends. So
- 10:29in particular, there's an anecdote where I saw that our
- 10:34conversion rates in the previous business were dropping
- 10:39significantly, and I managed to, after digging around a little
- 10:46bit, figure out that there was a relatively niche issue in one of
- 10:53the implementations of the onboarding flow that impacted
- 10:58the subset of users that would have otherwise taken a
- 11:03significant amount for someone to spot because it was such a
- 11:05niche issue. But nevertheless, because I had this intuitive
- 11:10feel for what was going on, I managed to figure out that
- 11:14something was wrong and relatively quickly identify it,
- 11:18even though, as a marketer, I was strictly responsible for the
- 11:23onboarding process, but nevertheless, it impacted the
- 11:26performance significantly so I could drag it out and try to
- 11:31figure it out with with the responsible Product Manager.
- 11:39Tom Ollerton: This episode of the shiny new object podcast is
- 11:43brought to you in partnership with madfest, whether it's live
- 11:46in London or streamed online to the global marketing community,
- 11:49you can always expect a distinctive and daring blend of
- 11:52fast paced content, startup innovation pitches and
- 11:55unconventional entertainment from madfest events. You'll find
- 11:58me causing trouble on stage, recording live versions of this
- 12:01podcast and sharing a beer with the nicest and most influential
- 12:05people in marketing. Check it out at www dot madfest
- 12:09london.com.
- 12:16So we're going to move on now to your shiny new object, which is
- 12:20predictive modeling of conversion events. Now I think I
- 12:24know what exactly what that is, and I'm really buzzed to talk to
- 12:27you about it. But can you explain to the audience what
- 12:29that is for someone who may have never heard of it?
- 12:32Boaz Sobrado: Yeah, I think I first started thinking about
- 12:34this concept when I worked at my previous company, which was a
- 12:39consumer credit business, which effectively you can think of as
- 12:44kind of like a credit card. And when you're selling credit
- 12:49cards, when you're quote, unquote, selling money, when
- 12:52you're giving money to people, the outcome is actually quite
- 12:59different than when you're selling furniture, right?
- 13:01Because if you're in the credit card business, you have a
- 13:04customer that comes in and takes 100 pounds. Well, they can be if
- 13:10they're a loyal customer, and they stick around for years, and
- 13:13they pay back their debts on time, etc. They could be a very
- 13:16valuable customer, but they could also be a customer that
- 13:19comes in, takes 100 pounds, never pays anything back, right?
- 13:22So effectively, you've got this, this range of potentially
- 13:26negative values for a conversion event, which is not the case in
- 13:31E commerce, right? In E commerce, if you sell a chair,
- 13:34if you sell a table, if you sell a bag, whatever piece of
- 13:37furniture, then generally, you know how much you sold it for,
- 13:41and you know exactly what your margin on that is, and you can
- 13:44try to estimate it, and that's the conversion value that for
- 13:47you feedback to the advertising platform and that you optimize
- 13:50against. Now I know that that's not entirely true, because
- 13:54occasionally you get returns and etc, etc, so there's some
- 13:57complications there. But on the whole, it's a little bit more
- 14:00straightforward. However, when you work in a FinTech company
- 14:05and you're looking at the value of the user that you've just
- 14:08acquired, the range is a lot bigger, and in the case of
- 14:13credit, can potentially be negative as well. So I then
- 14:17started thinking, Okay, well, how do I do a better job
- 14:22actually giving the algorithms that end up optimizing all the
- 14:26ad delivery, etc, the data that they need in order to be able to
- 14:32optimize better. And that's where my thinking and obsession
- 14:35with these predictively modeled conversion events started.
- 14:39Tom Ollerton: So I would say that was a fairly long answer.
- 14:41So could you be more specific? You could just help someone give
- 14:44me, give me a basic description of what it is.
- 14:47Boaz Sobrado: So the basic description would be that
- 14:50instead of feeding back the raw data of what actually happened
- 14:54on your web platform or your app or your website, you take some
- 14:58function of that data, and you modify the raw data based on
- 15:04what you think is going to be the real value and not the
- 15:08actual observed value. So to give you an example, sticking to
- 15:13the furniture business example, when you sell a table, say,
- 15:19online, and you know that your margin on that table is, say, 20
- 15:22pounds. Instead of giving the platform the 20 pounds value
- 15:27that they can then optimize against, you give it some
- 15:29function of the 20 pounds based on maybe, if it's a new
- 15:33customer, you expect them to shop again. You multiply the 20
- 15:36pounds by two, and you feed it back 40 pounds. Where, if it's
- 15:39whereas, if it's a repeat customer that buys from you
- 15:41every week. You apply some sort of discount to it, and you feed
- 15:44it 10 pounds, for instance. So effectively, it's taking the
- 15:49observed values that you get from your advertising activities
- 15:54and modifying the real value to generate some sort of predictive
- 15:58or synthetic value that hopefully will drive better
- 16:01outcomes for you within your marketing process.
- 16:04Tom Ollerton: So basically, you're taking conversion events
- 16:07that happen on site, furniture, financial services or otherwise,
- 16:12and then you're predicting the lifetime value of that customer,
- 16:16or a short term value of that customer, and reporting that
- 16:19back to the business.
- 16:20Boaz Sobrado: It's effectively kind of like that, except you
- 16:24can also... it's basically that. So if you simplify it to
- 16:31individual users, let's say your your goal is user acquisition.
- 16:35You want to acquire a user instead of looking at the first
- 16:38transaction that a user makes. You can also try to make an
- 16:41estimate of the user's lifetime value upon the first
- 16:44transaction, and feed it that feedback to the algorithm
- 16:48itself. And that is a perfectly legitimate approach, and a lot
- 16:49And so how common is predictive modeling of conversion events?
- 16:54of companies do this. In fact, it's very valuable thing to do
- 16:58in cases of, say, sparse data, where you have relatively few
- 17:02values to actually go off. So you actually need to give the
- 17:05algorithm a bit of a confidence boost and give it a big number,
- 17:08whereas you only have small numbers, could also be valuable
- 17:12in the case in the example that I mentioned earlier, with
- 17:15negative lifetime values. So for instance, if a user is probably
- 17:19actually not worth very much based on the behavior you've
- 17:22seen from them, then, instead of giving a potentially high value
- 17:26that their first transaction would warrant, you could
- 17:28actually feed the algorithm at low value. But in general, it's
- 17:35not just using the LTV predictions. It's about
- 17:40modifying the real values to make them a little bit more
- 17:44palatable to the algorithm that's effectively making the
- 17:51bidding so to give you an example, I don't know if you're
- 17:54familiar with UAC campaigns or universal app campaigns, but
- 17:59effectively, these are campaigns that are trying to optimize on
- 18:02Google towards an in app event. Now I've heard from people that
- 18:08used to work at Google that effectively any value over $500
- 18:16that is fed to this algorithm is actually just truncated. So it
- 18:24will just take the the you know, if you send it 501 as a
- 18:29conversion event value, it will just truncate it to 500 right?
- 18:34So there is a limit to the values that the algorithm will
- 18:38actually accept. And there's various reasons for that, but
- 18:42the underlying reason for it is that the algorithms that are
- 18:46used to optimize advertising performance in this sort of
- 18:50black box world that we live in are algorithms that make certain
- 18:55assumptions about the distribution of the underlying
- 18:58data. Now what that effectively means is that if Google's doing
- 19:03it on their end, then maybe there's something that you can
- 19:07also do on your end to optimize the algorithm to performance. So
- 19:12truncating values, for instance, or normalizing them, or all
- 19:17sorts of things like this is a good example that is actually
- 19:21relatively commonly used across performance marketing to try to
- 19:27nudge the algorithm into finding the results that you want it to
- 19:30find.
- 19:35Now, how much of a shiny new object is in your experience?
- 19:38I think it's it's pretty shiny object, in the sense of a lot of
- 19:44people want to do it. A lot of people try to do it. I think it
- 19:51is potentially something that can be quite tricky to do,
- 19:56simply because, depending on theinfrastructure and the size
- 20:01of the team that you might have, you might not actually have a
- 20:05dedicated data scientist sat there being able to actually put
- 20:11together machine learning models to try to predict the value of
- 20:15every single account or user that you acquire. That being
- 20:19said, what I think is quite common is a much simpler version
- 20:23of this, which is some sort of simple lead scoring algorithm. I
- 20:28think most companies that do user acquisition have some sort
- 20:33of lead scoring whereby they draw these lines of, if the user
- 20:36meets this requirement, then it's worth x, and if the user
- 20:40meets that requirement, then it's worthwhile from that
- 20:43perspective. It's, I think, relatively common.
- 20:52Tom Ollerton: So how would you advise someone? Let's say advice
- 20:56to people so that you so a team that is big enough to have a
- 21:00data scientist and a team that isn't how would you go about
- 21:05kicking off predictive modeling of conversion events within
- 21:08those two scenarios?
- 21:10Boaz Sobrado: I think the underlying philosophy, the most
- 21:15important thing is to be familiar with the underlying
- 21:18philosophy, which is the goal is to make the advertising
- 21:24platforms algorithm work for you as a business, not necessarily
- 21:32for the advertising platform itself, right? So you have to
- 21:35keep in mind that there, you know, most of the time there's
- 21:39alignment of interests, but there are some sort of scenarios
- 21:42in which it's not necessarily the case. I think perhaps
- 21:50there's this familiar anecdote. I don't know if you're familiar
- 21:53with it, Tom, but I'll repeat it for the sake of the audience of
- 21:56the pizza flyer operation around you know, there's, there's this
- 22:04person with a with a restaurant, and they hire these three
- 22:08teenagers to hand out flyers in front of the pizza shop and to
- 22:14hopefully get people into the restaurant. And the three
- 22:20teenagers go out, and every single flyer has some sort of
- 22:23unique identifier. So you know, you can tell which customer came
- 22:26from which teenager. And then what you see is first teenager
- 22:30at the end of the day gets three customers in, second one gets
- 22:32four, and the last one gets 34 and the restaurant owner is
- 22:36quite surprised, and he asked them, Okay, like, how? How are
- 22:40you getting so many people to come in? And effectively, the
- 22:44teenager says, Well, I hand in the flyer before they step into
- 22:46the restaurant. So that's sort of a long winded anecdote
- 22:51towards incrementality. But the point I wanted to make is, you
- 22:55know, you have to be aware of the dynamics of incrementality
- 23:00when it comes to the actual conversion events that you're
- 23:03feeding back to the platform, such that whether you've got
- 23:10data scientists or not, you need to be aware of what is actually
- 23:15driving value for you and your business. Now if you do not have
- 23:18a data scientist, I think keeping taking this to heart and
- 23:23being very, very conscious of the fact that not all of the
- 23:26conversions that the ad platforms take credit for are
- 23:30actually real, and they can legitimately take credit for
- 23:33them, I think that's the single most important thing you can do.
- 23:37And if you keep that in mind and you interrogate the data based
- 23:41on that notion, you'll do very well. With regards to when you
- 23:47actually do have a data scientist and you can actually
- 23:50work towards building these predictive lifetime value
- 23:54models. I think there's two things you can focus on. First
- 23:58of all, generally when it comes to user acquisitions, what
- 24:03people use as a signal to estimate the lifetime value of
- 24:07the user is the first or the first couple of transactions
- 24:11that a user makes will very much inform the value that of the
- 24:17user going forward. Now the that's kind of the baseline that
- 24:21the data scientist needs to work to, you know, on top of, and
- 24:25need to try to improve that model iteratively, right? Go
- 24:31through it, okay, I know that if you make two transactions in the
- 24:34first day, then you're most likely to be worth X. Okay.
- 24:37Well, how can I improve this model? There's the amount of
- 24:40times you log in before transaction inform it, et
- 24:42cetera, et cetera. And then that's the actual value, if
- 24:45you're technically capable enough and have the right
- 24:47infrastructure that you can feed back to the advertising
- 24:50platform. That's one thing that I would focus in that respect.
- 24:53The second is actually just testing, trying to test as much
- 24:57as possible. What is it that the. Algorithm will work with
- 25:02well with if you feed them these values versus what are the
- 25:06things that will simply not be functional. And the last point I
- 25:13wanted to make is, generally, the more data points you can
- 25:17feed an algorithm, the better. So the more you can multiply
- 25:22your conversion events, and not necessarily just limit them to
- 25:27the sparse, high value ones, but rather to the more frequent,
- 25:34occurrent ones, even if they're not necessarily truly events
- 25:39that drive value that will definitely help improve the
- 25:45quality of the ad campaigns. Now, I appreciate this has been
- 25:49a long answer to a simple question, but hopefully I
- 25:51haven't rambled too much.
- 25:52Tom Ollerton: Well, look, unfortunately, we are at the end
- 25:54of the podcast now, and you've opened my eyes to some things,
- 25:58and I have to say, I'm not sure I totally get all of it, and
- 26:03that's how you learn, right? So thank you for showing me a
- 26:06different way of thinking about what is a niche of data driven
- 26:10marketing. So I really appreciate that. So if someone
- 26:14wants to get in touch with you about anything you've talked
- 26:18about, be it books or predictive modeling, how, how and where is
- 26:23a good way to get in touch with you.
- 26:25Boaz Sobrado: You can find me on Twitter, on SobradoB. You can
- 26:29find me on LinkedIn, looking for Boaz Sobrado. There's no one
- 26:32else in the world with that name. And you can also find my
- 26:35website, boazsobrado.com.
- 26:36Tom Ollerton: Brilliant. Boaz, thank you so much for your time.
- 26:39Boaz Sobrado: Thank you.
- 26:46Tom Ollerton: just before you go, I'd really appreciate it if
- 26:50you could take the time to write a review of the shiny new object
- 26:53podcast on Apple podcasts or iTunes, whatever it's called
- 26:56these days, or whichever podcast provider you use. We're an in
- 27:01your podcast, so it would go a long way for us. If you could
- 27:04just share the Word and give us a bit of a support on those
- 27:06channels. That'd just be fantastic. If you haven't got
- 27:09time. That's also cool. And yeah, if you could tell your
- 27:11colleagues about the podcast, and also, if possible, don't
- 27:15forget to subscribe. And I'd love to hear your feedback, if
- 27:18you'd like to speak on the podcast, I'll be a guest, or you
- 27:21think I'm asking the wrong questions. Anything? I'd be
- 27:24super interested to hear what you think. So please email me.
- 27:27Tom at, automated creative.net, that's Tom at, I'm not going to
- 27:34bother spelling it anyway. You'll work it out. Thanks so
- 27:36much.