Latest / Gimbal Podcast / Ex-DOAC Employee Explains Why Thumbnail Testing Just Isn't Enough Anymore

Transcript

Ex-DOAC Employee Explains Why Thumbnail Testing Just Isn't Enough Anymore · Gimbal Podcast

• 0:00
  One of the biggest things for us was pre-predicting as much as we could before anything went live.
• 0:05
  How the heck do you identify whether a guest is going to perform or not?
• 0:07
  How formal is like your experimentation process?
• 0:10
  Do you like write out a hypothesis?
• 0:12
  And for me, like that was the most amazing quality in Alita because... Grace, my first question for you is just around this incredible, unique title that you had working with the Diary of a CEO.
• 0:25
  What were your main responsibilities as head of failure and experimentation?
• 0:30
  Yes, it is quite a unique title.
• 0:32
  If we wanted to experiment on our production and how we were editing the episode, for example, there was two parts.
• 0:38
  The first was like, how do we improve everything we're already doing?
• 0:42
  So that was looking at like, what are we doing on YouTube and how do we improve that current performance?
• 0:46
  Or how does the episode look or the guests and how do we improve that?
• 0:50
  And then the other side of it was looking at what are the new opportunities that we could have and we could experiment on
• 0:55
  to get ahead of the industry.
• 0:57
  When I think of experiment, I think it could go off the rails very, very quickly.
• 1:00
  So how do you have the right constraints on the experiment to know that it's worth the time and money and energy to put into it?
• 1:06
  This is a massive question because it's something we considered for a long time as well.
• 1:10
  At the start, when I was in the experimentation team, we looked at all these ideas and were like, let's try everything and see what works.
• 1:18
  And it actually became, like, made us so stretched thin of what we could try.
• 1:23
  And also then the results were stretched thin because we couldn't delve deeper.
• 1:26
  So what we ended up doing was looking at what our company KPIs or show KPIs were and going, what are the main goals we want to achieve?
• 1:33
  And how do we experiment towards those goals?
• 1:36
  So instead of just running experiments, because you could run experiments on a million different things within a year.
• 1:42
  But we were like, what are the three or five KPIs we really want to achieve?
• 1:46
  And how do we make sure everything we're doing is working towards them?
• 1:49
  And that really helped us align on actually focusing experiments.
• 1:53
  Say, for example, we really wanted to improve click-through rate.
• 1:56
  So click-through rate could be in the middle of our diagram.
• 1:59
  Yeah.
• 1:59
  And we run experiments all around that to try and improve.
• 2:02
  So it has that one overarching goal rather than before.
• 2:06
  It might have been, there's one experiment for click-through rate, one experiment for an average duration, and they were all running around each other.
• 2:13
  Okay, so if you were to try to improve click-through rate, you mentioned that there was a bunch of different ways of doing that.
• 2:21
  Let's say that that is the KPI that you're tracking and it's day one.
• 2:25
  What does day one look like to get to the point where eventually the CTR is higher?
• 2:31
  What we would do is go and brainstorm in a session.
• 2:34
  How many ways can we try and improve our click-through rate?
• 2:36
  Also, click-through rate is probably the worst example.
• 2:41
  I'd say other ones like arbitrary duration are better.
• 2:45
  But say we're doing it on click-through rate, we'd sit down, brainstorm, like, what are all the ways that we could improve it?
• 2:49
  Working off real-life analytics.
• 2:51
  So we're not just, like, thinking of things.
• 2:54
  We're looking at what has performed well before, what hasn't, and why.
• 2:57
  So we're using, we used to call it killing the guesswork, using the data to go what will work and what won't in the future.
• 3:03
  So we're almost trying to, before we've even started the experiment, pre-predict with the previous data what's going to work.
• 3:10
  The way we would test it is not all at the same time, obviously, because you want one variable.
• 3:15
  You don't want to test a million different ideas.
• 3:17
  Were you thinking at a level where you were testing more than just the thumbnail to get CTR?
• 3:23
  Were you thinking about, well, what if we tested it on Facebook first and see with paid ads, those kinds of things?
• 3:28
  Yeah, we started doing that four years ago, maybe.
• 3:30
  It was in my first year that I started there.
• 3:33
  We started thumbnail testing, or we called it pre-testing, ahead of time.
• 3:37
  So we used to pre-test all of our thumbnails before they went on YouTube.
• 3:40
  And obviously, at the time, YouTube didn't have the thumbnail testing tool, which the one they have now is obviously the best because it's based on the real algorithm.
• 3:49
  The closer we could get to knowing what would work before is why, over the last few years, we've had such insane growth of the show.
• 3:57
  Is there like a go-to example where something that came out of your department exponentially changed the way that you operate at Diary of a CEO?
• 4:06
  Well, I'll share a few.
• 4:08
  The first one was we didn't used to have Steve's face on the thumbnail, which sounds so little.
• 4:14
  But one day we were like, let's try his face.
• 4:17
  And this was years ago.
• 4:18
  And we're like, let's try his face and see what happens.
• 4:20
  And we added it and it won every single time.
• 4:22
  And we were almost like, why did we not do that before?
• 4:25
  Right.
• 4:26
  The other one is definitely translations.
• 4:27
  Like, it is the coolest part of just before I left.
• 4:31
  Like, what I had done was helping to build that.
• 4:34
  Because I think I didn't realise when I started how big of an area it would be.
• 4:40
  Especially three years ago, the industry wasn't really like, Instagram didn't have the translation feature.
• 4:45
  YouTube had only brought it out more recently.
• 4:48
  And so for us, we were like, let's try it, let's see what happens.
• 4:51
  Right.
• 4:51
  At the start, we were like, okay, this is not really working.
• 4:54
  And we thought, like, do we park it?
• 4:56
  Do we keep going?
• 4:57
  But we just, like, kept trying to see, will it at some point pick up?
• 5:01
  And it was at the start of last year that we were like, this is it.
• 5:05
  And it just started to take off and skyrocket.
• 5:08
  And I think the best part of that for me is, like, if you were a normal team or a normal company where we had, like, KPIs to hit every year.
• 5:15
  we probably would have stopped that after the first year because it wasn't that big.
• 5:20
  And it wasn't really taking off.
• 5:21
  And we were probably investing more time and money than what it was worth.
• 5:24
  Whereas we were allowed to continue over the years to keep trying.
• 5:28
  And I think now it's taken off.
• 5:30
  And now we were ahead of the industry because people are now trying to, it was AI auto dubbed by the way, but we were ahead of the industry and it was just lucky that we thought this is worth trying and keep testing and,
• 5:40
  How do you actually implement what you did at Flight Story into a smaller team so that testing is actually happening and not getting brushed off for something else?
• 5:50
  I get that question a lot.
• 5:52
  And usually it's even like, I'm a solo creator or solopreneur.
• 5:56
  How do I have the time to experiment?
• 5:58
  Or how do I find the time to experiment?
• 6:00
  Especially in YouTube, I think sometimes you can get caught up.
• 6:02
  And this happened to us sometimes is you would build the episode, release it, and then move on to the next one so fast.
• 6:09
  And you don't actually get to sit there, review it, go, what actually happened?
• 6:13
  You might watch like the first 24, 48 hours and then go, okay, onto the next episode.
• 6:17
  Rather than actually going like, what has happened here throughout this whole episode?
• 6:21
  And it's even harder when it's a two-hour, three-hour episode.
• 6:24
  But like, what has happened and what worked and what didn't work?
• 6:27
  And I want to say it's even harder.
• 6:29
  I think some YouTube channels would be amazing on because if the guest doesn't change, but for podcasts especially.
• 6:35
  It can be really hard because the guest is such a massive variable in a show.
• 6:39
  But if you say a channel like a MrBeast desk or Sidemen, those sort of channels where you can have kind of the same core, like people in the episode, it's really fun to experiment with.
• 6:51
  And like from a smaller team's perspective, I think it's about the accountability.
• 6:55
  So when I said culture before, it's when you're reviewing those episodes or the videos saying like, what experiments have we run this week?
• 7:02
  Who ran an experiment?
• 7:03
  Who didn't?
• 7:04
  How do we come together and like keep that accountability up?
• 7:07
  Because we know that's how we're going to grow.
• 7:08
  I think some people are very data driven people and then others are not.
• 7:12
  And it doesn't mean that the people who are not data driven can't experiment.
• 7:16
  They definitely can.
• 7:18
  Like one of our previous social media managers in our company who used to manage Steve Socials, she was like very creative driven rather than data.
• 7:26
  And I was like, let's mix the two and become still creative driven, but use the data to help us inform the creative.
• 7:34
  And there's a whole debate of it, of like, should it be brand or data led?
• 7:37
  At the end of the day, like controversial, but I am very much saying led by the numbers rather than just a nice colour or brand design.
• 7:45
  But I think in terms of a team, the size should not matter.
• 7:48
  It should be, what can we do to change one element or some elements within this piece of work?
• 7:54
  So if you're always seeing like a drop off at 10 minutes, let's say, so you have a 30 minute episode and you're always seeing a drop off at 10 minutes and that's where your AVD changes.
• 8:04
  What can you do just before that 10 minute mark to see if you can keep people going?
• 8:09
  And again, not a perfect experiment because there's so many variables, but we want to try like as much as we can to try and experiment within it.
• 8:17
  So if we said, OK, we're going to put something on screen that's like this really interesting thing is coming in two minutes and we put that at nine minutes and see if people stay an extra two minutes on the AVD.
• 8:28
  Like, does it work?
• 8:29
  And I think, don't get me wrong, it's not going to work for every channel, but I think it's worth those little things of like, what can you test?
• 8:36
  And also, it doesn't have to be massive.
• 8:38
  Like, they can be one percents and little things.
• 8:41
  And I think that's what experimentation can scare people, because they go like, I am a solo creator.
• 8:46
  I can't, don't have time to test anything.
• 8:49
  And it's like, well, if you're making the content, let's at least try something new or tweak something slightly every time.
• 8:56
  How formal is like your experimentation process?
• 8:59
  Do you like write out a hypothesis and like have a budget and things like that?
• 9:02
  Yeah, usually budget wise, it's pretty flexible.
• 9:06
  So we would have, well, we had like overall yearly budgets recently, but before it was really flexible, like we would say, hey, this is the experiment.
• 9:13
  So the actual steps we had, like we had multiple steps of like, this is what we need to fulfill for an experiment.
• 9:18
  So definitely a hypothesis.
• 9:20
  Then we would write out like, what is the experiment actually going to be?
• 9:23
  And then the last one was the determinant.
• 9:26
  actually deem the experiment a success or a failure.
• 9:29
  And so we would run that for every experiment.
• 9:31
  And also it's evolved even more now.
• 9:33
  Like that was when I started.
• 9:34
  Now it's starting to grow and evolve into even more like different team and experiment and procedures as well.
• 9:42
  But I think the most important part in it was people actually sharing things
• 9:45
  the successes and the learnings or the failures, because there's no point in me going and running an experiment and then keeping the results to myself and not sharing it with everyone, because we're going to learn faster from 100 people rather than just one person.
• 10:00
  How did people share them?
• 10:01
  So there's a few different ways.
• 10:02
  We had a Slack channel so people could share straightaway updates in there.
• 10:06
  Then we had like a database of all the experiments that have ever run so that people could then go in there and see like what was run on this date or on this channel?
• 10:14
  What was the performance?
• 10:15
  When do we need to retest it?
• 10:17
  So, for example, with the thumbnails, we started adding images to the Diary of a CEO ones.
• 10:22
  And that was tested because we thought, oh, we haven't tested anything on thumbnails in a while.
• 10:26
  Let's keep testing and seeing what works.
• 10:28
  Yeah.
• 10:28
  How many people are on the team overall?
• 10:30
  And you said you had a team under you doing experiments.
• 10:33
  And how does that play out throughout the team?
• 10:35
  Yeah, it's a good question because it's something we had debated for a long time of whether experimentation should sit in its own team within Flight Story as the parent company or should there be an experimenter in each team?
• 10:50
  So an experimenter in the social team or the production team.
• 10:54
  And we ended up having it right now as...
• 10:55
  a separate team that works with every team, which is like actually worked out for the better because it then gives the people in those teams the not only autonomy, but the responsibility to experiment themselves.
• 11:08
  Whereas if there was an experiment in their team, they'd probably be relying on that person to experiment.
• 11:13
  rather than taking the experimentation culture and ingraining it into their work.
• 11:19
  And that was the most important part, to be honest, was people actually experimenting because they wanted to, not because they had to.
• 11:26
  And it was really hard at the start because...
• 11:29
  Everyone was super busy, as they are in every company, and they were like, I don't have time for this.
• 11:33
  I don't have time to try this new thing.
• 11:35
  But the whole point is, if you just keep doing the same thing, you're usually not going to grow.
• 11:40
  You need to keep innovating or testing new things every single episode.
• 11:44
  And, like, the best person who ever said this is MrBeast, obviously.
• 11:47
  Work out what didn't, didn't work, and keep improving.
• 11:50
  Yeah.
• 11:50
  And so within our teams, we were like, you take the full autonomy to go and run these experiments.
• 11:55
  We're here to help.
• 11:56
  So like help them write up the experiment procedure and help them write results.
• 12:00
  And we also used to be able to like give out an experimenter of the week trophy, which was really fun because it ingrained it as part of it.
• 12:07
  Yeah, it was like it became part of the culture.
• 12:10
  And I think the most special part was I left the company five weeks ago.
• 12:15
  And what was really special is since I've left, I've seen people posting on LinkedIn that they wanted the experimenter of the week trophy.
• 12:22
  I'm like, that is so nice to know.
• 12:24
  I had a little bit of impact on the culture of that company and that the trophy that we started is now still going even after I've left.
• 12:31
  I felt that I was able to then go and do my own thing now because the team.
• 12:36
  So I had three people within the experimentation team when I left, which was big for us because we're only a team of 100.
• 12:42
  But only a team of 100.
• 12:44
  That's like the biggest team we've had on the show.
• 12:48
  But we are, we do have lots of shows.
• 12:50
  It's not just the Diary of a CEO.
• 12:52
  What is like the turnaround time for a video?
• 12:54
  Between the time you record with the guests and publish it, like how many weeks is that?
• 12:58
  And then like you experimented on this day and it took this long to get back to you.
• 13:03
  Every guest was different.
• 13:04
  Sometimes like this may sound crazy, but sometimes we had a guest record on say a Saturday night and it came out Monday morning.
• 13:13
  Whoa.
• 13:14
  Oh my gosh.
• 13:15
  Yeah.
• 13:15
  I can't even explain how little preparation.
• 13:19
  Sometimes it was a long time, but sometimes it was very tight turnarounds.
• 13:23
  And like when we used to do the translations, we had to wait for the full episode to be ready and say the full episode.
• 13:29
  So we used to release like 6am and 8am on Monday morning.
• 13:33
  Sometimes the full episode was ready at midnight on Monday, which meant we had to translate between midnight and 8am going live.
• 13:40
  A lot of like, even before we even got to the episode, our guest booking or production or pre-production teams used to pre-test guests where possible as well.
• 13:50
  So they would do heaps of research and testing on like, will this guest perform?
• 13:54
  Will this topic perform?
• 13:56
  Is the topic even interesting and are people searching it?
• 14:00
  So there was things that happened before even the recording a lot of the time.
• 14:04
  Do you think of Diary of a CEO as a podcast or as a YouTube show?
• 14:08
  We stopped calling it a podcast maybe in the end of my first year, and we called it a show because, number one, podcasts are not – well, they're seen as audio only, even though they're not anymore.
• 14:21
  And so we wanted it to be a show because it is video first.
• 14:25
  And that was like four years ago we started calling it a show.
• 14:28
  But also because there are so many elements that are not just a podcast anymore, like the visual elements on screen, the props.
• 14:35
  So like definitely a show and not a podcast.
• 14:37
  And I think everyone should call themselves a show now if they're doing video as well.
• 14:41
  One thing that you mentioned that kind of like piqued my interest was you're pre-testing guests and you're pre-testing topics.
• 14:46
  And when I think about YouTube, you know, outlier theory is very, very known and popular.
• 14:50
  And that makes a lot of sense for talking head creators or, you know, any kind of creator.
• 14:55
  But for a podcast slash show, I haven't seen many people do that kind of research and then go and find the guests.
• 15:03
  Is that what you're saying happened?
• 15:04
  You would you would identify how the heck do you identify whether a guest is going to perform or not?
• 15:09
  So it used to be very manual.
• 15:11
  Also, to preface, as I said before, nothing is ever perfect 100% of the time.
• 15:16
  Because you can see a guest on one show be an outlier and perform 10 times better than they use your views.
• 15:23
  And then you can see that same guest on another show completely bomb out and not even get the average views for that show.
• 15:30
  Guests are very dependent on the audience of the show as well.
• 15:33
  not just that that guest is an amazing guest.
• 15:36
  It's like there's so many nuances to performance.
• 15:38
  But again, killing the guesswork, I don't know how many times I will say that.
• 15:42
  It was like, it was a massive part of like, how can we find as much data as possible before we even like send the guest as an option to that host?
• 15:51
  And then one day we were like, why don't we make this an actual platform that we can use?
• 15:56
  And so our data team built that.
• 15:58
  But one of the platforms that our data team ended up building was a profile page of every single person that's been on shows on YouTube.
• 16:10
  And say you wanted a guest on women's health.
• 16:13
  You could type in women's health and it would come up with heaps of different options of potential guests for that topic.
• 16:20
  Or you could go even more niche.
• 16:21
  Let's say like diabetes.
• 16:22
  I'm making it up.
• 16:23
  diabetes and you wanted someone that's an expert on talking about that.
• 16:27
  You could type in this specific topic, would come up with all of the guests, and then it would say, here's all the videos they've been on.
• 16:34
  They have been an outlier on these many videos.
• 16:37
  So say they'd been on 10 shows and they had overperformed every single time on those shows, that person would have a percentage score that was way higher than others.
• 16:46
  And it also was not relative.
• 16:48
  Like, say they'd been on a massive show.
• 16:50
  That didn't matter.
• 16:51
  It was like, did they overperform for the show size?
• 16:54
  And so it would rank it on that and we could see, like, does the guest perform before they've even come on a show?
• 17:00
  And not to say, like, some guests, like, especially for Diary of a CEO when it was, like, health experts, some of them have never been on podcasts.
• 17:08
  So it could have been the first time.
• 17:10
  So it wasn't the holy grail, but it was helping us able to, like, pre-analyze
• 17:16
  anything beforehand of people who had been on shows.
• 17:19
  And then there was so much other data that came into it and testing.
• 17:22
  Sometimes we would pre-test topics or potential lines from, not lines that they would say, lines of topic, beforehand to see is there interest in that topic from an audience.
• 17:35
  But all of it, like every single guest I would say was different, and I'm sure our research and guest booking team will attest to that, is like some of it was relying on like, for example, a health expert.
• 17:46
  If they didn't have a profile, we obviously couldn't use the platform that would analyze their previous performance, but we would then use other data to go like, is their topic different?
• 17:54
  about to take off.
• 17:55
  And it kind of is like gambling in a way of like the risk taking of saying like, okay, we think this topic is about to take off.
• 18:01
  Like, I guess GLP ones before they took off, we think a topic is about to take off.
• 18:05
  How can we pre-predict that and get it at the right time when it is going to skyrocket on search and volume?
• 18:12
  Okay, this is blowing my mind.
• 18:14
  Right now on our show, you know, we're kind of scouring the internet and finding people that we think would make really interesting guests.
• 18:21
  So what you've just described has made me rethink how we're going to make our show, which is how do we find really, really interesting guests, but also what's coming up in the zeitgeist in social media?
• 18:30
  And can we find someone who perfectly matches with that topic so that when it comes out, it hits the internet at the perfect timing?
• 18:38
  Yeah.
• 18:38
  What are the topics or areas you're thinking of now of future guests?
• 18:43
  I'm always looking for people who work for YouTube channels or shows that have, I don't know, I would kind of set the benchmark at like 10 million subscribers or more.
• 18:53
  I learned it from Becky, who was previously the head of YouTube from Ali Abdaal's team.
• 18:58
  And when she joined Ali Abdaal's team, she said, you know, Ali had 6 million subscribers at the time that she joined.
• 19:03
  And she was like, if I want to learn this platform really, really well, I need to be speaking with people who have 10 million subscribers.
• 19:09
  So that I level up.
• 19:11
  Experimentations and failures are things that I'm intrigued about.
• 19:14
  It's something that I'm admittedly not doing enough of.
• 19:17
  And so that makes you a really interesting guest for this show.
• 19:19
  And as long as I'm really, really interested, I think it would make a great episode.
• 19:22
  But now, now I'm thinking, what is coming up in like November and December around experimentation and failures?
• 19:31
  Or something like that, you know?
• 19:33
  Yeah, there's so many.
• 19:34
  Like, I was even thinking as you were saying that, I met a guy in another podcast I did, and we were talking about Instagram.
• 19:40
  I know different audience, but like Instagram trial reels.
• 19:43
  And he was testing, like, he had a Red Bull can in one and then changed the color of the Red Bull can, then did no Red Bull can.
• 19:50
  And he was testing, like, tiny, tiny elements that maybe a team with 10 million subscribers might not even think to bother doing anymore.
• 19:58
  Yeah.
• 19:59
  And I was like, it's so interesting, I think, even hearing, depending on, like, I think, can you pre-predict their success and their growth?
• 20:06
  But, like, catching people as they're about to take off.
• 20:09
  Because they're still in that element of, like, the testing, like, deep, deep testing and learning to see what works and how do they find that, like, moment and rocket moment that catapults them forward.
• 20:20
  But I just thought it was so interesting.
• 20:21
  I was like, sometimes you can learn...
• 20:23
  You learn a lot from people who've already done it for sure.
• 20:26
  But then also these people coming up and Steve always used to say, he used to say this is we want to make sure that the 18 year old in their bedroom or the 16 year old in their bedroom, that's learning and just deep in YouTube, like the old Mr.
• 20:38
  Beast when he was younger, he will overtake us one day.
• 20:41
  So how do we make sure we are always outworking and out experimenting and
• 20:45
  that 16-year-old sat in their bedroom learning at the fastest rate possible.
• 20:49
  And that was like always the driver for me.
• 20:51
  And that's why I find it so fascinating to learn from, for me, I'm like, I want to learn from every level, whether you've got like 10 million or like 10 subscribers at the time to see like, what are you doing to get from 10 to the next 10 and keep growing?
• 21:05
  You brought up Steve, who was your employer for a very long time.
• 21:10
  I'm really curious what it was like to work alongside someone who is, you know, one of the most respected business owners, hosts in the creator economy.
• 21:18
  What are some things you learned about excellence from working so closely with Stephen Bartlett?
• 21:23
  You have to sacrifice a lot, whether it is time, whether it is missing out on things.
• 21:30
  Because Steve would always spend as much time as he could learning and honing his craft.
• 21:36
  And my favourite story is him and Mr Beast.
• 21:38
  When they were recording the Dary of a CEO episode, Steve said, he is the only person he's met that felt so much harder working than him, which I thought was fascinating because to me, he was the hardest working person I'd ever met.
• 21:51
  And I was like, if Mr.
• 21:52
  Beast is harder working than Steve, that is a different level.
• 21:55
  But Steve particularly, like, not only is he the most hardworking person I've ever met, but he also cares so much.
• 22:02
  Like, and when I say care, not just cares about the people, he cares about the work so much to the point where, like, the 1%s.
• 22:10
  It has been so ingrained in me, and I think this is why he's been like, Grace, you can stay here for as long as you want.
• 22:14
  Like, at the start, I used to, in a polite way, if there was something wrong with something that was live, whether it was a spelling error or...
• 22:20
  even an alignment issue on a thumbnail, whatever, we would tell each other.
• 22:24
  I would say because I cared about the performance and the overall assets.
• 22:28
  And I think that's why Steve used to say, thank you so much for feeling comfortable to share and speak out loud when something's not right.
• 22:36
  That's why I think I've been able to have such an amazing career at that podcast.
• 22:41
  is because I've like really embodied that part of him, which was caring so much.
• 22:46
  And like whether things blowing up and going amazingly or whether something had underperformed, like we would still care just as much on any performance of an episode.
• 22:55
  And I don't think it was just Steve.
• 22:57
  I think at the start, every single person, when I started, there were seven of us and every single person in that team at the start cared so deeply.
• 23:06
  And it's quite rare to find that, I think, people who will do whatever it takes for that episode to go live.
• 23:13
  We interviewed the producer of Chris Williamson's show, Modern Wisdom, and he said a lot of similar things that you just described.
• 23:20
  Chris being exceptionally hardworking.
• 23:22
  And he said, Chris really cares about the minutia, the 1%.
• 23:26
  And Chris will take note of the top pinned comment in a YouTube thing, and he'll make adjustments for the next time.
• 23:33
  So it's interesting that...
• 23:34
  This might be like opening a huge can of worms, but how is like Stephen or even you personally like using AI?
• 23:42
  Yeah, we've had like multiple AI competitions within our team of like having stints of where we will build things and then run them.
• 23:52
  So like mid last year, we had our first AI competition and I...
• 23:57
  had maybe used ChatGPT, and this is so naive, but I'm not embarrassed to say.
• 24:02
  I had zero AI experience.
• 24:04
  And I'd used ChatGPT, and then for this AI stint, I was like, okay, I'm going to challenge myself to really, really build something.
• 24:12
  And I built this TikTok translator that...
• 24:16
  took our social clips that had performed well on our English content, uploaded them to be translated, then wrote a copy for them and put on the translated text.
• 24:25
  Then it would upload them to a Spanish TikTok channel and then it would go live.
• 24:30
  And it was basically this rotating thing.
• 24:32
  And I built the whole thing myself.
• 24:34
  I had no idea.
• 24:34
  It took me so long, but it was the best challenge of like actually trying to use AI.
• 24:39
  And I think if people are still using it for like note taking, fine.
• 24:43
  Or like asking questions,
• 24:45
  like the most basic form but it's still a good first step at least you're using it but i think if you can like having that test where you go like okay for the next two weeks whoever builds the most innovative or most valuable thing in ai for our team we will do this for you and i like it really changed what we did because since that point of testing myself to use the ai i now use it across everything
• 25:08
  like whether it's something super simple with like granola where you've told someone you're recording them and it like can pre-look up who the person is so you can jump onto a meeting you don't even have to do research it already says all of your previous context or about the guest or the person you're speaking to then it can like take all of the summary of that meeting if you needed to write an email it can help you write the email like that is such a basic form of it let alone like
• 25:32
  We have been experimenting with the AI translated languages.
• 25:37
  So there's such different scopes, but everyone across our team used AI because it's such an important thing.
• 25:44
  If I can take my job and get AI to do 100% of my job for me, then I can use all of that time to go and do other things for the company still, but go and do other work and do other
• 25:54
  And like, Steve never saw that.
• 25:56
  He was never like, if you replace yourself with AI, you're going to be fired.
• 26:00
  It was never that.
• 26:01
  It was like, if you replace yourself with AI, we will probably promote you.
• 26:05
  It's like the concept of how I would see it.
• 26:07
  It was never feared.
• 26:09
  It was always so encouraged.
• 26:11
  What are you building now?
• 26:12
  What are you experimenting with now?
• 26:14
  Yeah, it's pretty scary because I only left five weeks ago.
• 26:18
  So it's still very fresh.
• 26:20
  Can we ask why you left?
• 26:22
  Yeah, I am basically before I even joined the diary of a CEO, I had always wanted to start my own company, which is why Steve obviously inspired me so much.
• 26:33
  And then when I moved to London, I saw the job and I was like, before I start my company, I'm going to apply for this and see if this works out.
• 26:39
  And it did.
• 26:39
  And I was like, okay, either way, like amazing.
• 26:42
  And so Steve had known, like, even in my first meeting with him, I was like, I want to be an entrepreneur one day.
• 26:47
  And four and a half years later, obviously, it was the most amazing journey.
• 26:51
  And I wasn't ready to leave any sooner before that.
• 26:54
  But I'd had so much self-confidence and an itch to do my own thing.
• 26:59
  And basically, long story short, I had been doing talks about experimenting and failing on stages for the last two years around the world.
• 27:07
  I have done some in America and some in Australia or in different spots.
• 27:13
  And I'd realized there was such a gap in the market for people needing to know more about experimenting because,
• 27:19
  Not just in the media industry, like across every industry.
• 27:23
  And now, like I've spoken at medical companies, like in their away days to teach them, like, how do you experiment more in a highly regulated industry versus like creators, which is really fun to experiment with.
• 27:34
  But...
• 27:35
  I kind of saw that and I was like, there's this really big gap in the market and I don't want to miss out on an opportunity to go and be that person to help educate people around the world of how they can try new things and not fear failure.
• 27:47
  And I was like, now is the right time because I'd just been given so many opportunities, maybe manifested, maybe the universe working in amazing ways.
• 27:55
  Now I'm looking, like I have my personal brand, which is Grace Experiments, which is where I do all the talks.
• 28:00
  And then I have a new company called Experiment Media.
• 28:03
  which is helping businesses, shows, teaching them how to experiment more, and also helping them through that experimentation, which is like my favorite things of like production, marketing, how do you experiment more within those areas in the company.
• 28:16
  And then I also very crazy started a YouTube channel because I could not leave YouTube and never be involved with it again.
• 28:24
  So I've been, and this is the best learning journey for me, because as talking about experimentation, I'm now like, I have to experiment on this channel as much as possible, and like test and learn and see what works.
• 28:36
  But I've been recording every day since I left.
• 28:40
  Because I really wanted to be honest and show like, what is it actually like when you leave a job?
• 28:44
  And I think a lot of people, especially in the creator world, say, I left my job and now I have a million followers and I'm getting paid heaps of money and this is what it's like.
• 28:53
  And I'm like, it's not really like that when you leave.
• 28:55
  You leave and you're like, what am I doing on my first day?
• 28:58
  I have 12 hours or more in the day of just sitting here doing what?
• 29:02
  And so I wanted to create that like real reality of what is it like when you quit a job?
• 29:07
  Is it just you doing all these things?
• 29:09
  Yeah, it's just me.
• 29:10
  Like, my brother helps with the YouTube editing sometimes and things.
• 29:14
  But I love it.
• 29:14
  I'm like, it's so fun to work on all these things.
• 29:16
  So I don't want to leave anything.
• 29:18
  And then I have another idea for an app, but that's a whole different conversation.
• 29:22
  It sounds like that hardworking attitude from Stephen has rubbed off onto you, because that's a lot of things to be taking care of all at once.
• 29:29
  Yeah, it is.
• 29:30
  But I also am at the stage where I want to – I think for me, over my career, the thing that has got me so far is saying yes to opportunities.
• 29:39
  And so I don't want to say no yet, until I get to a point where I have to say no because of time.
• 29:44
  I'm like, I will say yes.
• 29:46
  Someone asked me, do you want to come to Paris tomorrow?
• 29:48
  And I was like, yes.
• 29:49
  So I'm going to Paris tomorrow, which is –
• 29:52
  Wow.
• 29:53
  It's so random.
• 29:54
  But I was like, saying yes has got me so far.
• 29:56
  And like being friends and connected with people, like even talking about experiments, the reason we thought of translations was because I was chatting with other people in the YouTube world, which is why this podcast is so amazing.
• 30:07
  It's hearing from other people in the industry is where you get further.
• 30:10
  It really is.
• 30:12
  It's not just like-
• 30:13
  sitting alone in a room trying to think of things.
• 30:15
  That is one way.
• 30:16
  But actually brainstorming with people and sharing.
• 30:18
  And I was at 1 Billion Followers Summit in Dubai in January.
• 30:22
  100% go to it is the best.
• 30:24
  I'm not paid to say this.
• 30:25
  It is the best conference ever.
• 30:27
  Because it was everyone in the YouTube creator sphere.
• 30:31
  talking about what they're doing right now.
• 30:33
  And I was like, that is... And talk about when you're saying, oh, these are the people ahead.
• 30:37
  You're right, they are.
• 30:39
  And it was fascinating to hear, what are they talking about?
• 30:41
  Translations was a massive topic, especially AI translations.
• 30:45
  And hearing about what they were talking about and being involved in that to be like, okay, we should do this, or we should test this, was really, really valuable.
• 30:53
  So yeah, those connections are everything.
• 30:54
  But saying yes, I need to probably say no more, but I'm still on the saying yes train.
• 30:59
  And you'll be at VidSummit.
• 31:00
  Yes.
• 31:01
  Yeah, yeah.
• 31:01
  Are you guys going?
• 31:02
  We'll be there.
• 31:02
  We're going.
• 31:03
  Of course.
• 31:04
  Yeah, conferences are really important to my year because like you said, like you're meeting all these different people who you wouldn't see otherwise.
• 31:12
  It's good for finding out what's right and what's wrong and what people are talking about.
• 31:15
  So I'm a big advocate of conferences, too.
• 31:17
  And I hope to go to the one in Dubai in January.
• 31:19
  We'll see.
• 31:20
  Who is a creator that's on your radar right now that's worthy of a shout out?
• 31:24
  Ooh, I know exactly who this is.
• 31:26
  It's actually a show.
• 31:27
  It's called The Art of the Brand.
• 31:29
  Her name is Camille.
• 31:30
  She's the host.
• 31:31
  I think she's from Canada, actually.
• 31:33
  Oh, let's go.
• 31:34
  Okay.
• 31:35
  Yeah.
• 31:35
  And it is the most amazing show because number one is because of timeliness.
• 31:41
  Like they released content, for example, Coachella Week, they released the podcast within the weeks of Coachella about the Coachella and everything that was happening.
• 31:49
  Number two is they're actually experts.
• 31:51
  They're not people who have just decided to launch a show about branding.
• 31:56
  They're actual experts.
• 31:57
  Like, they go and do branding with Nike and those sort of, like, big companies.
• 32:01
  They have the expertise.
• 32:03
  And also, they have the most hilarious personalities.
• 32:07
  It's a man and Camille and they argue live on the show.
• 32:11
  And I'm sat there like, I don't know if I should be listening to this.
• 32:14
  So it's almost like a combination of like expertise being super relevant and some like comedy, like BTS-esque in there.
• 32:23
  And it allows me to be smarter.
• 32:24
  That's what I always think about in shows.
• 32:26
  I either want to be entertained or I want to become smarter, as you said before.
• 32:30
  And so for me, they are the most amazing show that I'm watching right now.
• 32:33
  Do you know them?
• 32:34
  Have you met them?
• 32:35
  No, I haven't met them in person, but I have chatted to them before.
• 32:38
  Where can people connect with you if they want to continue the conversation?
• 32:42
  Yeah, I am Grace Experiments on Instagram and then Grace Miller on LinkedIn and both of those on YouTube.
• 32:49
  But if you want to see the most real version of me with all of the failures, it's definitely YouTube.
• 32:55
  Thank you so much for watching or listening to this episode.
• 32:57
  If you're new here, please consider subscribing or leaving us a review on the audio platforms.
• 33:01
  It helps us as a very new show.
• 33:03
  If you want to check out another episode, just look through our library.
• 33:06
  It's starting to build up.
• 33:06
  We're at like 40 episodes now.
• 33:08
  And we have a new one coming out every single Thursday.
• 33:10
  So thank you for being here.
• 33:11
  And I will see you then.