Latest / The Experimentation Edge / How Preply runs close to 1,000 experiments a year

Transcript

How Preply runs close to 1,000 experiments a year · The Experimentation Edge

• The Experimentation Edge - Sebastien Polis Recorded 2026-08-28 [00:00] Sebastien Polis: Losing is the norm, right? Losing is just what you're going to encounter. If you're building an experimentation program where you are embarrassed of failure, then your program's going to end up lying to you, and people are going to find ways to just claim spurious wins. So losing is what you should expect. And at Preply, I would say probably about 85 to 80% of our experiments fail in some way or other, right? They don't get scaled, they don't get scaled up to production. But what does happen, and what we do encourage people, is to iterate, to learn from the failure, to understand what happened, and make sure that they take those learnings into the next iteration of a particular experiment. [01:14] Ashley Stirrup: Hello and welcome to today's episode. I'm excited to have Sebastien Polis, Senior Experimentation Manager at Preply. Welcome to the show, Sebastien. [01:23] Sebastien Polis: Hello, thank you for having me. [01:25] Ashley Stirrup: Yeah, really excited to have you on the show. Why don't we start with you telling us a little bit about Preply? [01:32] Sebastien Polis: Yeah. So at Preply, we're all about powering people's progress, right? Our mission is quite simple. It's honestly connecting people through learning and helping them go further than they thought possible, right? So that's what we're all about at Preply. [01:46] Ashley Stirrup: Got it. And could you tell us a little bit more about the kinds of services you provide? [01:52] Sebastien Polis: Yeah. So Preply is a learning marketplace, primordially language learning, but that's not exclusive. So you can come to Preply to find a brilliant tutor to teach you Portuguese, Spanish, French, German, English, chess, business, data science, whatever example you can imagine. We have over 150,000 bookable tutors, millions of learners. So yeah, you can pretty much find whatever you want to learn at Preply. [02:19] Ashley Stirrup: Yeah, that sounds amazing. How about your role? Can you tell us a little about your role? [02:24] Sebastien Polis: Yeah, so I'm the experimentation manager here at Preply. It basically means that I run the experimentation team here. At Preply we have our own program, our own platform and tooling for running experiments. And so my role is basically ensuring that we are continuously pushing the experimentation capabilities of Preply, building up the culture and safeguarding what we've already built, and making sure that we are rigorous and speedy in the way we run our experiments to ensure that we are actually building for our customers. [02:58] Ashley Stirrup: Got it. And how long have you been in the experimentation world? [03:04] Sebastien Polis: Exclusively in the experimentation world, I'll probably say about 5 years now, give or take. In the wider data science world, probably about 9-ish years. So I first came across experimentation when I was working as a data scientist at DAZN back in 2018, 2019. That's where I encountered A/B testing as a concept and started running some very basic, simple experiments at the time. And then as I've progressed my career, I've kind of gone deeper into this particular domain because I find it really, really interesting to, you know, understand what works, what doesn't work, and apply science to product development. And so yeah, experimentation has been the main focus of my career for the last 5 years, particularly at Dojo, my previous role, and now at Preply, where I run the experimentation team. [03:55] Ashley Stirrup: Got it. And you work with a pretty wide number of different teams across Preply, right? [04:02] Sebastien Polis: Yeah, yeah. So Preply, you know, we have about 800 employees across the world and in 4 main hubs. We are in London, Barcelona, Kyiv and New York. I think we have something around 25 to 30 product teams in different domains, be it in growth, be it inside the product experience, the classroom, you know, ranking, retention, customer support, et cetera. We have a lot of different areas in the product, right? And at Preply experimentation is the norm. So there's not a change that will go through to our customers that is not experimented on, or at least some sort of staged A/B rollout. So yeah, everything we do is an experiment, and that sets the bar quite high for my team's capabilities and support. [04:52] Ashley Stirrup: Yeah. And with that many teams, how do you create a culture of experimentation and keep the same rigor across teams? [05:00] Sebastien Polis: Yeah. So in that regard, I'm very thankful that experimentation was something that was baked into Preply from the very, very start. Our founders are very big believers in experimentation, and they've always encouraged Preply to use experimentation from the very, very start. And so when I joined Preply about 2 and a half years ago, we already had a pretty strong culture and pretty strong tooling to enable experimentation. And in that regard, we have continuously been building on that. So we do it in many different ways. We first of all have the tooling aspect, where we make sure that our tooling is as frictionless as possible, right? Where experimentation is truly just an extension of every developer's toolkit and just how we run the product and engineering org. So in that regard, it's pretty core. The other way we do it is that, because experimentation is so central, we attach all of our measurement frameworks and OKR attainment and accountability of how we evolve as a team, as a business, to the experimentation platform. Right. So to give you a clear example of that, every product team in Preply will be given a set of objectives for their quarter. And those objectives are only attainable via the launching of experiments and claiming results based on the experiments. And those claims can only be made if a certain experiment is using a particular target metric. So we're very much controlling the culture and controlling the rigor, if you want, from how we are setting our OKRs and how we are setting the processes as a business. [06:39] Ashley Stirrup: Yeah. Wow, that's a high bar. How do you balance that with creating a culture where it's okay to have losing experiments? Because, you know, that's often how you learn. [06:50] Sebastien Polis: Yeah, for sure. I mean, I think if you're setting up an experimentation program, you need to be accustomed to losing. Losing is the norm, right? Losing is just what you're going to encounter. If you're building an experimentation program where you are embarrassed of failure, then your program's going to end up lying to you, and people are going to find ways to just claim spurious wins. So in that regard, I'd say, you know, losing is what you should expect. And at Preply, we lose a lot. I would say probably about 85 to 80% of our experiments fail in some way or other, right? They don't get scaled, they don't get scaled up to production. But what does happen, and what we do encourage people, is to iterate, to learn from the failure, to understand what happened, and make sure that they take those learnings into the next iteration of a particular experiment. To give you an idea, I would say about 30 to 40% of experiments that we run in a year are iterations of their original idea. So it's very common that in the experimentation platform you'll see the experiment name and dash v1, v2, v3, all the way up to v5. Because teams are launching a hypothesis, seeing the results, understanding why it didn't work, refining the hypothesis for the next iteration, trying again, trying again. Until either it succeeds and we have very strong evidence, or until we say, hey, this is no longer worth pursuing, we should change course and try something else because this idea clearly has no legs, right? So yeah, I'd say we accept losses, but we just don't give up. We learn from them and we move on. [08:35] Ashley Stirrup: Got it. And are you able to attach an ROI to your investment in experimentation? [08:43] Sebastien Polis: Yeah, I mean, I can give you very rough numbers, right? I won't reveal all the details, but we run a global control group to measure our uplifts across all experiments. And I'd say, you know, on a good year, we will be in the double digits in terms of overall growth on our key metrics. You know, I'm not saying 50%, but definitely double digits. And we've seen that to be fairly consistent. Some years not as good as others, but yeah, it's good to know that there is actual real merit in the things that we're scaling, and it's not just a bunch of false positives one after the other, right? [09:20] Ashley Stirrup: Yeah. Well, that's great. And then of course there's always the benefits of avoided losses as well. [09:27] Sebastien Polis: Of course, of course, right? Yeah. Who knows how much spaghetti code would be put in the code, or how much bloat there would be in the product, if we were scaling everything we've tried. As I said, about 80% of the things we try never make it to the final product. So yeah, who knows how many different directions we would have gone had we not had an experimentation program running. [09:46] Ashley Stirrup: Right. Do you have an example of an experiment you've run where you've had a lot of learnings? [09:52] Sebastien Polis: I can give you maybe not a specific experiment, but I can give you some learnings we had this year around control groups and how to manage them, and some learnings we got from some experiments. So as I said, we run a lot of control groups at Preply. And what happened this year is we launched one of them, and within a month we were seeing very negative results across business metrics. And that was really surprising to us, because we had only successfully scaled a few experiments, right? So they had run and we had said, yeah, these are winners and we should scale them to the users. And we were very surprised as to why we were seeing such negative results. And obviously control groups are meant to establish long-term performance, so a lot of senior leadership were looking at these and raising questions: what's going on? So my team did a very long and exhaustive deep dive. It cost us a lot of time that could have gone to other initiatives towards further experimentation in the company. And ultimately what we uncovered was this was mostly the result of incorrect allocation of users, or biased allocation of users by random chance, right? So yeah, we didn't suspect that our coin flip was wrong or anything, but we were in that, you know, bad luck scenario where a lot of the users in one side were just poor users that came across. And so the key learning was, if you're going to be running something that is high stakes, that is meant to explore the overall impact, you should make every effort possible in making sure that it is as balanced as possible. You know, I don't know if you have the seeding capability to balance your control group from existing users, or a stratified sample, or something of the sort, but be ready to have that. And the other thing that I learned is to probably have a lot of diagnosis capabilities ready in hand to be able to explain these sorts of things, because it can be very expensive time-wise to have these sorts of issues. The other thing we uncovered in this process is that we had scaled a few experiments, and we observed the impact of these experiments post-scale. So we have an ability to understand how they're performing after they were scaled. It's not a perfect comparison like an A/B test, because it's sort of all the experiments versus none of the experiments. [12:10] Ashley Stirrup: Sure. [12:11] Sebastien Polis: But we were surprised that some of them were quite poor compared to the original runtime of the experiment. And when we looked deeper into them, these were experiments that should have raised an eyebrow in terms of the results. So to give an example, we were changing some of the wording around our matching flow, to find you the right tutor, to change things like, for example, budget to price and things like that. So very minor changes that we thought were going to be more aligned to different users and so on. And what we found is that these changes were increasing conversion rate to new subscribers, and we were like, wow, that's fantastic, let's scale this. But once we scaled, we weren't actually seeing that, right? [12:56] Ashley Stirrup: Yeah. [12:57] Sebastien Polis: And then what we did is we said, well, let's unscale these experiments, rerun the test, confirm that the result was real. And then what we saw was the result was completely flat. So what we had scaled was positive, and that meant that our expectation was the control group should be trending positive, but actually it's very, very negative. What is going on? We don't understand. And I think that's one of the learnings that we got: if you have a nice but surprising result, you know, you're changing some simple wording and seeing a big change in one of your key business metrics, you probably just want to double check that. Rerun the test, take the time, because it's going to save you a bunch of headaches in the future. So yeah, that's one of the learnings now: if you run something simple, you shouldn't expect big, big business changes. [13:44] Ashley Stirrup: Yeah, we had a webinar at GrowthBook with Ronny Kohavi where he talked through running trustworthy experiments, and he talked about a lot of the things you mentioned. There's always the "if it sounds too good to be true, it probably is" type of... [13:57] Sebastien Polis: Yeah, it's Twyman's law, right? [13:59] Ashley Stirrup: Yeah. That's right. That's right. [14:01] Sebastien Polis: I think that's Twyman's law. Yeah. So we've encountered Twyman's law ourselves. [14:08] Ashley Stirrup: Yeah. And going back to your example with the situation with the assignment, yeah, assignment can be super tricky. There's a lot of ways people can get themselves in trouble. And Ronny's mentioned studies back from the Microsoft days where they went back and looked at experiments and uncovered biases that really skewed results in a number of experiments. And you know, if Microsoft is struggling with it, I think we can all know that we need to pay extra attention to making sure you get that right. [14:38] Sebastien Polis: Yeah. The basics have to be in place, right? And you should always have a healthy dose of, you know, what's the word? You should question those assumptions every so often and just test them and check that, yeah, it's still holding true. Because if you build a whole thing on shaky foundations, then you're going to have a lot of difficult times later. [14:57] Ashley Stirrup: Yeah, and it can undermine trust in the whole program. [14:59] Sebastien Polis: Very much so, right? And it all runs on trust at the end of the day. [15:03] Ashley Stirrup: That's right. Yeah. So important. So let's say you're working with somebody new to experimentation, a new team member. How do you help them design an experiment to make sure you get the most learning? I mean, often with somebody new, they're excited about their feature. They're just sure it's going to be a winner, they're ready to test it, and they aren't thinking about the loss and the learning side of things. How do you help them design an experiment to get the most? [15:41] Sebastien Polis: So that's a great question. I guess I'd preface this by saying my team necessarily is not involved in the design of individual experiments as much. We get involved in the complex, you know, high-impact, high-visibility ones, just to make sure that it's all correct. Probably a new data scientist, a new PM, anybody who joins will join a seasoned squad of experimenters that know the drill, know how to work it, know how to make their experiments run smoothly and correctly, right? So the way it works usually is the product team will design their experiment. They will have a very strong hypothesis. What we require them to do is to explain what they know, what is their supporting evidence for this particular experiment, what is the reasoning behind why a particular experiment will work. And we're usually looking there for numbers, we're looking for analysis linked to Amplitude, Looker, whatever tooling that we're using at the time, to support that this particular experiment will work. A lot of influence as well from our UX research team, who, you know, will have a strong input on features. And then we'll get the team to write a hypothesis. We'll be very strict now about having a target metric that is well reasoned, supporting metrics, guardrail metrics in the experiment. And then, you know, you'll have your MDE calculator, all your standard experiment hygiene concepts, and then the experiment will run. And what we want at the end is, we are ultimately very strict in that you must follow a pretty well-defined set of rules about what can be scaled and what cannot be scaled. So you must meet significance in your target metric. You must not have any negative impact on any of the guardrail metrics, which are also the business North Stars. And only if that happens can you scale. And if you don't meet any of those criteria, then you need to do a very in-depth analysis as to why. And so that's the process that we follow generally to launch and make sure that an experiment is successful in terms of generating learnings and generating institutional knowledge. [17:53] Ashley Stirrup: Yeah, clearly a lot of rigor is going into it. [17:57] Sebastien Polis: Yeah, yeah. And maybe I could give another example. Preply is a very transparent, flat organization, and we have a Slack channel where every product team will post about the experiment they're launching and the experiment results, right? And everybody there has to explain why they're doing it, what they're doing, why they think this will succeed. Or if it went wrong, why did they kill it? What happened to the experiment? What did they observe? How are they going to take the learnings to the next iteration? So this is a very public forum. Everybody's in there. Our exec team is also present there and will come in and question the PMs about a certain experiment or a certain result, or why are we scaling this, how we investigated this particular idea, and so on. So not only do we have a very rigorous process, but I think we have done a really, really good job as a business to create the culture where we're happy to share our ideas, we're happy to share our failures, we're transparent, we put everything on the table, and we communicate about what we're doing. [19:02] Ashley Stirrup: Right. [19:03] Sebastien Polis: And I think if you're somebody who joins Preply, you will see this very high bar in terms of transparency, accountability, and communication, and you'll want to match it, right? That's what you'll do. And I think that's the cornerstone of why we're so successful in terms of experimentation, of how we can manage to run close to 1,000 experiments a year. It's because everybody buys into the methodology and is a participant in it, even if they're not running experiments. Because, you know, if you're the COO or CPO, you're not running individual experiments, but you will look at the results, you will question your teams. And I think that's what it takes to really embed experimentation across an organization. [19:45] Ashley Stirrup: Yeah. Boy, you can tell just how important it is to the business, the fact that you've got, it sounds like, multiple senior leaders really paying attention to this. [19:55] Sebastien Polis: Very much so. Very much so. Yeah. [19:57] Ashley Stirrup: Yeah, I mean, it makes sense. If experimentation is leading to double-digit growth, it's something that as an executive you should be paying attention to. [20:06] Sebastien Polis: Yeah, yeah. I mean, maybe as a small anecdote I can share that one day, I think it was a year ago or so, one of the execs just links me a small experiment and says, "Why are we running this test? This test seems to have such a small sample size." And I was almost amazed, because I had not even known that this test was running, because, you know, we run quite a few hundred at any single given time, so I'm not looking at every individual test. But this person had actually gone and looked at it, and then messaged me directly: why have we allowed this to happen? And I was like, wow, this is fantastic that they are doing this, that they care enough about going through the detail, and care enough to flag it, right? [20:48] Ashley Stirrup: Yeah, yeah. I want to come back to the learning question again, because especially with the fact that you're iterating as many times as you are, I would imagine that for a lot of features you say, okay, well, the North Star metric is revenue, but the first thing I expect to see is more clicks or more engagement or use of a new feature or something like that. And so when it doesn't work, I imagine you're looking at all those micro leading indicators to figure out why this didn't actually lead to the lift, and what part of the behavior did we change and what part didn't we, and how should we think about iterating next? [21:29] Sebastien Polis: Yeah, very much so, right? I mean, even if you have great click indicators, you're not going to be able to scale unless, you know, there's a very strong foundational reason: technical reasons, security, whatever. But if your product is great at, say, getting people to the end of the funnel, but the people don't buy, then we're like, well, we're probably not going to scale this, right? We're going to go back to the drawing board and understand why it is that they're not taking the final leap, why they're not buying the thing that we want them to buy, or subscribing to the tutor. [21:58] Ashley Stirrup: Right. [21:58] Sebastien Polis: Yeah. And so, you know, there will be a pretty deep dive. There'll be an attempt to understand why that has happened. And then a good team will go and say, okay, well, we'll iterate, right? We think we have a new idea as to why the thing that we want to happen is not happening, and we'll iterate. Maybe we'll change something at the actual checkout page and see if that works. And eventually, at one point, we might say, hey, we might need to pivot away from this because none of the things we've tried works. But we don't throw in the towel quickly, right? We'll try multiple times, multiple angles. And sometimes we do find winners, right? Sometimes, after tweaking a few different things, we do uncover that it was this particular insight that actually leads to users converting. [22:46] Ashley Stirrup: Right. Yeah. And those kinds of insights can be so valuable, right? When it's the fourth or fifth try. Like, well, okay, what do we do then? And how do we make sure we remember that for the next experiment, type of thing. [22:57] Sebastien Polis: Yeah. Yeah. And in that way, I think our product teams are fantastic at doing this. They are really good at capturing the learnings at a tribe level. So every quarter, as far as I know, they go and say, hey, from these experiments, these are the key things we learned, right? And they have very clear statements about learnings that we have about user behavior, and overall paradigms that we use to develop the product in further instances. [23:23] Ashley Stirrup: Yeah. Yeah, I think that is such an interesting topic, because, you know, an organization like yours, you're running 1,000 experiments. Even if you dump all those into AI and say "find the patterns," AI's probably not going to find all the patterns, or the right patterns, things like that. And so I think that's a really interesting next step for the industry: how do we help you take all these experiments and create more learnings out of them, find patterns across experiments, all that? I think there's a lot of opportunity around AI, but I think humans are going to need to do things to help with that process as well. [24:00] Sebastien Polis: For sure, for sure. And that's one of the things that we're working towards at Preply in how we evolve experimentation: making sure that we're a lot more systematic about capturing our learnings, right? As I said, today it happens at the squad level, and maybe the tribe, the domain. So you'll have growth, you'll have the teams that are involved in the classroom, and so on. I think one of the things that we're trying to push now is to really develop our learning system: systematically capturing these learnings, making sure that we have metrics attached to them, confidence, where we have ideas like, hey, here's an experiment that's confirmed a previous learning, we now have a lot more confidence in this particular one, let's try it again and see if we can further confirm it. And then it kind of elevates in status. And, you know, the future is, once you have a very well-curated data set of what the learnings are, with certain degrees of confidence and so on, that's where I think you can actually start thinking about feeding it into an AI, an LLM, and understand, okay, well, what can you do with that kind of power? Right. But in today's world, context is key, and I think that's what we're looking to build at Preply: building excellent context out of the tremendous effort that our teams put into running well-designed, well-run experiments. [25:20] Ashley Stirrup: Yeah. Super interesting. Are there other things that are on your roadmap where you want to see experimentation evolve at Preply? [25:29] Sebastien Polis: I mean, certainly one of my pipe dreams is that I want to have an experiment entirely run by AI and see what happens when you do that. I can certainly imagine the world in which you have an AI completely ideate something, implement the experiment, you know, the code, run the experiment, and then come with a conclusion of, hey, we should scale this for X, Y, Z reason, and then eventually have a human approve that, because ultimately it's us building the product, not the AIs. But I think that is certainly on the roadmap for a lot of people now. [26:05] Ashley Stirrup: Yeah. [26:05] Sebastien Polis: Right. If you think about all of the data that we're generating, all of the context that we're generating from all these learnings, I can only imagine that that's what is in the minds of a lot of people, and certainly something that we'll try to do. [26:18] Ashley Stirrup: Right. Yeah. Yeah. It's so interesting because, you know, you think about AI and, like, well, what if it gets smarter, da da da da, and at some point, in theory, it's going to get smarter than a human, and so of course it can go do all those things. In today's world, I think AI can be so powerful at accelerating so many of the steps of the process. But at least in my own experience, I wouldn't just set it free. Because, you know, some of the things that I've personally gotten back from coding up AI apps and stuff like that, it's like, this page looks really smart, but I'm not sure it's actually telling me the right thing. [26:57] Sebastien Polis: So no, I think we always have to be in control, right? And I think especially when you're in the world of experimentation, you know, at least I really care about our customers and our users, our learners and our tutors, right? And at Preply, one of our principles is "we do it for the learners," and I think the corollary is also "we do it for the tutors." It's really important that we must be very careful with our craft and really careful with what we do, because we're not only serving a single customer, right? We're not only maximizing for Preply, we're maximizing for the learners, we're maximizing for the tutors. And we're always trying to find the trifecta that's good for all three of us. It's not always possible, right? Sometimes it's only good for two and neutral for one. But yeah, I think we must also always exercise care, right? And we must always be excellent at what we're doing, especially for our customers. [27:54] Ashley Stirrup: Yeah. Well, that's definitely, in general, good business. Not only do you want to optimize for yourself, but you want to optimize for your customers. But yeah, I definitely think it adds a whole layer of complexity when you're optimizing for 3 different parties. So I'm sure that's created some unique challenges for you around how you run experiments. [28:13] Sebastien Polis: Yeah, yeah, for sure. I can maybe share a fairly simple challenge, but quite an interesting one, right? So let me give you a scenario. When you subscribe to a tutor on Preply, you will do most of your lessons within the Preply virtual classroom, right? And it's basically a video call where you have access to a series of tools to enhance your own learning experience. So you have a canvas, you have whiteboards and a bunch of different tools. And we're always experimenting on that surface, right? We're always saying, hey, does this tool work? Do we have bugs? Do we have issues? Does this improve retention? Does it improve conversion? And so on and so forth. Now, one of the peculiarities that comes from having a marketplace is that you're not just randomizing the experience for one side, because both parties have access to the classroom at the same time, right? So if I'm experimenting on the classroom, I cannot randomize the students, because a tutor might have multiple students. And so you will therefore, as a tutor, have a different experience for every single one of your students, which violates a lot of the assumptions that we make in experimentation and also makes for a really poor user experience, right? [29:24] Ashley Stirrup: Right, right. [29:24] Sebastien Polis: And so what that means is that we then need to randomize the tutors and measure the students' outcomes, which is actually not your standard t-test, because you now have a new variable in your metric, a confounding variable, which is the tutor and their quality, right? So different tutors, because of how good they are at teaching, will have different baseline retention rates and capabilities. And it also depends on the subject and so on and so forth. So what we use in those scenarios is a clustered analysis, right? Where you have to account for the confounding variable, and it's not your straightforward t-test. And it's something, for example, that you find in the healthcare sector, where you cannot randomize patients, but you can randomize hospitals, right? And so you measure a secondary outcome that way. And so that's just a simple example of the complexities that come from a marketplace, if you're going to really think about who is being impacted by this experiment and how that experience will differ, right? And how you need to adapt your experimentation program to account for that. [30:31] Ashley Stirrup: Yeah. And it requires a lot more power in those situations too. [30:35] Sebastien Polis: Yeah, I mean, that's one of the big challenges of a marketplace: usually one side, the demand side, is a lot bigger, right? As I said, we have millions of learners, but 150,000-plus tutors. So that's a complexity. Power is lower, sample size is lower, metrics also tend to be noisier because you have a lot more variability around tutors and subjects and so on. And it sets the bar higher in terms of the level of care you have to put in your experiment. So an example of one of the things we've learned in the last couple of years is that you must be really, really careful about the quality of the code and the feature, right? Your experiment cannot have any bugs in it. Because if you have a bug and you need to scale down the experiment for the tutors, chances are that you've already exposed a substantial portion of your eligible sample size to that buggy experience, and you've biased anything that comes downstream, right? If you're launching an experiment that is 50/50, it probably is going to cover all tutors in some way. So we've learned to be really, really careful, you know: take your time, make sure that everything is right, because if things go wrong, then everything becomes a lot more complicated further downstream. [31:55] Ashley Stirrup: Yeah. Well, that really further explains why you have all the rigor that you were talking about earlier. [32:01] Sebastien Polis: Yeah. Yeah. We didn't just learn them out of nowhere. They come from a lot of battle scars of trial and error, right? We've experimented and learned about how to do things well and what to avoid. [32:14] Ashley Stirrup: Yeah. Such an interesting story. Well, thank you so much, Sebastien, for coming on the show. I feel like we learned a ton from your business and from you. So thank you so much for sharing everything. [32:26] Sebastien Polis: No, thank you very much for having me, Ashley. It was a pleasure to chat and to share our story. [32:30] Ashley Stirrup: Terrific. Thank you.