Latest / Code Story | Startup Podcast for CTOs, CEOs and Technical Founders / S12 E14: The "Ship and Pray" AI Trap: Moving Beyond Vibe Checks to Give Product Managers a Code-Free Validation Engine with Catalina Turlea, Co-Founder & CEO of Lovelaice

Transcript

S12 E14: The "Ship and Pray" AI Trap: Moving Beyond Vibe Checks to Give Product Managers a Code-Free Validation Engine with Catalina Turlea, Co-Founder & CEO of Lovelaice · Code Story | Startup Podcast for CTOs, CEOs and Technical Founders

• 0:00
  [SPEAKER_00]: So the very very first version was this small benchmarking tool that I mentioned, which was basically just something that I did on the side as a little project.
• 0:11
  [SPEAKER_00]: When I'm building early stage, what is very important for me and where I see a lot of startups maybe going a bit in the wrong direction is to really focus on what makes your product great and not to get lost in all the technical cool things that you could do.
• 0:28
  [SPEAKER_00]: You don't have to redo authentication.
• 0:30
  [SPEAKER_00]: You can just reuse something that already exists.
• 0:33
  [SPEAKER_00]: And what this meant for me was building everything serverless with on AWS.
• 0:39
  [SPEAKER_00]: My name is Catalina Tulea, and the CEO and founder of Leblas.
• 0:47
  [SPEAKER_01]: This is Code Story.
• 0:49
  [SPEAKER_01]: a podcast bringing you interviews with tech visionaries.
• 0:52
  [SPEAKER_01]: Six, six months moonlighting goes.
• 0:54
  [SPEAKER_01]: It's the last and all of the backgrounds who share what it takes to change an industry.
• 0:58
  [SPEAKER_00]: I don't exactly know what to do.
• 1:00
  [SPEAKER_01]: It doesn't go as to get right.
• 1:02
  [SPEAKER_01]: who built the teams that have their bad company is its team's help each other, which is proud of our team.
• 1:09
  [SPEAKER_01]: Keeping scalability top of mind, all that infrastructure was up there.
• 1:13
  [SPEAKER_01]: Yes, we've been fighting it as we grew up.
• 1:14
  [SPEAKER_01]: Total waste of time.
• 1:15
  [SPEAKER_01]: The stories you don't read in the headlines.
• 1:18
  [SPEAKER_01]: It's not an easy thing to achieve.
• 1:20
  [SPEAKER_01]: To get yourself a deficit of off, try to begin to ride the ups and downs of the start-up line.
• 1:26
  [SPEAKER_01]: Need to really want it.
• 1:27
  [SPEAKER_01]: Not just about technology.
• 1:29
  [SPEAKER_01]: All this and more on code story.
• 1:33
  [SPEAKER_01]: I'm your host Noil Abhart.
• 1:34
  [SPEAKER_01]: And today, how Catalina Turlia is enabling you to streamline your AI implementation from weeks of iteration to validation in days.
• 1:49
  [SPEAKER_01]: Catalina Turlia is originally from Romania, growing up in the countryside there.
• 1:54
  [SPEAKER_01]: Post-getting her bachelor, she moved to Austria for her masters and landed in Germany for 13 years.
• 2:00
  [SPEAKER_01]: She is married with a three-year-old daughter and many, many pets.
• 2:03
  [SPEAKER_01]: She loves to spend time with her family and nature and the mountains.
• 2:07
  [SPEAKER_01]: She used to do a lot of sports, but being a startup founder doesn't really allow for as much running and hiking these days.
• 2:13
  [SPEAKER_01]: She also is into calligraphy,
• 2:19
  [SPEAKER_01]: Catalina has been building products for 14 years and recently was running a small tech consultancy for startups.
• 2:26
  [SPEAKER_01]: What she observed was that a lot of products contained an AI feature, but the feature, quote unquote, was based on a prompt, didn't work well and wasn't a good fit for the users.
• 2:37
  [SPEAKER_01]: Eventually, she and her co-founder realized they saw the same problem and built a platform to support product teams and building valuable AI features.
• 2:48
  [SPEAKER_01]: This is the creation story of Love Lays.
• 2:54
  [SPEAKER_00]: I have a lot of experience in building products and more than 14 years.
• 2:58
  [SPEAKER_00]: In last year, I switched from my previous role and before figuring out what I want to do next, I ran a small tech consultancy for startups, where I worked on more than 10 different projects from a strategic point of view, but also hand on coding.
• 3:14
  [SPEAKER_00]: A lot of these projects had some sort of an AI feature integrated already, but how that look like is that someone had written a prompt, more or less copy paste from somewhere or just a rather basic prompt.
• 3:26
  [SPEAKER_00]: They had chosen a model based on what was popular at the time mostly GPT4, and they released it and it was this entity like alive there in their products.
• 3:36
  [SPEAKER_00]: They knew it's not working.
• 3:38
  [SPEAKER_00]: optimally, but nobody really knew how to interact with it.
• 3:42
  [SPEAKER_00]: And, initially, I just built myself a benchmarking class one.
• 3:45
  [SPEAKER_00]: They would just take one from and run it at the same time across multiple elements, just for me to better understand the technology and to be able to better guide them how to deal with the technology, basically.
• 3:56
  [SPEAKER_00]: And I saw that even with this very small step, they could actually achieve so much more accuracy just by pairing the model together with the problem that they were trying to solve.
• 4:08
  [SPEAKER_00]: At the same time, my co-founder, whose background is in product management, she was working in an AI fintech and she was seeing the same things.
• 4:17
  [SPEAKER_00]: but from a product management side and she was also the first one that actually saw my vision building something that supports teams to build with the near technology in a sustainable way because as an engineer myself it's still mind blowing to me that we do so much testing like unit testing and integration testing and we have.
• 4:39
  [SPEAKER_00]: development and beta and production versions for the terministic code.
• 4:44
  [SPEAKER_00]: But for non-deterministic AI, we'll just take it and ship it to production.
• 4:48
  [SPEAKER_00]: And this I find fascinating.
• 4:51
  [SPEAKER_00]: Lovely is a platform that supports product teams to build AI features based on data and with measurable impact.
• 4:59
  [SPEAKER_00]: We are lowering the technical barrier of what it means to build with AI.
• 5:03
  [SPEAKER_00]: So you can do it yourself regardless of your technical background without any engineering resources needed.
• 5:10
  [SPEAKER_00]: So you come in with your prompt, with your test data set, you run it across multiple LLMs, and then you try to find the best setup for you with real data that you can back your decisions on.
• 5:25
  [SPEAKER_01]: Let's dive into the MVP for lovelace that and it'll be interesting where you start.
• 5:30
  [SPEAKER_01]: But tell me about that first version of the product to belt how long it took you to build and what sort of tools you're using to bring it to life.
• 5:36
  [SPEAKER_00]: So the very, very first version was this small benchmarking tool that I mentioned, which was basically just something that I did on the side of the little project.
• 5:47
  [SPEAKER_00]: I have been building in startups for many years now and I'm a second time founder.
• 5:52
  [SPEAKER_00]: I've already built one product from scratch and scaled it over six years.
• 5:57
  [SPEAKER_00]: And through my experience, I've seen some patterns that work over and over again, or at least that work for me because I think there's no perfect solution.
• 6:05
  [SPEAKER_00]: So when I'm building early stage, what is very important for me and where I see a lot of startups maybe going a bit in the wrong direction is to really focus on what on putting effort in what makes your product great and not to get lost in all the technical cool things that you could do, for example.
• 6:25
  [SPEAKER_00]: You don't have to redo authentication.
• 6:28
  [SPEAKER_00]: You can just reuse something that already exists, so you can just focus on the core of your product.
• 6:33
  [SPEAKER_00]: And what this meant for me was building everything serverless with on AWS, with like as many managed services as possible, but it can actually just focus on writing the actual code that makes love less what it is.
• 6:47
  [SPEAKER_00]: One important aspect of love lies because we target non-technical users.
• 6:53
  [SPEAKER_00]: We need the UX to really state or to make it better than what is out there in our in the other platforms.
• 7:02
  [SPEAKER_00]: Only enough my first version of the platform because I am an engineer looked very similar to the other platforms.
• 7:08
  [SPEAKER_00]: That's why these are built from engineers to engineers.
• 7:12
  [SPEAKER_00]: So, maybe in this aspect, was you would say the MVP quality has changed a little bit because we're constantly trying to see, okay, where is friction in this process that we can take out, rather than what features we still need to add.
• 7:27
  [SPEAKER_00]: It's more, let's focus on making with smooth, that is one part that is critical to the product, which you would normally not, let's say, make it a priority in an MVP.
• 7:39
  [SPEAKER_01]: Tell me about maybe a decision or trade off you had to make in, you know, making that smoothness happen, right, or moving some of that friction, a decision or trade off on that early version of the product to make it happen or maybe even some like, you know, feature limitation or, you know, acceptance of tech debt or anything like that.
• 7:58
  [SPEAKER_01]: Tell me about one of those you had to work through.
• 7:59
  [SPEAKER_01]: I cope with the decision.
• 8:02
  [SPEAKER_00]: There's so many ways in which you can experiment with AI, and because it is not the terministic, any of this experimentation aspects might bring you them surprising new insight.
• 8:16
  [SPEAKER_00]: But the challenge is if you put
• 8:18
  [SPEAKER_00]: 20 different configuration toggles in front of a user that's just starting, they're just going to get lost.
• 8:25
  [SPEAKER_00]: So this is maybe one of the biggest challenges that we are solving, still is how do you make it flexible enough that people can
• 8:37
  [SPEAKER_00]: Still, try out all the different combinations, but not overwhelming for someone just turning over, or just turning into the field.
• 8:45
  [SPEAKER_00]: That is definitely something where we're trying to take out as much as possible, and still, okay, we still have the possibility to do this, very complicated thing, and this very complicated thing.
• 8:56
  [SPEAKER_00]: But let's focus on
• 8:58
  [SPEAKER_00]: The first experience of lowering the time from starts to this first aha moment.
• 9:03
  [SPEAKER_00]: So that's maybe the kind of compromises that we have been doing so far.
• 9:09
  [SPEAKER_01]: Okay, let's move forward then.
• 9:10
  [SPEAKER_01]: How are you progressing and maturing the product and kind of taking it to the next level?
• 9:14
  [SPEAKER_01]: And grab that question in a box form and look, how do you build your roadmap?
• 9:18
  [SPEAKER_01]: How do you go about deciding that?
• 9:20
  [SPEAKER_01]: Okay, this is the next most important thing to build or to address with Levelace.
• 9:25
  [SPEAKER_00]: We spent the best part of this year validating our hypothesis, so we talked to maybe 50 product teams to see which of the pain points that we identified resonate with their experience.
• 9:38
  [SPEAKER_00]: And try to understand where it are, their biggest pain points at the moment.
• 9:42
  [SPEAKER_00]: And this was, of course, or still is, a very big source of inspiration or guidance for our roadmap.
• 9:49
  [SPEAKER_00]: One thing that I have learned throughout all my startup experiences is also that you shouldn't build for what you think people need but actually what people really want.
• 9:59
  [SPEAKER_00]: So we have a lot of ideas how things could look like, but we try not to implement it from our best understanding but if possible to have implemented first on an actual use case, I will give an example.
• 10:13
  [SPEAKER_00]: We have a built-in evaluation framework in the platform, so it does deterministic evaluations, it does error analysis, but also LLM as a judge.
• 10:23
  [SPEAKER_00]: And we actually built this as part of a feedback process with one of our pilot customers.
• 10:30
  [SPEAKER_00]: And I think the result is different than if we would have built it from our best assumptions beforehand, if that makes sense.
• 10:39
  [SPEAKER_01]: Okay, here he's saying, we, how'd you go about building your team?
• 10:42
  [SPEAKER_01]: What do you look for in those people to indicate that they are the winning horses to join you?
• 10:47
  [SPEAKER_00]: my co-founder, she's also my sister.
• 10:50
  [SPEAKER_00]: She was the first one that actually was able to understand my idea and see the potential behind it.
• 10:57
  [SPEAKER_00]: And we basically both of us spent our career building impactful products.
• 11:02
  [SPEAKER_00]: She worked in fin tech, I worked in health tech.
• 11:05
  [SPEAKER_00]: It's been the red thread throughout our careers.
• 11:09
  [SPEAKER_00]: And since it's also my sister, I know her really well.
• 11:12
  [SPEAKER_00]: It just makes
• 11:16
  [SPEAKER_00]: This was a bit like a dream come true.
• 11:19
  [SPEAKER_00]: In terms of the rest of the team, I'm a very big fan of building very small efficient teams.
• 11:25
  [SPEAKER_00]: I don't believe in growth or the sake of growth.
• 11:28
  [SPEAKER_00]: Even right now it's not a topic, but also back a few years back when everybody, what's the only thing that mattered was head count.
• 11:37
  [SPEAKER_00]: That's not the sort of value that I relate to.
• 11:40
  [SPEAKER_00]: I really like to build teams where everybody believes in the vision, but also where everybody has a chance to develop themselves.
• 11:48
  [SPEAKER_00]: Further, to learn something and to keep growing in the role or exploring new possibilities.
• 11:54
  [SPEAKER_00]: Going from front end to full stack or to back end development, I've always encouraged such transitions or learning path.
• 12:03
  [SPEAKER_00]: So yeah, right now we have a fully female team.
• 12:08
  [SPEAKER_00]: Ali, our co-founder Anton, whose background is in B2B, he's the only man on the team, at the West, who are all women.
• 12:18
  [SPEAKER_01]: Let's move into scalability.
• 12:20
  [SPEAKER_01]: And this will be interesting.
• 12:21
  [SPEAKER_01]: I'm curious about how you approached scalability from the beginning, and have there been interesting areas where you've had to fight scale as you crown.
• 12:29
  [SPEAKER_00]: So by taking the approach with serverless, there's two main reasons why I prefer this.
• 12:36
  [SPEAKER_00]: First is because as early stage when you don't have so many traffic, you actually only pay for what you use, so it's much more convenient.
• 12:46
  [SPEAKER_00]: But also because it is serverless, you actually have this implicit scalability that built into your service.
• 12:52
  [SPEAKER_00]: As I mentioned in my previous startup, the base was also the same and is killed from 100 users to 100,000 users with the same no problem.
• 13:02
  [SPEAKER_00]: Of course, they were architectural changes to the infrastructure, but the backbone of it still stayed the same.
• 13:09
  [SPEAKER_00]: With Levele's dough, there is one part of scalability that I maybe did not expect in the beginning and it's related to the LLMs, because they do enter, they can run for a long time.
• 13:22
  [SPEAKER_00]: So Levele's is a big aspect of it, and serverless is made for quick responses, quick feedback, and so on.
• 13:31
  [SPEAKER_00]: So there I had to rethink my approach a few times and see, okay,
• 13:36
  [SPEAKER_00]: How do we run things in parallel, but not trigger any API limits at the same time, not let the user wait too long, and how do we do this at scale like how the scale being the amount of LLM calls you run at once.
• 13:52
  [SPEAKER_00]: So you can, in theory, can run as many as you want.
• 13:55
  [SPEAKER_00]: So how does it perform with 100 with 1000 with 10,000?
• 13:59
  [SPEAKER_00]: How does it look like as it is everything still working in?
• 14:02
  [SPEAKER_00]: If something breaks, how do you find it out?
• 14:07
  [SPEAKER_01]: So, as you step out on the balcony, you look across all that you've built thus far with love-wise, what are you most proud of?
• 14:13
  [SPEAKER_00]: We're still very early stage.
• 14:18
  [SPEAKER_00]: What really pushes me, what gives me the motivation, is right now I'm actually leading the company, so I'm also the CEO and the CEO, and I do everything from fundraising to sales to...
• 14:32
  [SPEAKER_00]: content, marketing, outreach, I don't do it alone, but I am part in all of this and also in the strategy behind it, which was not the case before.
• 14:41
  [SPEAKER_00]: It was more focused on the technical side.
• 14:43
  [SPEAKER_00]: So this for me is it's a huge learning opportunity and I've learned so many new things.
• 14:50
  [SPEAKER_00]: And it's, of course, very exciting to see how to tackle this topics that are not related to tech and how we can do it in a way that makes sense for the business.
• 15:00
  [SPEAKER_00]: And what I'm really proud of is that we are a female founded company.
• 15:05
  [SPEAKER_00]: So we have two female founders on board.
• 15:07
  [SPEAKER_00]: I think that's pretty cool.
• 15:09
  [SPEAKER_00]: And as I said, I also have a daughter and one of my main motivation is to be a role model and make sure there's female founder faces out there that hopefully she can relate to when she
• 15:23
  [SPEAKER_01]: Okay, let's flip the script a little bit, tell me about a mistake you made, and how you and your team responded to it.
• 15:30
  [SPEAKER_00]: When you have the idea and you start looking into the market and okay, there's nobody doing this.
• 15:36
  [SPEAKER_00]: And you write like your little pitch deck, the first version of it, and some sort of a kind of hour you try to do your best.
• 15:43
  [SPEAKER_00]: Then you start sending it around and you have this impression that people are going to immediately see what you're building and just jump on it from the beginning.
• 15:54
  [SPEAKER_00]: And if I look now at the first pitch deck that I sent, and probably all these investors that I burned.
• 16:01
  [SPEAKER_00]: because they send them something that right now we probably be embarrassed of.
• 16:05
  [SPEAKER_00]: It's just like the process of or like the learning process that you go through from the beginning until you get more validation on your idea and more feedback and understand how to phrase it in a way that people actually understand what it is and how to sell it to customers and how to sell it to investors because it's two different things.
• 16:26
  [SPEAKER_00]: So yeah, I would say that's probably, if I could do something different, I would probably not sense not have sent those early emails.
• 16:34
  [SPEAKER_00]: So early stage and probably sent them with a bit more meat on the bone about now.
• 16:46
  [SPEAKER_01]: Okay, let's move forward then.
• 16:48
  [SPEAKER_01]: What does the future look like for
• 16:58
  [SPEAKER_00]: As we look into how product teams are building currently with AI, there's still a lot of, let's say, unstructured processes.
• 17:06
  [SPEAKER_00]: And it makes sense because it's a new technology, everybody's figuring it out.
• 17:11
  [SPEAKER_00]: And that makes a lot of sense that it's also so easy to get started, right?
• 17:16
  [SPEAKER_00]: AI always returns something.
• 17:17
  [SPEAKER_00]: So it has this magic that what a sort of pulls you in.
• 17:22
  [SPEAKER_00]: I can make up a seemingly very good answer very fast that you can already integrate me into your product.
• 17:29
  [SPEAKER_00]: I do believe that as time goes by and we start getting, as users start getting, maybe a bit fit up with just AI slot everywhere, actually building features that have an impact and they deliver value to your users.
• 17:47
  [SPEAKER_00]: beyond just the AI high-play label, I think it will be what makes will make products survive in the next years.
• 17:57
  [SPEAKER_00]: Like very generic chat, but it's not going to be enough.
• 18:00
  [SPEAKER_00]: It's already not enough, but let's say market penalty for that is not yet that big.
• 18:06
  [SPEAKER_00]: But the more products raised is bar, the lower, the bigger the difference is going to be in the more obvious.
• 18:13
  [SPEAKER_00]: the gap between product teams that actually have a systematic way of testing their features and they want to deliver expert level insights confidently and with data and the product teams that just ship AI to say that they have something.
• 18:30
  [SPEAKER_00]: We really see the level is becoming a backbone of this process of building products.
• 18:35
  [SPEAKER_00]: Similar to how product analytics became a backbone of productivity development, right?
• 18:40
  [SPEAKER_00]: Before product analytics it was management that said let's do this feature or the button should be here and once we had amplitude and mixed panel they actually proved them wrong on every second decision and we started building product step actually
• 18:57
  [SPEAKER_00]: deliver value and we can measure that value for the users.
• 19:01
  [SPEAKER_00]: And with AI, it's a bit more than just tracking the data.
• 19:05
  [SPEAKER_00]: AI, drastic, or changes the economics of software, right?
• 19:11
  [SPEAKER_00]: Before, if you would just get one more user on your platform, it would actually cost you nothing.
• 19:16
  [SPEAKER_00]: maybe not nothing but very small infrastructure cost right now with AI every single usage costs to something and your most dedicated users are actually the ones that cost you the most.
• 19:29
  [SPEAKER_00]: I don't know if you know this but the CEO of Loveable posted this a story in a few months back.
• 19:35
  [SPEAKER_00]: that when they launched loveable in the beginning on a 20 dollar a month subscription, someone prompted the app for a 36-hour straight, basically summing up AI credits much more than the subscription and this is something that still products new to understand that AI is cutting through your margins and you should implement it only in parts that bring you a return on investment and they are a valid business case for you.
• 20:04
  [SPEAKER_01]: Let's switch to you.
• 20:05
  [SPEAKER_01]: Who influences the way that you work?
• 20:08
  [SPEAKER_01]: Name a person or many persons or something you look up to and why.
• 20:12
  [SPEAKER_00]: So there's a lot of female founders that I have in my network that have achieved great things, and there's sort of dedication and determination really motivates me to keep going.
• 20:25
  [SPEAKER_00]: And yeah, I'm very close to the startup.
• 20:28
  [SPEAKER_00]: Let's say startup ecosystem in a unique way.
• 20:30
  [SPEAKER_00]: There's a lot of examples of that.
• 20:32
  [SPEAKER_00]: I'm also very, a very big mental health advocate.
• 20:37
  [SPEAKER_00]: So having built a mental health company before,
• 20:39
  [SPEAKER_00]: For me, it's also a lot of these things start with me, also after leaving.
• 20:45
  [SPEAKER_00]: So building in a way that is sustainable, that as you also start up in family life, it's a lot to handle, and you still have to make sure you have your priorities straight.
• 20:56
  [SPEAKER_00]: So I also admire a lot of the start-up founders or founders in general, doesn't have to be start-up.
• 21:03
  [SPEAKER_00]: that actually can put this together and show up in both aspects of their lives, which I find really impressive and inspiring for me.
• 21:14
  [SPEAKER_01]: Catalina, last question, so you're kidding on a plane, and you're sitting next to a young entrepreneur who's built the next big thing.
• 21:20
  [SPEAKER_01]: They're jazzed about it that can't wait to show it off to the world, and can we show it off to you right there on a plane?
• 21:25
  [SPEAKER_01]: What advice do you give that person having gone down this road a bit for 14 years now?
• 21:31
  [SPEAKER_00]: So first is talk to people and validate your idea.
• 21:35
  [SPEAKER_00]: I think this startup world is filled with good ideas that never took off.
• 21:40
  [SPEAKER_00]: There's three key aspects into making a startup work, and two of them are up to you, and one is out of your control.
• 21:48
  [SPEAKER_00]: So it's the product, the team, and then it's the timing.
• 21:52
  [SPEAKER_00]: So you can very, very strong and have the perfect product in the perfect team.
• 21:57
  [SPEAKER_00]: But if you have are there at the right, at the wrong timing, there's really nothing you can do.
• 22:04
  [SPEAKER_00]: So my biggest advice would be to validate before building anything.
• 22:10
  [SPEAKER_00]: Probably we don't know what the future is going to look like with the eye.
• 22:13
  [SPEAKER_00]: I think it's hard for everybody to imagine it.
• 22:16
  [SPEAKER_00]: But still, making sure that people are willing to pay for something before you actually start building it, it's a fantastic way to actually identify the pain points.
• 22:26
  [SPEAKER_00]: And you do have to pivot a lot in the beginning to match
• 22:31
  [SPEAKER_00]: Your skills and the problem that you're trying to solve with exactly how the pain point looks like in real life.
• 22:38
  [SPEAKER_00]: And that might be phrasing it differently, but also tweaking the product a little bit, tweaking the audience, so that you actually match the two.
• 22:46
  [SPEAKER_01]: That's fantastic advice.
• 22:47
  [SPEAKER_01]: We're Catalina.
• 22:48
  [SPEAKER_01]: Thank you for being on the show today.
• 22:49
  [SPEAKER_01]: Thank you for telling the creation story of Love Lace.
• 22:53
  [SPEAKER_00]: Thank you so much for having me.
• 22:57
  [SPEAKER_01]: It's been a pleasure.
• 23:06
  [SPEAKER_01]: Code Story is hosted and produced by Noah Labhart.
• 23:10
  [SPEAKER_01]: Be sure to subscribe on Apple Podcasts, Spotify or the podcasting app at your choice.
• 23:15
  [SPEAKER_01]: And when you get a chance, leave us a review.
• 23:17
  [SPEAKER_01]: Both things help us out tremendously.
• 23:22
  [SPEAKER_01]: And thanks again for listening.