Latest / Machine Dreams / Snowflake’s VP of AI on the Cost, Data, and Future of AI
Transcript
- Sam Adebayo: Most of us now use AI every day without thinking twice about it. But you know, behind every quick answer from ChatGPT or Claude, there's a real cost, a real choice about which system does the work and a real question about whether that system can be trusted or not. Welcome back to Machine Dreams, where we speak with the people building AI and we talk to them about what actually is working, what doesn't work. And what the rest of us are about to experience. My name is Kolawole Samuel Adebayo. I'm a technology analyst covering AI, cybersecurity, and enterprise technology. And as always, I'm here with my remarkable co-host. leah: Hi everyone, I'm Leah Stern, global communications strategist and former venture capitalist. Hey Sam, how are you? Sam Adebayo: Very well, very well. I probably have run out of adjectives to to describe you whenever I try to bring you on the show or introduce you. So I have to look for new words to fill my lexicon. leah: They're always complimentary and great to see you. Hope you're having a great summer. We have a special guest today. We're joined by Baris Gultekin, VP of AI at Snowflake, where he leads the team building the company's AI products. Snowflake is one of the biggest names in the world of enterprise data, the place where a huge number of companies keep and make sense of their information. And Baris sits right at this point where all that data meets. Baris, welcome to Machine Dreams. Baris Gultekin: Thank you, Leah. Thank you, Sam. Nice to be here. Sam Adebayo: All right. great to have you once again. there's a lot of things that we can actually talk about, but I'd like us to begin from, you know, the very beginning. You joined Google back in 2006, before any of this worked, the way it does work right now. So I wanted to get a sense of what was AI like back then and what it is right now, what has changed between that period, especially now that you're at Snowflake doing what you do as VP of AI. Baris Gultekin: Yeah, it's been... it's been remarkable to see the progress, you know. I joined Google in 2006, in the ads world, and at the time the ads system at Google was the state of the art when it comes to machine learning. And we did not wanna call it AI because AI was this kind of, you know, thing on a pedestal and we were building kind of, you know, systems that would predict, you know, which ads users would like and click on. and then fast forward, I worked on Google Assistant, which was now... is now a precursor to Gemini at Google. And it was a very humbling experience because again, it was at a time when the technology didn't quite work. So we had to think about every single use case and codify that, you know, so that for each use case we'd have a, you know, separate work stream, if you will. So it was all stitched together versus the current system now, which is a kind of more general purpose general intelligence that is remarkably more capable than what it was. So so it's been an incredible journey. I love how, you know, technology has evolved so fast, especially in the last few years, like to see what it can do now. leah: And you actually left Google to go to a startup. I myself spent years in Venture working with pre seed and series A founders. And there's this permanent push and pull, I would say. People inside big companies yearn for the freedom of a startup and founders yearn for the stability of a big one. You've actually done both. where were you happier or what was easier and what was harder? Just out of curiosity. Baris Gultekin: Yeah. I mean I was very, very fortunate both at my you know job at Google as well as at Snowflake to go to work on projects that were all kind of zero to one. And it's startups are of course really, really difficult because it's not only zero to one, but you also are in a very kind of... it's a challenging environment to go kind of prove yourself, you know, figure out, get the ball rolling, and and then find product market fit. they have different challenges. you know startups of course have to go fundraise. It's more difficult to recruit. and then the product market fit exploration is always challenging. At the same time the benefit is that you're a small company you can move a lot faster. You can kind of move and change direction a lot faster and that's the biggest advantage. Plus, the ecosystem is incredible. you know, from VCs to other startups that are working closely with one another. that ecosystem is what gets the machine going. and in a large company, of course, the benefit, and I love it, is you know, at Google you have access to users, you have access to data. at Snowflake, similarly, we have customers that love the product, that love the company, and being able to go easily talk to these customers, to understand what works, what clicks, is very, very beneficial. And of course, the resources of the company also helps you know, great talent also helps. so there's a lot of you know benefits to this. But on the flip side, you know, it's a... you're operating in a larger organization, so you're not as agile. so at Snowflake, we try to optimize for this. It feels like we're operating in a much more entrepreneurial environment, which is absolutely required to work in AI. You have to move at the pace of AI. Sam Adebayo: Great. you know, let's talk about AI. when a regular person asks Claude or Chat GPT a question, it seems almost like it's free and you know, very instant, right? But the reality for a company running this at a huge scale is that every one of those questions actually costs something, right? can you help us to picture what's actually happening behind the scenes in that millisecond of, you know, prompting and asking a tool, a question. Baris Gultekin: Absolutely. So it basically starts with first figuring out what context to give to the model. Ultimately, the models are great, but they are as good as the data that you provide to them. So you know, when a question gets asked, you know, given that question, we try to figure out what is the most relevant data and the context, explanation of that data that goes with it to give to the model. After that, you know, the model basically takes that as input, and there is a massive amount of computation that happens, trying to predict, given that input, what is a good response, and then doing that, you know, token by token, response by response, trying to score you know which word is the best fitting word, you know, given that set of questions that you've asked. you know, The part that is expensive is that computation. you know, that massive amount of computation is because these models have a lot of parameters, a lot of weights that need to be used. and that computation, trying to kind of figure out what is the most relevant word to churn is very computationally heavy, and therefore expensive. leah: And one could make the argument that there isn't one single AI that's best at everything, right? There's lots of them and they're good at different things. Why does that Baris Gultekin: Absolutely. leah: matter for the average company? Baris Gultekin: So, you know, AI is moving from the phase where many companies were experimenting with it to, you know, large scale production deployments. you know, you have every company that have an AI mandate trying to make sure that all of their employees are leveraging AI to the... to their fullest. When that's done, when every company has their users now using AI heavily, there's a large amount of use cases to cover. And also, you know, given that demand, there's a large expense. so currently it is very much top of mind for every single company that I talk to to ensure that, you know, the efficiency of the intelligence is there so that everyone is using AI to the fullest in the most cost-effective way possible. So you're trying to get the most out of AI, the most intelligence out of AI at at the most reasonable cost. so that means You have a lot of... you know, we provide a lot of model choice for our customers. Each model is good for certain things. Some models are kind of larger, as I mentioned, with a lot more, you know, complexity from a computational perspective. Others are smaller and more efficient. Some models are very good for certain tasks, others are for other tasks. So there's a you know, there there's an opportunity to create a highly optimized outcome, you know, to give you high intelligence at low cost. Sam Adebayo: Great. So you've talked about the importance of data, and that's sort of like the conversation that I've been having with a lot of people in this space. but can you frame to us just how important it is for enterprises to sort of like structure their data because you there's one conversation about how data is important but there's the other conversation about which company is ready in terms of like the data that they have or they're sitting on for the data to be used actually for AI. Baris Gultekin: Absolutely. Yeah. We like saying there's no AI strategy without a data strategy. When I talk to customers, even though everyone's very, very excited about AI, a huge amount of effort goes into preparing their data so that it is AI ready. so what that means is you start with governance at the core. you know, When an AI feeds on data, so when you're giving access to large amounts of data, our customers are Putting in the governance frameworks so that the right people have access to the right data and not more. At the same time, AI needs to understand the semantics of that data. so when you, for instance, ask a question like, you know, what's my revenue in North America? You know, How revenue is defined, how revenue is calculated changes from one company to the other. So AI needs to understand all of those semantics around metrics. Around you know acronyms that the company uses, the ontology that describes how the company operates. So that also needs to be there from a foundational perspective. And then there's also once all of that is there, the agents need to be fully governed. you know, when you're giving access to sensitive information, you want to make sure that the agents have only access to, for instance, you know, not modifying things in sensitive places or not having access to sensitive data. so all three need to come together to provide high quality, governed experience. leah: That's really interesting. And I'd love to learn more about the culture inside Snowflake. I read Amp It Up, your previous CEO at Snowflake's book, I'm sure you have as well. And you mentioned entrepreneurial spirit within the company. I'm curious what that culture feels like from the inside. I know Snowflake's sales culture has a real reputation. What actually makes it work? How does something that intense scale? And what are the people on the ground like who who work there? Baris Gultekin: So Snowflake has an incredibly pleasant culture to... you know if from my eyes, in terms of like there is a lot of entrepreneurism that's happening in the company. The company operates very, very quickly, and which I mentioned, we have to, in the world of AI. So we operate as a startup and at the same time, because of its size, we have to work together as a unit. And the culture of the company is such that there is incredible collaboration between teams so that you know we're moving towards one direction together. a huge part of it comes you know from top-down. You mentioned you know Amp It Up now our current CEO, Sridhar, has been at the helm in terms of kind of transforming the company culture to be AI first. So there's a lot of, you know, AI pill the executives and employees in the company were... we were early to adopt AI to transform how we work so that we could do the same for our customers. so we've spend a lot of time and effort to transform how we do finance, how we do sales, how we are doing our product work, all with AI at the core. so I'd say both from an execution perspective, the company operates very fast, from a culture perspective, very pleasant to work with, and from a technology perspective, very much you know, AI pilled, AI first company. Sam Adebayo: Thanks, Baris. so I'm aware that Snowflake just rolled out a couple of things. and a lot of it sits at the intersection of what you said about how that some AI tools are a better fit for some particular tasks. and some others are a better fit for other tasks. So, you know, one of the things that you're rolling out is what you call a dynamic model routing, which is basically a way to automatically send each task to the AI that, you know, fits it best; a cheaper or simpler one. so can you frame to us in really, really simple terms what all of this actually means and how it's actually going to help to change what we currently have in the general AI landscape today because that's a real problem that I am aware that enterprises are dealing with today. Baris Gultekin: Yeah, so we actually talk to our customers on a regular basis and notice, you know, when you ask a simple question, you do not need to use the most expensive, most capable model. you know, that model again is costly and it's it's also slower. and then as as we mentioned, there's also other models that are really, really well suited for certain tasks. So we are introducing dynamic model routing in our AI gateway. dynamic model routing essentially, as the name implies, automatically routes the question to the right model given the task. so for instance, you know when you ask a simple question, it absolutely can use a a simple model. Under the hood, you know, this is a active research area and and we employ a couple of different ways this works. You know, The first way is commonly known in the industry as the advisor model. So you start with a simple, simpler, smaller model. And if that model does not you know do well, it can always consult the more capable model. almost like a kind of student-teacher relationship there. and and that works incredibly well. It reduces the cost substantially and increases the quality. There's also other classifiers in the system. we know, for instance, that, you know, certain questions are easily answered. so whenever our coding agent, for instance, is using the auto mode, meaning the customer has chosen to have us route to the right model, given the past experiences, we can then route to a simpler or a more capable model, based on the historical data. leah: You've worked on this since before it was interesting to anyone and I'd like to know: is AI where you imagined it would be? And anything new or exciting we can see on the horizon that you're working on? Baris Gultekin: I think AI has absolutely exceeded my wildest imaginations. you know, I feel like it's difficult to imagine exponential change. you know, human mind... it works more linearly. So I've been incredibly pleasantly surprised of what AI can do today and how fast it's growing. you know, when I think about the future, there's a couple of things. So one is AI is now very, very capable in using tools, using you know software big similarly to a human. so we're seeing, for instance, you know, what we call computer use, being able to go and emulate what the user would do, open a website, click through to go get things done, immediately opens up a vast amount of capabilities for AI. And you know, this area will only continue to grow. so, you know, what I'll call you know, long running tasks, more complex tasks are going to be more and more possible. So today you ask a question, you get an answer. and we've moved from that to taking action. You know, when you, you know tell AI to go research something for you and then to, you know, email you the results, it'll go do that or to change things, to book a restaurant for you on the consumer side. So that's where we are today. And a lot of the research is going into much longer running tasks. being able to delegate things that would take a day, multiple days even, to AI. AI would go work on it and then we'll notify you when it's done. That ability to do long running tasks is growing at an exponential phase. it... we... AI used to only do tasks that would take, you know, a few seconds for humans, then to a few minutes. Now we're in the multiple hours phase where AI can work, you know, 10 hours, 12 hours to give you an answer. And, you know, the the next exponential growth will be days and weeks and months. so it's hard to imagine what that would look like, but absolutely going according to that plan. Sam Adebayo: Baris, I have two questions for you: You've just said that AI has exceeded your imagination in many ways. What do you have to say to skeptics who still believe that the gap between the promise or the noise about AI and what is the reality today is just a far cry from each other. Just today, like about 30 minutes ago, I was reading a piece from a well known AI skeptic, who believes in AI, but just also says that there's a lot of a lot of narrative that is not true, you know. He was basically calling out Dario Amodei, the CEO of Anthropic to say that no, AI is not gonna solve cancer problems in the next five years. What do you have to say in that regard? Baris Gultekin: I think AI is transforming companies today. we're already seeing this... you know, how software gets written has completely transformed. you know, Companies are building more code faster and developing in a much faster pace. And similarly, we're now seeing work is transforming. you know, the way Snowflake runs our... we run our sales organization, for instance, is very different from where we were, even just a year ago. and that gives us a lot more capabilities, a lot more efficiency, and that's real, that's happening today. At the same time, you know, as I mentioned, it's difficult to imagine, you know, the future. Five years is so difficult to imagine. You know, even a year is barely kind of imaginable today. you know, that leads to a lot of speculation that leads to, sometimes, fear mongering of what, you know, what might come. So I would say just rather than kind of speculating, you know, five years out, focusing on where we are today and how AI can be useful today to transform how we work, how we find information, how we get things done, is absolutely real. There's of course issues, you know, there are hallucinations that happen sometimes, but those issues are kind of getting resolved at a pace that's been never seen before, right? hallucinations are going down. the capabilities are going up, the cost is coming down. so I would say focusing on where we are today and then transforming life, transforming work with the tools that are today is possible. Sam Adebayo: my last question was gonna be to ask you to frame to us in just one word what the future of AI would be. But I think that you've sort of touched on that already in your response. there's a lot to talk about, but thank you so much for this valuable conversation. it's been quite enlightening and you've done a great job of making something that would otherwise be really technical to be quite clear to understand. So thank you. Baris Gultekin: Yeah. Thank you. I appreciate the conversation. This was fun. leah: Thanks to everyone listening. This is Machine Dreams. More honest, unfiltered conversations with the people shaping the future of AI are on the way. See you next time.