Latest / The MonkCast / Sanjay Sarathy on Core Principles of Developer Trust

Transcript

Sanjay Sarathy on Core Principles of Developer Trust · The MonkCast

• speaker-0: This episode is sponsored by Cloudinary. speaker-1: Hey everybody, it's James Governor from Red Monk. we're here for another podcast. We're gonna be talking about developer trust today. I'm here with Sanjay Sarathe from CloudNary. He's the SVP of self-service and developer experience. So welcome, Sanjay. speaker-0: Thank you, James. Thanks for having me on the show. speaker-1: It's great to have you here. and yeah, we the the the the issue or the question today is about trust. Obviously, everything is changing quickly in this AI era, but I think it's interesting to get a bit of context in speaking to you. Why is why do you think trust is important in terms of of platforms in general and Cloud and area specifically. It's not necessarily a word that I I hear platform companies talking about as a as a primary issue. Why do you think why do you think trust is so important? speaker-0: I think it's important because developers are facing so much change in their environments, they face changes in terms of how they're coding, in terms of how they're interacting with different systems. And when your consumers, your users, your customers are facing a lot of change, the ability for platform companies to sort of provide a level of In some ways, not guarantees, but a standardized way of working and interacting with your system that will support them through that change is ultra important. So that they feel like a company they trust they they bet on maybe a few years ago will evolve with them as their working habits have evolved. And so I think that's really important these days. And I think trust has always been important. I think it's just become heightened from our perspective because of these changes. And so one example of how this manifests itself is when developers inquire into our support channel, how would I do X in cloud and in my Claude or cursor environment? And or how would this work when I'm integrating with XYZ through the MCP servers? And so I think that level of they want to understand that we're evolving with them. And that's the sort of the notion of how trust manifests itself in a lot of ways when we're interacting with developers and our users on a day-to-day basis. speaker-1: So and I think this is important. This isn't just something new like, AI is happening, developers are worried, so now we need to focus on trust. This is something that Cloud Mary has been doing for a while as a as a sort of a stated principle or at least an an organizing principle for the company. speaker-0: Yeah, it's it's always been at the root of how we operate. And and part of the I think the the thesis behind CloudNerry, in addition to being focused on media management, in addition to being API first, is to say, how do we find out about all the bumps in the road our developers may face? And how do we solve those or help developers overcome those issues so that The next generation of users that come our way never face those bumps. And so today we handle something like 22 million uploads a day. 22 million assets are uploaded a day. We have hundred nearly 150 million transformations a day. And to an individual developer, that doesn't mean anything. But what it does mean is our ability to handle that is because we've s we've figured out all these bumps in the road. And we've hardened the infrastructure, we've strengthened our APIs, we've figured out what needs to happen to support that that next developer's use case. And that's where that trust gets built in, independent of whether AI is in the picture or not. speaker-1: Okay, okay. And I I you know I think that's right. and there's certainly a a trust deficit in the industry right now. you know, what what can we rely on? it is it is a very good question. So I I I thought it was interesting when we were discussing this that you've sort of got a set of principles and you know, we'll walk through them. You know, you were talking about maintaining developer trust in an AI first world. I really like the first one because I think consistency is super important. And particularly in a world of LLMs, it's all too easy to, you know, i think a different method every time. because that yeah, that sort of consistency. You're your the the principle here is the product must work as promised every time. So A, what does that mean as a principle? And B, how do you actually enforce or or regulate and manage that at Cloud Nary for the developer experience. speaker-0: Sure. So I think it starts off by as a developer first companies, we have a set of very well-defined APIs that essentially help the the developer understand you know how to interact with our system and how to interact with our platform. And what it also means is if it doesn't work and we get feedback through the channel. whether it's the support channel, whether it's through our customer advisory board, whatever the channel might be, that we spend the time to essentially update and fix and and ensure that those APIs actually operate as client. And we spend a huge amount of time on that. I mean, there's a whole engineering team focused on that. It it also means that We go through a pretty rigorous set of when we're introducing new capabilities that that rely on those features that we're going through. And again, this is not necessarily unique to us, but we go through and we pick the folks that have that have expressed interest in either the new capabilities or updates and go, does this work in your environment? Not according to specs we've written, but in your environment. And only once we've received that. that feedback then and at a certain level of scale that we go, great, we can introduce this to to the masses because we have, again, four thousand people a day registering for CloudNary. We need to make sure that all four thousand of them feel comfortable and tru you know, to use the word trust that the APIs are going to do what they say. speaker-1: Mm-hmm. And so on that four thousand a day, I I know that you're sort of justifiably proud of your developer signups. What is the the total numbers and the total number of developers that you've got signed up as Cloud Very users today? speaker-0: Yeah, we we crossed the four million mark earlier this summer, James, in the in the late June timeframe. And you know, we've seen and we've been fortunate in that we've had AI tailwinds help us, because we've seen that that number grow significantly. I think it's something like seventy percent year over year. and Today, forty percent of the registrations that come in are sourced from either an AI-based IDE or LLM, sort of an AI referral, if you will. And so that's that's also sort of rewired how we think about trust in in that sort of environment where you're you're placed right in the middle of a workflow that's in cursor, or name name your tool of choice. Yeah. It's led to some other decisions around developer experience that we've made. speaker-1: So okay, four million sign-ups, which brings us neatly, I would say, to our second principle that you you're organizing around, which is support that scales always. And I've got to ask you about this because quite frankly, a lot of modern modern platforms are just not very good at at support. the the you know, there is an idea that you can make the developer experience good enough that people don't need the support. you know for so many of the experiences if I want if I want support unless I'm paying a huge amount of money I'm gonna end up you know going to you know read the facts. Right. You know or yeah so I mean there are some orgs that are good at this but tell me a bit about how do you scale support to sort of that kind of user community and why Why it important? Because I think these days people are as I say, there is a bit of a view that the the platform does the work. So what does support look like? And how do you how do you meet that promise of support that scales? speaker-0: always say that I think support is one of our hidden weapons, right? It's it's it's a differentiator for us. It's to be honest, after the founders started CloudNary, the first, you know, beyond the platform work that they did, one of the first areas we invested in was support, not go to market, not developer relations, but actual support like People that found CloudNary worked with CloudNary, how did we make them successful? And the premise was if we make CloudNow CloudNary users successful, especially in the early days and all the way through today, they'll be happy and they'll recommend CloudNary. And WordAmouth remains a powerful referral source for us. And so we've always believed strongly. And the folks who do support are highly technical. They're developers who happen to like in enjoy working with our users and solving problems that the users might have. And so we've we provide support to people who use the platform for free, all the way up to the people that we have platinum, you know, platinum support or diamond level support. I don't even know what the highest level is, but the highest level of support. And obviously the levels might vary in terms of the you know, twenty four by seven year etc. But the premise of how we have a developer who signs up today and says, I have a problem, doesn't change. Now, how do we scale with that? So obviously it's you know, this is where we use AI to both speed up the the answers that are given. But back to your trust question, we have to ensure that what our our AI assisted answers provide is are accurate. Right. So that's a critical part of of of the trust equation as well. And we will always have a human backstop. So the way we've if if the if they don't feel comfortable with a response that a chatbot may provide, whether it's on our documentation site or or elsewhere, there's a human that they can say, Hey, I'd like to chat with XYZ because I'm a little confused about this. And great, we will we will we will respond. So We're we're scaling with AI as as every company does. You know, I go back to that 75% growth. We have not increased the support team by seventy-five percent. But we will always have human interactions with those developers should they choose to interact with us that way or request that. so that's that's the way we do it. But the principle behind always, you know, making sure the that we answer questions that an individual developer passed through our support channel hasn't hasn't changed. The means will evolve and will continue to evolve, but we'll always have a human there to to help the developer out if if they want it. speaker-1: And y you mentioned the the the danger of of or at least the the possibility that the the the model doesn't kind of give you an answer that you find helpful. heaven forbid that there are any hallucinations in in your your customer sport. But that brings us to a point that that I think goes along a bit with the the must work as promised. But the One of the principles that you put forward is the notion of accuracy in an AI assisted world. And how do you how do you support and manage accuracy as as as a principle when we are dealing with non-deterministic systems? and and yeah, what what is what does accuracy mean to Cloud and Airy and and how do you support yeah, how how do you support that in this in this world of assistance? speaker-0: Yeah, so part of it is, for example, I'll I'll give you an example where that my team's responsible for, which is the the the AI assisted bot that we have on our documentation site, which is an enormously trafficked part of our website. And we see all the questions that come in, we see the responses that the bot gives, and we review it, right? And we ask ourselves again. documentation team is very technical. They're essentially developers who write well. And so we do a few different things with this. One, we check to see how the developers have responded to to the how they react to the responses that the AI assisted bot gives. And we understand if in fact they feel like the problem was solved or not. If we see multiple threads associated with that question, then it it it it it's it's like an alarm bell for us that says, okay, why did why does it take three or four responses to get to that answer? Is there something unclear about the way we've written it? and and then a bunch of times what we'll do is we realize we need to train. the bot better to look at different components of of of our documentation. And so we've become better at not just understanding whether our responses are accurate, but we're also getting better and better all the time about training the AI to ensure that because we're updating the documentation all the time, right? Yeah. And so we need to we need to make sure that as we update it that That our AI assistant knows what those updates actually mean and we train it associated with with specific types of responses that might be needed. So that's that consumes a fair bit of our time, but it's important because we get so many thousands of inquiries, you know, around from from the documentation site. And we just need to make sure that what we provide is remains accurate. And so it's it's important to us and it's And it's a good forcing function to say, hey, does the documentation site actually respond or actually showcase content in a way that the developer's question expects, right? Because prior to this, we didn't know how developers wanted the documentation to read, right? We made assumptions. Yeah. But I think one of the benefits of AI is you actually see the questions. You can see the types of questions that are asking. And I think the other piece that goes maybe unreported is We know, we've got sixty percent of questions this month are related to upload API. what does that tell us? Right. that might mean that there's something confusing about that section, independent of whether the response is correct or not. Why is it that they couldn't couldn't use it seamlessly right away? And so that that forces us to make changes there as well. speaker-1: I mean, obviously would have been doing, you know, these sorts of things before AI AI come along came along, but models are very good at parsing huge amounts of correct question marks and summarizing. So yeah, I I guess it it it it it does make things more eff effective for that perspective. So alongside accuracy, and I I I think the the the next question, I mean certainly I mean, I grew up in and around open source. another one of your your principles is transparency as a foundation of trust. So what is same thing. What does transparency mean to CloudNary? How are you how are you supporting that? In w in what sense are you are you is the CloudNary experience transparent? speaker-0: So a few different ways. again, a a good example is and and I'll I'll go to the other extreme, which is our customer advisory board, which is made up of some of our heaviest users both in Europe and the US. And so we have customer advisory boards with both groups. And essentially one of the foundational parts of of that meeting is What did we talk about last time? What what was your feedback last year? And how did we implement or how did we take that feedback into account six months later? And so our ability to say, here's what you said, here's what you told us, here's what we did with that feedback. And and independent of whether it's big or small, our ability to actually talk through that. And help them understand, okay, we heard you and this is what we're doing, I think is i is a huge part of saying, okay, they they're not just doing this for performative reasons, they're doing it because they want to make us successful. And I think that goes a long, long way. I think the the other And and that also then is also a part of we have public roadmaps that we share with prospects that we go, hey, here's what we what we're committing to and under NDA and here's what we're planning to do and we oftentimes our CSMs and our s our account executives and our account managers are held accountable to okay, if you're gonna show that then we need to make sure that in fact that that that holds true. So There are always caveats to that, but that's that's one example of transparency. I think the other piece of it is when our support team, for example, gets requests around specific capabilities, we'll come back and say, Hey, here's what we can do and here's what we can't do. We won't commit to it. And I think it's almost as important for trust for people to hear, no, we won't do something and we can't do it. Here's a workaround for you. We're not gonna, you know. I think I think users, developers, marketers are tired of people always committing to something, because it's at at some point it's non it's not it's not credible. speaker-1: No, sometimes the the no is a lot more powerful in terms of trust and predictability and transparency. As you say, you can't do everything. The every every individual user, every organization, every customer, they think that their particular requirement is incredibly important. Correct. but but yeah, sometimes they're not gonna get that. So so yeah, that that that certainly makes sense. And and I think if you're you know saying no to things, perhaps that brings us to the next principle, which is the things that you've said yes to showing a real commitment around them. So you you you know we we you know in preparing for this you talked about the importance of stability, predictability, and a long-term commitment to APIs. Now Certainly in the world of of of models, it's moving so damn fast. the models seem to change their behaviors on an almost daily basis, which is is not always super awesome. but in the API world, there've always been a you know I I would say sort of a you know a a couple of different philosophies. one is, you know, deprecate early, deprecate often. to your point, tell people. The other one is the sort of, you know, the kinds of things we've seen from an IBM or a Microsoft, where if you write to an API, it's still going to be there in in 20 years. So where are you in terms of of your commitment to APIs? What's Cloud Nary's approach? How do you support developers so that they could feel comfortable and ha maintain their trust in the in in your APIs? speaker-0: I think we're certainly not as old as IBM or or Microsoft, but our our I think our principle adheres a little more to the core APIs we we don't want to deprecate because we we've built a lot of goodwill around them and we've s we believe that there's a lot of value around them. We will certainly deprecate capabilities, and we have a think a a notice period and a standardized deprecation process that we've built in for capabilities or end-of-life, even even product areas that we said, you know, there's just not enough business value in this to the to the end customer or or for us. But we're a little more stingy about about about our APIs and and keeping those going unless there's a clear, clear, clear reason for for deprecating. And then we'll we'll we'll make that decision and and and talk through that with with our users and why and what if they are using it how they can what the workaround is because that's the other important piece is we know every time you're making a change to to APIs, you know, you're impacting workflows that rely on it. And the feedback we get from a lot of from a lot of our developers and users around CloudNary is it works so well we set it and forget it. It just works day in and day out and we're off doing other things. And so if you're gonna all of a sudden deprecate APIs, then that set it and forget it concept all of a sudden is is a jarring scre comes to a jarring, screeching halt, right? So we we are cognizant of that. And do take a lot of time around if if we really feel like we need to deprecate APIs that we spend a lot of time understanding the why behind it and not just because, it's reduces our internal workload. Th there's gotta be a good reason for it. speaker-1: Now you mentioned, you know, customer advisory board, you you know, sort of diamond or platinum customers. Obviously, you know, when you're dealing with a you know major major media company or or or something like that, yeah, you're you're on the hook for that. You you you you know, you are listening to their, you know, I mentioned requirements and PRDs. But I mean, how broadly, you know, given that you're Remitt is around self-service and and and and developer experience, developer relations. How do you ensure that there's a developer voice in the product loop? Which is one of the things that you support or put forward as a question of trust. So how do you listen? How how how do I, as a, you know, maybe I'm a student in Bangalore and I'm, you know, I want my voice in the product loop. Just as much as yeah, like a media company in in California. Like how does that work? speaker-0: Yeah, so I think it there are a few different channels for that, maybe volume-based feedback, as I like to call it. one of it one one of the channels certainly is our developer relations team that is in practically daily contact with developers around the world and and engage with them. And in fact, we're doing it's funny you said a developer in Bangalore, we're doing a series of hackathons that we're supporting in in India right now. And And I think the feedback on how some of the tooling works for a newbie to Cloudnery, what happens if I want to do X with Cloudnery and these other three systems? and we're there where we're engaged with them. We're not always physically in every single place, but oftentimes we'll go through on on a call with them and help them out through their, through their their build process. That feedback gets collated and sent back to the product team as well, right? So there's that's one channel. The other channel that I I t I take pretty seriously, not just because of the score. We have NPS, which has been around for decades. The NPS for us is not about the score, although that's important. The numbers always always tell us something. But more importantly are the qualitative responses that we get. Because we always ask in addition to how would you rate CloudNary, what works and what do you like about Cloudnery and what doesn't work and what suggestions you have. And those qualitative responses are segmented by types of users. You're a free user, you pay in self-service, you're an enterprise user. So I see all of those responses. And the team sees all of those responses. And so again, we don't overindex on the score, we definitely focus on the commentary, right? And and try and create an understanding of how that commentary, what that commentary is telling us. So I think there's the there's in some ways the push out in terms of how we are proactively engaging with developers, primarily new developers, and and what that means and how that might impact the product. And and get feedback on new capabilities that the enterprises haven't even started using yet, right? And oftentimes they're the the develop the individual developers at the vanguard of that of that engagement. And then there's the poll where we get feedback from the thousands of developers that are currently using the product and get qualitative responses, and then we segment that and say, Okay, here are the three things we know we need to work on. So so we I I give you two sort of examples of of ways in which we get that feedback, but it's I think it's important also to To marry that and I think one of the great things, my peer on on the product management side, they've become really good about both the product of the PRD process but also the listening to external voices. And it's not just for the big customers, it's also for, hey, what do the volume developers tell us? Because that's our primary user audience and and our ability to to engage across both those segments is is something that we try and continually improve on. I'm not saying we're always, you know, we're perfect at it, but we're we're we definitely put a lot of emphasis on it. speaker-1: And you know, going back to that, you know, in developer in Bangalore, or you know, maybe maybe they're a student, you know, or they might be in I mean I know that you invest a lot in in global support, so you know, you're you're trying to get new users in you know across Africa as well, you know, across Asia. but just in general You know, we think about developers today, particularly well, in fact, no, I mean, new entrants obviously more challenging than it was a couple of years ago. but but for I think everyone, people are sort of feeling what are my skills worth? It it's sort of a bit of an existential question and it's kind of AI era. People are feeling a certain amount of fear about their futures. Are speaker-0: Absolutely. speaker-1: they doing the right thing? You know? sort of even even getting into software. So I think one of the things I thought was was quite interesting when we talked about the dimensions aspects of trust was the the prioritization of education and developer growth. How do you thread that needle where, as I say, people worried and and and there's, you know, people say, you don't need to write software anymore. It's all going to be done by agents. Why why should people continue to learn? And how do you support them in helping them understand that there are opportunities and that it is worth coming on a journey with CloudNerv? speaker-0: Yeah. I was having this conversation with our director of developer relations the other day and she told me this interesting story about a student that she met at a hackathon and they were building with Cloudnery. it was an AI focused hackathon themed event. And we we judged I don't know hundreds of of entries that used CloudNerry that day. And this is one of the two or three that were selected as the top winners. And what she found out, you know, three months later, I think the hackathon was in early, early spring, is that the application that they built, and these weren't just simple, this is an interesting proof of concept. These were the joy of AI these days, is you can build in a 24-hour period pretty much a full-blown app that scales pretty quickly, as she says, You know, the the app that we built was selected as a Y combinator, you know, entry. And and and you you go from I'm talking to a student about a particular tactical, how do I build, how do I upload images and videos and use transformations to do XYZ to, you know, this might actually be a startup idea that has legs to it, right? And I use it as an example simply because I think part of our philosophy is we want f especially students and early career folks to explore how CloudNary can help them with interesting application development bills. And that all the creativity is on that side. I think what we want to do is support that creativity through through the way they can Manage images and videos and and and areas like that. And so education isn't just about how to use, here's how to use our APIs and here's what we do, but I think there's a there's a broader, hey, explore your creativity using CloudNary, and that might lend itself to some really interesting applications that help you through a job interview, help you through pitching your own business idea, et cetera. And again, we're not the only ones in in in the ecosystem doing this. But I think the the nice thing about images and videos is that they're very engaging as part of the app and and and so that process of of using CloudNery and that we want to continue to support through whether it's this you know early early career developers or students is that exploration of what's possible for for their career. So we try and spend a fair bit of time on it relative to to our our size. And we've developed a series of courses that can, as we say, help your media IQ, right? Independent of how you choose to use it. And and we we and we For a company our size, I think we try and put a fair amount of emphasis on on supporting these, even if we can't be f jetting around around the world to to to meet everyone. We we will spend time oftentimes virtually supporting these these students and others in as part of the build process. We want people to build more than anything else. speaker-1: Right, right. As you say, you can't necessarily travel everywhere. I know you invest a lot in in YouTube as part of your education efforts. Okay. Well, I think one of the ways to build trust is to keep things short and sweet. speaker-0: Yeah. speaker-1: And on that note, I think yeah, that was super helpful, Sanjay. I I you know, I I I definitely think that putting that you know, yes. There are enough companies out there that I wouldn't trust them as far as I would throw them. you know, and partly that's just the spirit of the age. Things are moving so fast, things are changing, breaking. we're expected to learn a new thing every day. So yeah, some things that we can hold on to and that are consistent are valuable. I think that trust words great. and so thank you very much for your time today. I hope everyone enjoyed the podcast. But yeah, once again, Sanjay Sarathi, thanks so much for joining us. Today on the monk cast. speaker-0: Thank you so much, James. Enjoy the conversation. Thank you.