Latest / Auto Collabs / Training AI To Fix The Little Things with Sanjay Varnwal
Transcript
- 0:00Michael, you have some new glasses. They talk to me. [upbeat music] This is Auto Collabs. I mean, I thought- [laughs] I thought... Well- [laughs] Michael, let me break it to you.
- 0:16I thought he was trying- Those are just the voices in your head. Yeah. They've always been there. [laughs] That's just what they did- You are the voices in my head... to make you feel better about it. It's a new therapy.
- 0:24They give you glasses, they say talk to you, so that you think it's, oh, I'm normal. But you're not. I actually thought you were trying out to be the new lead singer of Weezer. [laughs] Oh. That would be...
- 0:35Man, that, you just brought me right back to the seventh grade. [laughs] I mean, I would pretty much trade- Just throwing sweaters and- I w- I would, yeah, I mean- This is how I know it's the voices in my head.
- 0:44Just so you know, guys, if Weezer calls me one day- You're gone... and is like- See ya... "Hey, we need a new front man," or, "We need a drummer," like you're just never gonna see me again. I'm just- [laughs] Yeah.
- 0:55Pretty much any band. Like, there's, I have this, uh- Oh, really? Any band? You'd trade us for any band? Well- No... he's like, "Some death metal van- band?" Done. In. I, guys, I don't know- I mean-...
- 1:05if you saw these shorts that Nathan's- Jonas Brothers?... putting out. Right. The audience is throwing me under the bus, bro. I don't think- Are they? I'm not, I'm missing this... you got the better beards.
- 1:14They'd rather be your friend- [laughs] Oh, the comparison... and doing business with you. Listen, listen, it's just exposure. I'm like, any band, at this point I'd take any band.
- 1:22At this point I would take, like, a ch- uh, I'd, I'd join The Wiggles. [laughs] Strangely enough- Look, The Wiggles go hard with the kids... strangely enough, that is probably the most likely band to call you.
- 1:33[laughs] It probably is. Because you would make an amazing Wiggle. Just amazing. I'd just, I'd be the hype guy, like the mighty, mighty Bosstones. Yeah. I'd wear a gray suit in the background.
- 1:42With, with, with Jack Black vibes- [laughs]... in your dancing. [laughs] Just kicking everywhere. Yeah.
- 1:48[laughs] Well, all that has nothing to do with the fact that we are hanging out with Sanjay Varnwal from, uh, from Spine AI today. Talking AI, man. We're talking AI.
- 1:59We're gonna be way more intelligent, hopefully, than us talking about The Wiggles. But if you wanna listen to The Wiggles, you can find that- No, he's gonna br-... wherever you want to.
- 2:09No, Sanjay's easily gonna be, bring the collective IQ average up by many points. Way up. Way up. Once he enters this room. Yes. Well, hey, look, we really hope you enjoy this conversation, uh, with Sanjay.
- 2:21[upbeat music] Sanjay, how are you today? Thank you so much for joining us. Hey, thanks for inviting me for the, for the, uh, for your show, Paul, uh, Kyle, and Michael. Yeah. So- Hope you're great. Yeah...
- 2:35so, I'm, I'm sure we're gonna talk a lot about AI today, but I, I think this is gonna be the question I start asking people whenever we're talking about an AI topic.
- 2:43Tell me about the first experience you ever had with an AI product. Oh, that's tough. I mean, [laughs] no one has asked me this question before. [laughs] Yes. I've never asked this. Yes. I've never asked it before.
- 2:55[laughs] Nailed it. So, I'm remembering few of the things. I used to work at Amazon, uh, uh, in, in 2014 or '15 or so.
- 3:03And, uh, there I, I used to travel to Seattle often, and I saw Amazon doing a pilot of a store called Amazon Go at b- that point of time. And it was one of its kind when it was launched.
- 3:17You can just check, uh, and go inside the store. You don't need to do anything, just pick up goods from there and go out, and everything is, like, checked out for you. Right. The payment is de- debited, everything.
- 3:27So Amazon was doing that particular pilot.
- 3:29Of course, the smaller AI use cases I would have seen in the past, but that was the first big, uh, working practical use case that I saw on the ground, which was, uh- Uh, yeah-... Amazon Go.
- 3:41That was a- And I, I think that that's interesting because for the majority of the world, AI didn't really exist until, I don't know, 2022 in their brains, right? Right. Yeah.
- 3:49But th- these big corporations have been dealing with AI since, like, really the early 2000s. There were already- Yeah. Yeah, yeah... inklings of, like, how do we create things that can do stuff on its own, right?
- 4:02Absolutely. All those Google recommendations and all these things, like there is an element of AI behind it already, right? So- Go ahead. Go ahead, continue.
- 4:12Yeah, I mean, the, the, the AI in terms of data analysis, data analytics, some of these tech has been existing for long, long time.
- 4:20But computer vision, where you see, like, the magic happening in front of you, is something that, that started getting into prominence in the last, like, seven, eight years or so. Yeah.
- 4:29I wonder was that- And chat, we've just blown it up... when was that movie that came out with that, that little kid? I just think of him as, like, the sad little boy 'cause he's got those sad eyes.
- 4:38Haley Joel Osment, is that his name? Oh. And the movie was called AI. Oh. Yeah, yeah, yeah. And I feel like that was the first time that I even, like, thought about artificial intelligence. Got it. Yeah, yeah.
- 4:48Absolutely. Right? And that was probably... I mean, I gotta say, that, that movie had to come out 15 to 20 years ago.
- 4:53But if that movie came out, I bet, I bet there was something going on in the tech world that somebody was like, "You know what? We see this coming. We're gonna make a movie about it." Yeah.
- 5:00And now it's, like, movie is reality. I, I love guys like, uh, Neil deGrasse Tyson and stuff though, because they bring everything back to its most rudimentary form.
- 5:08They're like, you know him, he's like, "Oh, you want flying cars? We have those. They're called helicopters." [laughs] You know? Like, like [laughs]
- 5:18and Sanjay, you made me think of this because, you know, with Google w- with what Google's worked on over the last, I don't know, 20 years, anybody that's...
- 5:27If you want light reading, go learn about Google's RankBrain, 'cause that's been around for quite some time, and that is, I mean, that is what Skynet might be made of.
- 5:38RankBrain fundamentally is the core machine learning based on which their search algorithms are built, right? So,
- 5:46uh, I haven't studied in depth, but, uh, I've heard that, uh, this is, this is the base- based technology on which, uh, the, the entire Google search functions globally. So- Got it... yeah. So- And it's, like, super...
- 6:00It, it's like the Treasury of the- Yeah... United States of America. Like, it is ironclad. There are- Absolutely...
- 6:05there are, like, RankBrain, Google RankBrain offices all over the world that-You know, for example, we, we heard through the grapevine that there's an, an office in a lab at the University of Alberta in Edmonton- Oh...
- 6:18but there's no Google signage any- Like, they don't want people to know what's going on in these labs. Oh, I see. Wow. Yeah. I see. But it's deep. You know, it's funny, Sanjay, like we were just talking,
- 6:32um, I saw this notification in Slack that Slack's just released its AI to our, to our organization, and so I started playing around with it. Uh-huh. I'm curious, you know, what your take is on, on, on this. If AI...
- 6:49Let me back up. See, my brain's moving so fast. Maybe this is- [laughs] I need the AI. It's those Meta Ray-Bans feeding you the questions. It's asking me- Yeah... John Cena is in my ear telling me what to ask.
- 7:00[laughs] I saw a meme recently, and it said, "In the year," what was it? "In the year 2050, th- this is how the generation will communicate," and he was basically like a caveman again prompting GPT. He was like- Oh...
- 7:17"Tell me good thing to say to girl." You know, like, it was like that. [laughs] Because our brains have just morphed into GPT to the degree that we can't even communicate as humans anymore. What are you seeing?
- 7:30I mean, I know you're heavy into this space and software development- Yeah... and the implications and applications of AI. Where do you see... W- what's happening out there? What's going on?
- 7:41I think, uh, you said it right. I'm not too sure, uh, if, if that would be the situation, but potentially could be. But yeah, as, as, as you see things around, the way the behaviors are changing for all of us, right? So,
- 7:55I mean, earlier we used to think, and then you used, you used to write, or you used to say anything.
- 8:00But now I see myself, my behavior is changing to be whenever I need to respond to anyone, I, I, I ask ChatGPT, "Hey, this is the profile, this is me, this is the context. Can you come up with this, uh, direct?"
- 8:15I mean, uh, very, very, uh... I mean, the kind of response that, that elicits some response from them, right? So, and it, it, it comes out with very well-researched answer or the, uh, responses.
- 8:25So this is like just one of the use cases. I, I keep using ChatGPT for a variety of use cases and have stopped applying my brains to a lot of work. Mm-hmm. Right? So- Right. Yeah...
- 8:36you, you see like- Well, but I, I think that, you know, I think that there's different levels of usage. One is like, "Here's a, here's an email, give me a response." Yeah.
- 8:45But you actually are doing some critical thinking in that thing that you just said, which is like, "Here's, here's something that someone said. Here's a little bit more about them. Here's a little bit more about me."
- 8:54Context. "What I want is this, this, this, and this." Mm-hmm. "Give me some feedback in order to do that."
- 8:59And I think that that's, that's the layer that I think that we have to be thinking about any of these tools with, is how do I take my, like, knowledge of the world, my knowledge of the context of the situation, inject into- Mm-hmm...
- 9:14what I'm, what I'm using and, and, and put it in. I'd like to kind of pivot the conversation into you have a, a unique knowledge.
- 9:22You worked at Amazon, um, have worked at some of these larger companies and, and you saw it fit to move into automotive as a vertical that you wanted to serve. Right.
- 9:32Why was that the layer that, like, you, you brought to the conversation that was, "Hey, I'm seeing all of these things move in the world, and I'm looking at retail auto and saying, that's where I need to see a, a fit"?
- 9:45Yes. Yes. So we picked up, uh, merchandising as a core problem in the industry, right? So- Amen. Say that twice. [laughs] Yeah, yeah.
- 9:56So mo- m- mostly like digital merchandising was the problem that I picked up in the, in the, in this particular industry.
- 10:02Working at Amazon and so many other e-commerce firms, realized that sellers day in, day out struggle with producing high-quality merchandising.
- 10:10I mean, the bigger sellers can hire expensive photographers, agencies, but the smaller ones, long tail, do not do that. And the photo qualities and the, um, um, in general, media quality is shit. Very bad, right?
- 10:21So, and that also was not helping them sell more products online.
- 10:25So the idea was, can we create something which is more AI first in its approach, understands what product has to be shot, how to shoot it like a photographer, and then process those images into something that Amazon understands, that maybe AutoTrader understand, that...
- 10:40So these, like, larger platforms, they have given the guidelines, and the core shoots are, are tuned for those guidelines.
- 10:46Picked up this, worked in this idea, launched something, ran it for two years, and then figured out that if you, if you are going to do everything, we'll be average at e- everything, right?
- 10:58So real estate has to be shot differently versus fashion model, versus a product, versus a car, versus other, other objects, right?
- 11:06And that's where we realized that, okay, we need to p- pick one category, and car was always a passion. I knew a lot of dealers in my locality. So, uh, showed them the product. They were, like, very happy.
- 11:17Uh, uh, we got it rolled out to some of the biggest companies in the country, in India.
- 11:21And then we saw that this use case was working, like, phenomenally well, and US, Europe being extremely huge market, we thought, "Okay, this is, this is the direction that we'll head to."
- 11:30So you actually went at, like, just general merchandising. Could have been any type of product first. Yeah. Yeah. Is that what you're saying? Yeah. Yeah, yeah.
- 11:37Then, then we focused our energy to build, like, the best product in automotive. That was- In auto. Yeah.
- 11:41I think that that's, that's an interesting start and pivot because when you do look at Amazon and you look at these large third-party agg- aggregators- Mm-hmm...
- 11:50that are, that are very similar to auto, I, I, I always, I always make this argument, like, everyone's like, "I don't wanna be on the third parties," and it's like, well, the rest of the world operates that way, so why wouldn't we too?
- 12:00I mean, anybody like Amazon, right? That's a third-party aggregator- [laughs]... of products. Um, right?
- 12:06But when you think about, you know, the global mark- the global merchandising efforts, you know-Nike spends millions and millions and millions of dollars a year to merchandise their shopping results pages, right? Mm-hmm.
- 12:18Because they know that that next click is critical, right? Mm-hmm. Yeah. And, and we kinda like, you know, wait 11 days till we get a car photoed- Yeah... and online- Yeah...
- 12:30you know, because it has to get through the shop or whatever. And then, and then it's maybe got photos and we've wasted 11 days, and I think that that merchandising flow in auto, you know, obviously we have unique VINs.
- 12:42It's a little bit different than like one shoe that you can shoot one time and have available for six months. Yeah. Mm-hmm. Um, so it's a different type of problem, but I love that you're trying to solve it. Right.
- 12:51What have been some of the hardest things that you've encountered along the way when you're like, "Well, we set out to build it this way, but that was actually harder or that, that was a mistake or, or, or a thing that you didn't expect along the way?"
- 13:05Yeah, I mean, the goal that we, th- with which we started was we saw CarMax, uh, merchandising in very depth, CarMax or, uh, Carvana's merchandising in very depth, right?
- 13:16And understood that some of the largest companies are setting up those 100,000 worth of studios in every single location to do their photo shoots.
- 13:25And the idea was that a small dealer would not be setting up that 100,000 worth of booth because of real estate constraint and maintenance constraint and initial investment, right?
- 13:34But they all want that kind of a catalog, ki- ka- uh, uh, merchandising experience. Idea was can we get every single dealer to that particular experience without having to ins- invest in that studio? Mm-hmm.
- 13:48That was the idea. Mm-hmm. That, uh, I mean, it should be indifferentiable from human eye whether the car was actually shot in the studio or shot outside. Mm-hmm. That was the benchmark that we were tracing.
- 13:59Uh, and, uh, we launched the product. We, we... I mean, you can check some of the studio. These, these are very realistic, uh, uh, glassy kind of reflections of the car, uh, when, when we produce outcomes, right?
- 14:11And, uh, so we, we create behind the scenes some 70, 80 computer vision models focusing on reflections, focus- focusing on stabilization, focusing on interior view generation, focusing on like a lot of these problems.
- 14:25We understand like hundreds of parts of the cars, and every part are, are, are, are tuned in in a different fashion when we process the image for, for this car.
- 14:33So the more hard problem that we got into was, one was stabilization, because when you shoot a car and move around it, your hands are unstable. Like- Yeah...
- 14:42and you, when you try to place it in the studio, your car will look wobbly because the studio is perfect. Understood. How do you make that, uh, car spin perfect in that particular studio? So that was- Mm-hmm...
- 14:52one of the technologies that we worked almost like, uh, over a year to, to perfect with. Now, if you see our 360 spins, no matter how you shoot it outside, it'll always be perfectly placed in that particular studio.
- 15:03Very, very smooth. This is one problem. The second was lighting problem, right? So exposure, lighting, some of these things. Oh my gosh, yes.
- 15:10[chuckles] If, if you shoot in the open yard, like harsh sunlight or s- during the la- later half of the day, so you'll see all, like one side is lit, other, other side is dark.
- 15:20Some of these problems we, we kept on encountering, right? Mm-hmm. So we, we created those exposure correction, color correction algorithms that evens the light on the car.
- 15:29And, uh, based on the studio's, uh, outlook, it will, it will, it will project that kind of a tint on the, on that particular car. So it will look like, uh, that it was shot in the studio.
- 15:39Some of these things we did which, uh, which were like really hard problems. Like, we stayed on these problems for months, six months to 12 months. How long did it take you to make sure that mirrors didn't disappear?
- 15:51'Cause that was my favorite thing in, like, early... Uh, even like two, three years ago. [laughs] Like, it's just missing a whole mirror, right? Yes. It... Or like the tires are gone.
- 16:00[laughs] It's like, who, who lifted this car and made it a hover car, back to the hover car thing, right? Yeah. But, like, why, why was that an issue, and how did you solve it? Yeah, yeah.
- 16:11So, so whenever you create any kind of computer vision model or any kind of AI model, it has to understand the shape and size of that object on which it is operating, right?
- 16:20So that was also one of the reasons why we went into automotive, because the shapes were defined. Earlier, we were doing like multiple categories. So I mean, a human versus a, a bag versus anything, right?
- 16:32So everything has a different shape. So when we picked up cars, we trained our algorithms with, uh, I mean, few millions of images, which were custom trained by our own an- annotators.
- 16:44Uh, and we specifically fi- focused on this problem, like mirror was one of the biggest hardest problems that we solved. Mirrors and antennas. So the, the- Oh, yeah, the antennas, for sure...
- 16:52the, the, the antennas, right? So these two were one of the toughest problems that we solved in, in background removal, as well as those open, uh, open door shots that you take, right? So when, when things can go wrong.
- 17:04So some of these, like specific, harder use cases, we generated our own custom data in, in, in, like, hundreds of thousands, and then trained our models to make that to work.
- 17:12Now, now it works with like 98, 99%-plus accuracy. And you- So-... you give it any kind of image, it'll work... I mean, you all do, you all do so much more than photos.
- 17:21Um, you know, I heard that you're building, like, an AI-powered teammate, and I'm assuming this- Mm-hmm... kind of like delves into the whole realm of agentic A, A- AI. Yeah. And so, so tell us what you're excited about.
- 17:35Like, what does the roadmap look for, look like for what you're building right now?
- 17:40So, so we, we are, we, we are, uh, I mean, visualizing us as a company which will build AI-first experiences for automotive industry, right?
- 17:50So merchandising was one of the use cases that we targeted, and merchandising's natural extension is to go in, in other use cases of inventory, which is how do you price, how do you inspect, how do you syndicate the car well?
- 18:02So this is like one, one spectrum on which we'll go very deep and build AI-native experiences. Mm-hmm.
- 18:08The second, second part where we are now getting into is the, this entire, uh, uh, customer interaction of the dealers, right? Mm-hmm. Using a- agentic AI.
- 18:16So whatever we are doing now, we are looking to bring in agentic AI first behavior.So we are building like bots for sales, service, finance, uh, and these could be like voice chat, uh, emails, everything. Mm-hmm.
- 18:30And we deployed in, uh, in like variety of use cases. So we are already doing like, uh, a number of pilots with, uh, dealers, uh, here and, and seeing like amazing, amazing results.
- 18:40Explain the agentic AI just a little bit because I think some people get lost with that. Like- Yeah... explain exactly what it's doing and what you're hoping to have it, uh, accomplish.
- 18:49Because when we talk about AI, a lot of people are talking about like the ability to converse with it. But you're take- Yeah... talking about taking action.
- 18:56So go into like what's, like in, in a specific use case, what are, what are you seeing the agent- agentic AI doing?
- 19:04So agentic AI, I mean, at conceptual level it's, it is like you do set of things and, uh, that, that, uh, lead to certain results happening, right?
- 19:14Agentic AI is just automating all of these things where, uh, instead of a human- Mm... that part- particular agent is doing things and then presenting you with the results that, "Hey, I have done this.
- 19:25These all are the results." Uh, I mean, if, if you want to set auto rules that, okay, if these results, uh, are, are, are above 90% confidence, just go ahead and execute them.
- 19:35Or, or I need to like check with it and, and, and do the action that whether I need to take it or, or reject it, right? So agentic AI in general is automating a lot of stuff that you do.
- 19:46Uh, instead of you working on the software, the software is working for you, and it is telling you the options that, "Okay, I've done this. Do you want to present it? Do you want to publish it?" Mm.
- 19:56Even that ca- could be automated, right? So merchandising we are i- implementing a lot of agentic AI.
- 20:01Like we d- did the short, instead of users signing off, we are, we are, we are, we are basically signing off all of these things.
- 20:07We are changing the cars images in real time on website without informing the dealer, right? Mm. Based on which, which image is doing better.
- 20:15So some of these, these things, uh, we are, we are automating on its own because we are controlling the website infrastructure now for media. And similarly here- Mm... so some of these things, right?
- 20:25So the car- So like which, which image should be the first image or something like that. Yes. Yeah, yeah, yeah. That, that is one of the- Sure... one of the first use cases, right? Yeah.
- 20:32When you get into pricing, you will be able to... When you manage the ads, let's say there's an agentic AI which is, which is ad- which is contextualized or trained on the ads behavior.
- 20:43So it automatically will, will increase or decrease the budgets or, or choose the mediums where it needs to spend more based on how the clicks are, uh, performing, right? Mm.
- 20:51So you don't need a person to do all these things for you. Mm-hmm. The, the agentic AI which is trained on those use cases will do it for you, and it is way more efficient. Mm. Yeah. That's super cool.
- 21:02I think, I think that the, the name of the game with AI is efficiency, and it's how do I take the tasks that can be repeated, understood when it comes to data, and hand it off to something that can do it faster and more efficiently to free my people up to do the thing that they're best at.
- 21:19Absolutely. Sanjay, thank you so much for spending time with us today.
- 21:22We, we walked the gamut from like what you saw on Amazon and, and what we saw er- in the early 2000s to now what, what you're doing and what you're, uh, w- what you're innovating on top of, and I can't wait to see what you and Spine do as you deeper dive into automotive.
- 21:37Thanks for joining us today on Auto Collabs. Yeah. Thanks a lot for having me here, Kyle, Paul, and, uh, Michael. And great, great having chat with you. You guys are like cute, fun.
- 21:51[upbeat music] One of the things I'm super impressed with is the hyper-focus of Sanjay and his team to fix small problems and how all of those things- Mm... add up to fixing big problems. Yep.
- 22:03Uh, literally that sounds maddening to me, focusing that long on the problem of reflections.
- 22:09But i- it's indicative of the fact that they understand that the small things impact the big things, and we see that a lot even as we deploy AI quite a bit within our business.
- 22:18If you don't focus on the small things, the big things end up being terrible. This is like the worst part of our human nature, which is that we wanna perpetually be at the top step already. Yeah.
- 22:29And we never wanna climb the steps to get there, not realizing that every step gives us a lesson needed to actually stand at the top step.
- 22:37But like, you want a real-world, very painful implication of why people need to focus on the smaller thing and be great at it, look at that...
- 22:45Dude, I just watched that documentary about the, the, the guy, the deep blue ocean, whatever.
- 22:50The, the- Oh, he went down in like the, the whole submarine imploded And, and there was so many warnings of like, "You need to focus on that thing and make that thing better."
- 22:58And next thing you know, it's like poof, implodes in the middle of the ocean, right? And- Mm... and we see that, and we're like, "What an idiot." But we're doing these things in our businesses every day. Mm.
- 23:12We're ignoring- Mm... y- you know, the small thing. And so to your point about, that's very- Yeah... inspiring to have the discipline to focus on- Instead of, instead of asking ChatGPT, "How do I change the world today?"
- 23:23Right? Right. It's like, how do I make sure that this employee is able to deploy this specific task and function well within my organization?
- 23:36Those type of specific questions, like, how do I make sure that mirrors show up every time I do a background replace? Right.
- 23:43You know, that's how specific we have to be in everything, and especially when you're deploying AI tools across your organization.
- 23:49Doesn't matter if you're like doing communicative, uh, communicative AI, agentic AI, you know, photo backgrounding, whatever it may be.
- 23:58Being very specific with the task that that, um, person or technology is accomplishing is, is gonna be extremely important as we move forward. Fewer, better. There you go. Well- Fast...
- 24:10thank you to our friend Sanjay and the team at Spine for putting their heart, soul, and all the intention to making this industry better.
- 24:17On behalf of Kyle Mountsier, Michael Cirillo, Michael Cirillo's new Ray-Ban Meta glasses, and myself, thank you so much for joining us on Auto Collabs.
- 24:27Sign up for our free and fun-to-read daily email for a free shot of relevant news in automotive, retail, media, and pop culture. You can get it now at asotu.com. That's A-S-O-T-U dot com.
- 24:40If you love this podcast, please leave us a review and share it with a friend. Thanks again for listening. We'll see you next time.
- 24:47[upbeat music] Welcome to Auto Collabs. [laughs] Welcome to Auto Collabs. [laughs] Why are we recording? Are we rolling yet?
- 25:05Not yet.