Latest / Agentic Conversations (formally mlops.community) / From Robotics to Recommender Systems // Miguel Fierro // #240

Transcript

From Robotics to Recommender Systems // Miguel Fierro // #240 · Agentic Conversations (formally mlops.community)

0:00
I'm Miguel Scherer. I'm a Principal Data Scientist
0:05
Manager at Microsoft and I like black coffee.
0:11
Welcome back to the ML OPS Community Podcast.
0:13
I am your host Dimitrios. And today, talking with Miguel,
0:17
we actually had a bit of a moment after I stopped recording
0:23
and he told me that he changed the direction of his career
0:28
because of the value that he saw in recommender systems.
0:32
And this comes hot off of the tail of an episode that we
0:36
recorded a few weeks back with Sanket on the Spotify
0:40
recommender system. But he read a report from Amazon
0:45
and I think this is the Amazon shop, not AWS, and it mentioned
0:50
how recommender systems were 35% of revenue generated.
0:57
So basically just add a bead to that 35 and it was like $35
1:03
billion that recommender systems generated for Amazon.
1:09
And when he saw that he was in the computer vision and robotics
1:13
field and he said I got a pivot. This is too big and there is too
1:18
much value that is happening with Recommender Systems for me
1:23
to leave it alone. That was just one of the gyms
1:27
that he dropped in this conversation.
1:30
So the last piece on that Recommender Systems example that
1:33
he said that I am going to continue to replay in my head is
1:38
how at Amazon just tally up the amount of data scientists that
1:43
work at the company and think about the ROI that they drove
1:51
$35 billion worth of revenue generated from these.
1:57
Let's say maximum, there's 10,000 data scientists at
2:01
Amazon, which I doubt there are, but you do the math.
2:07
I don't do public. I don't even do back of the
2:10
napkin man. So in the spirit of recommender
2:15
systems, I thought let's do something new and let's do
2:18
something fun for this intro. And I've been listening to all
2:22
kinds of fun music, and I wanted to go into the conversation with
2:30
a music recommendation from my side that hopefully we'll infuse
2:36
some new musical recommendations wherever you listen to music.
2:42
So this song's called The Look, and it's from Metronome.
2:47
It's so good, I'll let y'all rock into the conversation with
2:53
him now. Here we go.
4:32
That song is the look from Metronome.
4:36
What a jam. As always, if you enjoy this
4:41
episode, just share it with one friend and let's get into the
4:45
conversation with Miguel. Let's dive into robotics.
4:54
And I think for me, one thing that I constantly think about
4:58
when it comes to robotics like deep learning use cases or just
5:03
robotic computer vision systems in general, is how much of these
5:09
use cases are similar to quote unquote traditional mill and how
5:15
much are different? What are some things that you
5:17
can take from the deep learning or computer vision use cases,
5:24
the robotics use cases, autonomous systems use cases,
5:27
all that, and they also work with traditional machine
5:31
learning. And what are some things that
5:32
are totally different? And you're like, Nope, you
5:35
cannot. That has nothing to do with the
5:38
traditional ML. That that's a very interesting
5:40
question. And I can start by saying that a
5:43
lot of robotics will say that in many use cases you don't even
5:50
need machine learning. So not even talking about deep
5:53
learning. So, um, if you think about the
5:59
So Boston Dynamics is arguably the best robotics company,
6:05
right? And they be running for several
6:09
decades and until very recently, they don't even have risen
6:13
learning like everything they had was an physics from the 18th
6:21
century. So it's LaGrange Newton is a
6:25
very, very deep understanding of the equation of motion because
6:30
that's all it is. So is the physics that people
6:34
will study in first year of engineering school or or math or
6:39
physics school? But I'm really, really deep
6:42
understanding of that. And they use a technique called
6:47
model predictive control that basically is a very simple
6:51
technique that allows you to in real time, you compute the and
6:58
movement dynamics of the robot and, and, and it's all the time
7:04
kind of doing a prediction and then getting information of the
7:10
posture from the sensors and stabilising the system.
7:15
And that is done in, in, you know, in C++ like really you
7:17
need, you need to code all that in, in a very low level
7:22
programme. And that's what they've been
7:26
doing for decades, actually. And then of course, if you start
7:34
thinking about computer vision or, you know, not only the force
7:39
and the control and the stability, which is actually one
7:43
of the biggest problems in robotics, maybe the most
7:46
important problem in robotics, the, the stability, the postural
7:50
control. Actually, my, my thesis was
7:52
precisely on, on that, on, on postural control, right.
7:55
And so if you, if you start thinking about the computer
8:00
vision, that's a completely different thing to the point
8:05
that in many research research centres and you know, typically
8:12
where, where people do a busy you, you, you typically go to
8:17
university right now. I think nowadays you can even
8:20
might do be able to do some business in companies, but you
8:24
know, traditionally is universities.
8:27
So interestingly, most labs, they decouple these problems.
8:34
So you have a lab where you have a humanoid robot or you have a
8:40
what is called a, a mobile manipulator, which is which is a
8:45
robot that maybe have has hands, but they have a platform with
8:49
wheels that moves. So they don't have the problem
8:51
of of legs, the stability. They might use the same
8:55
platform, but if they use computer vision, it's a
8:59
completely different thing. And maybe the thesis or the
9:04
research line is, is more about AI, more about machine learning,
9:08
deep learning and that kind of things.
9:11
But for example, and I so I did my PC between university in
9:16
Spain called Kafeero and Kings College in, in London an and in
9:22
both cases the, the teams were specialised, you know, like why
9:28
what people do typically is they, they kind of decouple the
9:30
problem. So that's why, for example,
9:32
that's the reason, that's the main reason why you, you
9:35
probably hear that Boston Dynamics, they, they kind of
9:38
started machine learning very recently because the, the big
9:43
problem that they had to solve is the stability problem.
9:46
And you know, they solved it amazingly.
9:49
And just to finish this point and something very interesting
9:54
that is happening is these kind of there are two school of
9:59
thoughts in robotics, particularly in humans, which is
10:03
kind of what I, I spent a lot of time researching, which is the
10:08
the type of actuators. You can have electric electrical
10:11
actuators, but you can have hydraulic hydraulic actuators.
10:16
And until I think two months ago or something like Atlas and all
10:24
the robots, they they had hydraulic actuators, right?
10:28
Like I think the dog also had some some electrical bat.
10:34
You know, there is these these kind of, you know, difference
10:38
right there. And it all has to do with the
10:41
torque and the, and the ability to control, right.
10:45
Like the benefit of hydraulic actuators is that they can
10:51
produce a higher torque. So if you if, if you are in the
10:56
situation of of and stability, what you're not stable like
11:00
everybody have seen all these people kicking the robots,
11:05
right? Like so a hydraulic actuator is
11:11
very useful in that case, right? Because you have, you need to be
11:16
able to stabilise and very quickly.
11:20
And and that stabilisation, actually it, it creates a lot of
11:24
forces and a lot of inertial forces that you need to
11:28
compensate. And doing the same thing with an
11:32
electrical actuator is much more difficult.
11:37
And if you, if you've seen, if you seen the actually you know,
11:42
if you've seen the, the, some of the demos of both of the, the
11:45
new robot, the newcomer robot of Boston Analytics, I was, I was
11:49
impressed that they don't use electrical, they don't use
11:52
hydraulic actuators. They use electric actuators.
11:56
But you haven't with these new robot.
11:59
They are not doing the backflips, they are not doing
12:02
all that stuff. Somehow I wonder and I'm curious
12:06
to see if they will be able to do to have the same stability
12:12
skill with with the with these these electric actuators that,
12:17
you know, they are cheaper. They you know, it's, it's kind
12:20
of easier to fit these to power these actuators.
12:26
So, yeah, you know, it's, it's a very, very interesting, you
12:29
know, difference. So if I'm understanding this
12:34
correctly, you go with the electric ones because they're
12:36
cheaper because they're easier to implement, but they don't
12:39
have that torque? Yeah, exactly.
12:41
It's not that easy to. It's not that easy to do
12:44
particularly, particularly when you have a very, a very strong
12:52
movement right like that is that is where the electrical actuator
12:57
is, is, is more difficult, it's much more difficult to control.
13:02
And so this is also fascinating because you're mentioning Boston
13:07
Dynamics just started throwing some machine learning into what
13:11
they're doing. It is that almost repetitive
13:15
mantra that we hear on this podcast quite a bit, where if
13:18
you don't need machine learning, there's no need to try and use
13:22
it because yeah, you're going to make your life a lot easier if
13:25
you can do it without it. So it's almost like putting
13:29
machine learning on your robot is just about as useful as
13:34
programming in string theory physics.
13:36
To the robot, it's like, no, we just need this 1800s Newtonian
13:41
type physics that's going to help us with force and help us
13:45
with movement. Yeah, and very.
13:48
You know I can give you an example, right?
13:50
Like if you, if you ask if, if you ask the traditional, like
13:56
old school robotics, like they control engineering roboticists,
14:00
they will say that they don't, you know, they don't need to use
14:03
that. And for example, I remember, I
14:05
remember when I was doing my PhD, I so my, I was researching
14:12
in in postural control. So I had a humanoid robot and,
14:15
and I needed to do posture different posters.
14:19
And one of the things I tried was to use a neural network to
14:24
compute a movement and to do what is called the inverse and
14:28
direct direct kinematics, which is basically how do you move the
14:33
actuators to produce a specific trajectory in the hand or in the
14:38
leg, right. And, and what happened to me,
14:42
you know, I started looking into that and it I realised that it
14:46
makes no sense because we know the analytical equations to
14:52
solve that problem. So we know the exact equations
14:56
and we can do the control group. So I'm I'm machine learning
15:01
solution for that problem. It will be just an
15:03
approximation. He he's like, you already have
15:08
the formula, right? Like machine learning.
15:10
What it does is just he's kind of doing an approximation of the
15:13
formula, but you do have the formula.
15:15
Why are you going to need that? But The thing is obviously, you
15:23
know, like understanding the physics and, and, and you know,
15:26
like formulation for something as complex as human robot is
15:31
challenging. I find it very challenging.
15:34
And, and, and I know, you know, it's challenging for everybody
15:37
because very few people actually solve, solve that problem.
15:43
And so that's why a lot of people don't use machine
15:46
learning. And even to the point that
15:48
instability and control for robotics, there are like two
15:53
kind of models. What is what is called the the
15:57
compressed mass model, say something called 0 moment point.
16:02
That basically what it means is that you create a model of the
16:06
robot that is like an inverted pendulum.
16:09
Like imagine you you have, you assume that all the mass of the
16:12
robot is in the centre of gravity and then when the robot
16:16
moves you use control algorithms to actually make the stability.
16:23
And the other approach is that distributed mode that actually
16:28
is you. You take all the different
16:31
joints and all the different masses of the robot and you
16:34
model that in equations. So you have many different
16:41
questions, as many as as the masses and joints you want to
16:44
put into the robot. And then you solve the equation
16:48
of motion that actually there are two formulations.
16:51
You have the formulation of Newton, which is the famous 4
16:54
sequels, mass, time acceleration, the famous formula
16:57
of neutral. There's another formulation of
16:59
LaGrange which based on energy, like potential energy and
17:02
kinetic energy. So basically you have these two
17:06
formulations, but at the end it's something that is solved in
17:09
the 18th century or something like that.
17:12
But it's it's really so complex that a lot of people use this
17:16
compress way of modelling the robot, which is made easier to
17:21
model in math, but is is less accurate.
17:27
Hit reminds me of basically like a mono repo versus micro
17:30
services, yeah. Exciting is the Mona Reaper.
17:34
Yeah, the famous. Moreover, Google.
17:36
Yeah, totally. So excellent.
17:39
I really appreciate you breaking that down because I had no idea.
17:43
And I do know that there are a lot of robotics companies out
17:46
there that aren't using ML, but then there are quite a few that
17:50
are incorporating a lot of different ML and computer vision
17:55
use cases into their mix. And so it's interesting to see
17:59
like how complex things are. Like you were saying, yeah, we
18:05
could use some ML and this use case, but if we already know it
18:11
and we can use it, we can map it out properly with a heuristic,
18:15
then let's just do that because otherwise the mill is going to
18:18
be an approximation and it may or may not get it right.
18:22
So that that's cool to think about and a nice little thought
18:26
experiment on whether or not it's it's totally used or
18:29
needed. I do know that you also messed
18:32
around when you were doing your little robotic stuff with
18:37
computer vision football use case.
18:39
Can you tell me more about that? Yeah, so I'm a football fan, you
18:45
know, in in the States, you you people call it soccer, like in
18:48
Europe with college football. And and that was that was that
18:53
was another case for it wasn't during my my case.
18:57
It was, it was later, it was when I had in Microsoft and
19:01
there is this big trend of a sport analytics and, and I
19:08
think, you know, sports are going to professionalise in the
19:12
sense of being more data-driven. So basically the the idea was to
19:19
track the different movements of the different teams and the ball
19:25
to, to basically get some analytics.
19:29
And I think, you know, at Microsoft there has been some
19:32
very interesting work around that area.
19:35
We partner with some football clubs in Europe.
19:40
I know Google also they they've partner.
19:43
There's a paper by Deep Mind actually with Liverpool.
19:50
They're in in the UK. They are doing this kind of
19:53
analytics and you know, even even Real Madrid, I know they
19:58
have, they have like a a CTO that is super into, into these
20:03
analytics. And, and that includes a lot, a
20:06
lot of computer vision, like, like object detection, object
20:10
tracking, umm, and you know, I think a lot of these, a lot of
20:16
these sports, not only football, but you know, tennis or
20:20
basketball or many other, many other footballs, they are going
20:24
to start kind of professionalising more.
20:27
And, you know, maybe you will see in the next 10 or 20 years
20:33
that they, they'll have a, like these professional clubs.
20:36
They, they, they have a lot of funding.
20:39
They will have their own data science team and then they will
20:43
get analytics and you know, like decides will be shared with the
20:47
with the coaches and specialists in the team for improving that
20:54
the performance on the team. And also another thing that I
20:58
think is very trendy is data science or machine learning for
21:02
the health monitoring of the athletes.
21:08
Again, you know, like this, typically these people, they
21:10
have the, the, you know, the best doctors and you know, like
21:15
if you heard, if you heard your knee, normal people who stay in
21:20
bed like six months. So these people they, you know,
21:23
it's like no, no, no, for for these people, it's maybe one
21:25
month or something like that, right?
21:27
So I think, you know, getting this combination of of really
21:33
advance medicine with new technologies like data analytics
21:38
or AI, I think, you know, is really, really powerful.
21:42
And what are some of these use cases with the object detection
21:46
and object tracking? I know I've seen a few that are
21:49
really cool and we have a few people in the Milos community
21:52
that also work with basketball and and making sure that you can
21:59
get a proper readout of the players, how far they're
22:04
running, if they're passing, if they're making mistakes, that
22:08
type of thing. But it almost feels like by
22:12
saying, OK, let's add some object detection to the videos,
22:16
it's starting with the tech and not really starting with the
22:19
problem. So what are the problems there?
22:21
Yeah. So I think, I think it's mostly
22:25
strategy. I I remember there is a, a show
22:30
that it was the Spanish national team when they when they won the
22:34
World Cup and they were actually talking about one specific goal.
22:42
And one of the one of the defenders said to he was a
22:46
corner, right? So he, he, he basically said to
22:50
the other guy, he said, you know, I notice that they, you
22:55
know, the other team, they are not defending in the middle of
22:59
the in front of the goalkeeper, right.
23:02
So if you put it to the centre, it try to centre.
23:07
And what I'm going to do is I'm going to go from back, I'm going
23:12
to run and I'm going to jam and I'm going to try to get a goal
23:16
with my head right. Yeah.
23:18
And then and then actually the guy, the guy did it and they
23:23
scored. So, yeah, like the way I think
23:27
about it is, you know, obviously these guys are professionals.
23:29
They they, they've been playing football or their lives and
23:32
they, they are the elite people, right?
23:36
And, and they don't, they are not only, it's not that they are
23:38
really skilled in, in plain, but they understand the, the game
23:43
as, as, as kind of like as a coach, right?
23:47
So I can imagine that that's the kind of information that an
23:51
artificial intelligence can also understand somehow.
23:55
And I, I don't think we are close to, to be able to
23:58
understand that. I, I also don't think that
24:03
there, there are, there is enough funding to reach that
24:06
level. Maybe maybe, you know, some
24:09
clubs start investing in that and you know, in, in 10 years,
24:13
we, we have an advisor that, that advise the coach and, and
24:16
can do some strategies. But you know, in a similar in
24:20
national way that these people kind of saw a hole in the, in
24:26
the strategy of the, of the other team.
24:28
I can imagine an AI can can do something similar.
24:32
Those real time analytics, yeah, this, it's incredible to think
24:36
about. And it's also really cool to
24:38
recognise that we're still very, very early in the abilities and
24:43
the capabilities. And I know with American
24:45
football, they're also doing a tonne.
24:49
And even when you watch American football games on like Amazon
24:54
these days, you can see that they'll break it down and you
24:57
get those real time analytics just piped onto your screen and
25:02
you get a bit of a a bit of a taste of what I imagine the
25:06
coaches are seeing. And so that that's awesome.
25:11
Alright, real quick, some words from our sponsors, Ellis, and
25:14
then we'll be right back into the programme.
25:16
Are you struggling with building your retrieval augmented
25:20
generation AKA RAG applications? You are not alone.
25:25
We all know tracking unstructured data conversion and
25:29
retrieval is no small feat. But there's a solution available
25:33
now. Introducing Soliz Cloud
25:36
pipelines from the team behind the open source vector database
25:40
mill. This This robust tool transforms
25:43
unstructured data into searchable vectors in an easy
25:46
manner. It's designed for performance.
25:50
Zellis Cloud platforms pipelines Zilis cloud pipelines also offer
25:57
scalability and reliability. They simplify your workflow,
26:01
eliminating the need for complex customizations or infrastructure
26:05
changes. Elevate your rag with high
26:07
quality vector pipelines. Try Zellis Cloud Pipelines for
26:12
free at sliz.com. You can find the link into the
26:16
description. Let's get back into this
26:17
podcast. Alright, so then let's go into
26:21
what you are most known for now. And I think that Microsoft, your
26:26
nickname is Mr Recommenders. Is that true?
26:29
Well, I mean like I, I get a lot of I allergies they call me dad
26:34
about. I definitely get a lot of
26:38
requests to support teams that are doing recommendations both
26:44
internally and externally. There's a.
26:46
And what are some of these patterns?
26:48
Because we, as I mentioned before we hit record, we
26:51
literally just had Sankey on here from the Spotify
26:54
Recommender Systems team. And he was talking about how
26:58
they had set up their vector stores and their embeddings to
27:00
be able to constantly retrain you as a user and your
27:05
embeddings to make sure that it's the most up to date and
27:08
gives the best music suggestions.
27:11
But then it also is powering many different teams.
27:14
So it's like the vector store, anybody can grab those
27:18
embeddings of the user and then plug them into whatever machine
27:23
learning model they are doing. So whether it's search or it's
27:26
Discover Weekly or it is the actual like next song that
27:31
you're going to play, all of that is quite useful because
27:36
they can transfer that learning and there's like one central
27:38
spot where they have those embeddings.
27:41
Are you seeing common system design when it comes to
27:45
recommender systems? Because I also know that there's
27:50
like feature stores for very real time recommendations and it
27:56
feels like the vector store and the feature store, depending on
27:59
the use case can be very complimentary or you may only
28:03
need one of them. What's your gist on all this?
28:06
Because I'm sure you've seen a tonne.
28:08
So you think the the cool thing of being at Microsoft in
28:15
comparison to being in a place like Spotify or Netflix or
28:21
TikTok, which they have amazing recommendation systems, is that
28:26
in Microsoft at least, I've seen many different use cases, right?
28:31
So I've seen and I've worked on like a team that they have a
28:38
recommender team. So they have many different
28:42
recommenders and they've been involved in that over the years.
28:47
And I work with companies, for example, that they are starting
28:49
to integrate maybe their first recommender or the the first few
28:53
recommenders. And you know, it is like the,
28:58
the use cases for this to kind of things are completely
29:04
different. And, and basically it all has to
29:06
do with the maturity of the team, the number of people, the,
29:09
the amount of funding each of these teams have.
29:13
And, and you know, what I would say is that in general, there
29:17
are three types of architectures in recommendation systems.
29:22
Yeah, the first one is what we call the bots architecture.
29:26
You get your data, you train your machine learning model, and
29:31
instead of taking the model, deploying the model and scoring
29:34
real time, what you're going to do is you're going to score for
29:38
each user and you're going to a star these recommendations in a
29:44
database. So you can think of a database
29:48
of the first column is the user ID and then you have all the
29:52
recommendations and then you retrain these every day, for
29:56
example, or depending on the depending on the company and the
29:59
use case. But imagine it's it's every day
30:02
right now. What is the benefit of that?
30:04
The benefit is that from from your front end you are just
30:10
querying a database. This is very easy to get a
30:14
request time. A very, very low request time
30:16
below 10 milliseconds in some use cases, for example in
30:21
ecommerce services or it is very, very important to to
30:26
create APIs that that have these this request time, right?
30:33
And what is the problem? The problem in that case is that
30:37
you're not understanding the needs in real time, right?
30:41
Yeah. So I, I always put, I always put
30:43
the, the example of of the Microsoft Store.
30:47
So if you go to so Microsoft has a store, microsoft.com and you
30:51
can buy from, you know, Office, like Microsoft 365, but you can
30:57
also buy Xbox, you can also buy games, right?
31:02
So what could happen is that I I've been I've been buying
31:08
software or I'd be I've been buying, I don't know, maybe a
31:11
computers and and mouse. So what it the the machine
31:15
learning, what it's going to do is if I train it is going to
31:19
recommend things are related to that.
31:22
But what happens if I go to to the website and I start looking
31:28
at an Xbox? Yeah, right.
31:31
So with the bots architecture, we will we won't be able to
31:35
understand that Miguel, even though Miguel bought a computer
31:40
and software, Miguel right now is interested in an Xbox.
31:45
So you you should recommend, you know, maybe maybe a games or, or
31:50
something like that, right? So that's the that's the what is
31:54
called the real time architecture.
31:55
So in the real time architecture, what you do is you
31:57
train your model and deploy your model.
31:59
So you need some kind of cluster that is able to hold this model
32:05
in real time. If you are using some kind of
32:09
deep learning model like LLM, obviously you know the your
32:14
request time is not going to be as good as if you are just going
32:17
in a database. So you'll you'll need to start
32:20
doing some gymnastics to be able to reduce this this request
32:26
time. And then you have the mixture of
32:30
the bats in real time that that it has different names.
32:34
Some people call it the hybrid architecture, some people call
32:37
it the two step recommender architecture and recall rerank
32:42
our detector. But the idea is that what you're
32:45
going to do is you're going to do an initial training of a
32:51
model and you're going to generate some candidates.
32:54
And maybe these candidates are 1000 candidates for each person
32:58
and you're going to store these candidates in a database.
33:02
And then in real time, what you're going to do is you're
33:07
going to create another model that what it does is rerank the
33:11
candidates for each person. So in real time, what you're
33:15
doing is you're taking one user with with a list of items that
33:22
maybe are thousand items. These are the candidates.
33:26
And then what you're going to do is based on the real time
33:29
information, you're going to reorder that list and show them
33:35
maybe the top ten or the top five for the top one.
33:38
Something like that. Yeah.
33:40
And that is kind of hopefully up from the point of view of of
33:43
developing this architecture is much more difficult, but it's
33:47
actually, it has the best of both worlds, right?
33:49
It's faster than the real time Umm, but he's not his his, you
33:56
know, you can get a really, really good solution in in that,
34:00
you know, and that architecture. Yeah, that's funny.
34:04
And you mentioned that and that's exactly what Spotify was
34:06
doing. And Sankey was saying how most
34:10
of his time goes into making sure that those candidates are
34:13
accurate. Then yeah.
34:16
Cause that's where those candidates and that evaluation
34:19
of candidates and we all I'm sure know this now too because
34:23
of Rags and how popular that has become.
34:26
That retrieval step is so important.
34:29
And for a company like Spotify, I think he mentioned they had
34:32
100 million items that they could recommend at any given
34:36
time. So, well for that, for him it
34:39
was like, okay, out of all these 100 million, which ones are we
34:43
going to make sure that we recommend or which one how can
34:46
we make sure that it is the most pertinent for the person that is
34:50
looking at it? And yeah, so this is fascinating
34:55
and it also makes me think about features and feature stories,
34:59
feature platforms and also the just the embeddings and how when
35:04
you're looking at these different ways of doing it, you
35:08
have batch, you have the real time or you have the hybrid.
35:12
Have you seen common ways of setting U your vector stores and
35:18
setting up your feature stores if you have it, or feature
35:21
platforms? And what does that look like?
35:24
So. Yeah, for us basically the
35:29
everything I worked on was on Azure, right.
35:33
So Azure has evolved from the way they do their vector stores.
35:38
Now we have the Azure ML feature starting vector.
35:42
So that, you know, is pretty, it is pre advance.
35:48
I I think, you know, like these guys, they they have been doing
35:52
databases for for decades, right?
35:54
Yeah. So you know, there's SQL, the
35:56
famous SQL Server, right? So I think basically what, you
36:01
know, like all the team around these databases, what they've
36:05
been doing is they've been kind of evolving the database for the
36:09
new use case and basically the new use case, the most important
36:12
use case right now is AI, right? So I think you know, there is
36:17
this component of of vector stars and and and feature stars
36:23
and and all this stuff that you know, it, it helps, it helps on
36:29
the accuracy and and. But at the end of the day, one
36:36
particular idea of recommendation systems that I
36:38
think is very different to other areas like you know, like NLP or
36:44
computer vision is that unfortunately in recommendation
36:48
systems there is not a clear better solution.
36:53
And that is very that's not good, right?
36:56
Because if you go, if you go to computer vision, you know,
37:02
everybody's doing right now either convolutional networks
37:06
or, or vision transformer and, and that's it, right?
37:12
In the old days, you know, some people might remember there is
37:15
this thing called SIFT key points, right?
37:19
And some other and some other structures like nobody's using
37:22
that anymore. And in for example, in in NLP
37:27
you have LM's, right? Yeah, you know like right now
37:32
nobody, I haven't seen anybody using convolutional networks
37:35
for, for NLP, right. Or few, very few people doing
37:39
LTMS, right. So like what happens in these
37:44
areas in is that you, you see that new algorithms make the
37:49
older algorithms obsolete, right?
37:53
So basically you, you actually, you, you just can try a few
37:57
things and basically you can focus on optimising the few
38:02
options that you have and you know, go from from there, right
38:07
In, in, in terms of recommendation systems, that's
38:10
not true. In recommendation systems,
38:12
depending on the use case, depending on the data, you can
38:16
use linear models and simple models.
38:19
You can use even no material machine learning at all.
38:24
So yeah. And from there to the Super
38:27
advanced LNG, right. So it's very it is quite
38:34
difficult to kind of fine tune your your model to make sure
38:39
that that you are providing value right.
38:42
And another of the and that that's a very big difficulty
38:48
right from of of recommendation systems.
38:51
And so this is a problem. And again, this is a problem
38:55
that as I mentioned at the beginning, if you are very
38:59
mature team, you've tried so many things that you kind of
39:03
know like maybe you have like companies like Spotify or
39:07
Netflix or Techno, they have different recommenders for
39:11
different kind of content. But if you're starting from, if
39:15
you're an ecommerce, for example, or if you, if you are
39:20
any other company that you're starting, you need to go through
39:23
all these different, different tests and different algorithms
39:28
to figure out which ones work for you.
39:31
So you know the same for example, a company like Natural
39:33
is that you mentioned they have millions of of songs to an
39:38
ecommerce that they may have in the thousands of items, right?
39:44
Yeah, the algorithm completely changes, right.
39:49
So one of the one of the problems I've seen recommenders
39:53
is, you know, being able to select the right algorithm.
39:57
And then the other big problem is that at the other day.
40:03
When you are using a recommendation systems, while
40:05
you're trying to do is you're trying to improve the user
40:09
experience, increase the active usage, increase revenue,
40:15
increased conversion, stuff like that.
40:17
So you need to do a Navy test. So basically the data that
40:22
processes the data. Engineers prepare data, the data
40:25
scientists create the model. Then you have either data
40:29
scientists or engineers or different, you know, different
40:33
companies use different names, but you know, some, some somehow
40:37
they deploy this model in any of the architecture that we
40:41
mentioned. And then there is this process
40:43
of AB testing and then again the A/B testing.
40:47
You need a platform as a platform to be able to check
40:51
this model and the, the AB tests, you know, you, you need
40:56
some time for diabetes to, to be significant for, for the, for
41:01
the statistics to, to, to be sound and, and and to be
41:04
accurate, right? So maybe you need one week or
41:07
two weeks for each experiment, right.
41:10
So if you think about it, every time you do an experiment is
41:14
very expensive in terms of time. So maybe you need to wait two
41:19
weeks, right? Yeah.
41:21
And one thing that happens in experimentation and, and, and
41:24
people who is in a B test in experimentation now is like the
41:29
average of industries that 80% of the experiments fail.
41:34
So you're going to try a new recommender and on average for
41:40
most industries, 80% of the time you're going to fail.
41:45
So it is it is a challenging, it is a challenging situation.
41:52
And you know, I believe I believe that of, of AI of of the
41:58
A solutions, the personalization idea is, is one of the most
42:03
powerful and, and solutions and, and, and probably one of the
42:08
ones that have the highest return of investment.
42:11
But getting to these return of investment, it is not easy.
42:16
So one problem that I've seen in many companies that they don't
42:22
fund enough the data science team of, of recommenders,
42:27
because you know, it is something that is not just you,
42:32
you just get a few trials and and then you suddenly you're,
42:36
you're doubling your, your revenue.
42:39
So you need to spend some time doing tests and you know, if, if
42:44
you're a company and, and you know, like this is some makers,
42:48
they maybe don't understand AI very well and maybe they don't
42:51
understand recommendations on personalization.
42:54
They don't see the value. But you look at it, you look at,
42:57
for example, companies like Spotify or TikTok or or Netflix
43:03
or Amazon, you know that they know.
43:07
They they, they, they, they spend resources in these teams
43:13
and their return of investment is, is crazy.
43:16
Yeah. Just to give you some numbers,
43:18
the 35% of the revenue of Amazon.com comes from
43:24
recommendation systems. So if, if so, think about, think
43:28
about that right? Like if last time I, I looked,
43:32
I, I think they were doing over 100 billion in revenues of 35
43:36
billion is basically a team of people like like us, right?
43:43
It's like like if you think about it, and I always put I
43:47
when I pitch, when I pitch this idea of recommendation systems
43:51
at Microsoft, one of the things I do is I go to the tree of, of
43:58
the DPP. So you have Satya Asia there you
44:01
have the different B's, right, and different products, right?
44:04
And you have the sales organisation like it's, it's
44:07
like a tree, right? And then I ask, OK, so who of
44:11
these VP's make 35% of the revenue, Sure they are not that
44:17
of Microsoft, right? There are not that many, there
44:19
are not that many. And there is like the question
44:22
is OK, so if you take the two or three people who make these
44:28
number of billions, how many, how many people are there and,
44:33
and and the numbers in the in the 10s of thousands of people
44:36
in general. So to, to generate 35 billion,
44:40
40 billion, you need, you know, in the 10s of thousands of
44:43
people. Well, I don't know exactly the
44:46
number of people working in in recommendations of on Amazon,
44:51
but I'm sure it's not in the 10s of thousands of people.
44:54
So for for our life that you can get from recommendation system
44:59
is inside. Yeah, wow.
45:03
I had never thought about it like that.
45:04
And this brings me to another point which I think you brought
45:09
up. And it's how expensive
45:11
experiments can be just in terms of not expensive as far as
45:16
headcount goes, but expensive in terms of time.
45:20
And also the fact that 80% of these experiments fail, it makes
45:24
it really hard to justify doing another experiment.
45:29
And so how do you as, because I know you're leading teams right
45:32
now at Microsoft, how do you get across to leadership that it's
45:38
worth making another attempt or trying another experiment and
45:44
getting that buy in to have leadership understand that and
45:50
then also not see you as just a cost centre?
45:53
Yeah, O, that's a very good question because that's
45:57
precisely a big part of my job is basically talk to the leaders
46:03
or my organisation and even other organisations and
46:07
Microsoft to, to actually help them see the, the potential and
46:13
the opportunity. And the only like at least what
46:19
what I've found that is more useful is when you come with
46:24
results. So basically my pets, my pizzas.
46:28
Hey, I'm Miguel. My team does recommendation
46:32
systems and I've done this number of $1,000,000 with
46:37
recommendation system. So this is my presentation.
46:42
So suddenly if you come with salts, actually not my, my
46:49
technique is very simple, right? Like I go, I say, hi, my name is
46:53
Miguel Ferrer and I've been working on this problem and we
46:56
made this amount of money. And then I pause.
47:00
Then you drop the mic. I weighed then I wait for them
47:04
to to ask, right? And because you know, you cannot
47:08
go, you cannot go to, to a person that doesn't understand
47:12
AI or recommendation systems and say, you know, AI is the future
47:17
or recommendation system is the future, right?
47:20
Because, you know, first of all, maybe it's not because and, and,
47:23
and also they have like 1000 different problems in their
47:27
head, right? But you know, if you go to to
47:32
these people and say, look, I I've got these results.
47:38
So I'm not saying I'm not, I'm not telling you that AI is the
47:41
future. I'm not telling you that
47:42
recommendation system is important and just telling you
47:45
that I got the results. They ARAI is 40,000%.
47:53
So what what? What do you want to do right?
47:58
Balls in your court. Yeah, exactly.
48:01
So yeah, now he's what do you want to do?
48:03
Right. And I found these very very
48:06
useful. And how do you calculate that
48:09
ROI and how do you calculate how much money that you've made so?
48:12
The the typical way, the typical way of doing this is and you do
48:19
a Navy test and the in the Navy test, you need to measure some
48:24
key metrics. Depending on the business is
48:26
different, right? You have a revenue leave like
48:29
average revenue per user, for example, is a metric that is
48:32
important in other companies. A conversion rate is important
48:37
monthly active usage like Mao or, or you know, depending on
48:42
the engagement, depending on the, on the business, you, you
48:47
have different metrics. But what you do is you take the,
48:53
you do the experiment and you say, OK, so we increase, we
48:57
increase this metric, you know, 3% or whatever number, right.
49:03
And then you say, well, based on the for example, amount of
49:08
revenue or amount of you know, average revenue per user or
49:12
conversion that we had last year that equates like if if we if we
49:18
use this model that equates to these $1,000,000.
49:22
And I think another thing that I found very useful is to talk in
49:25
terms of dollars, right? You're, you're asking, it's kind
49:30
of I'm asking for headcount, I'm asking for dollars.
49:34
You, you give a result, a result in terms of dollars.
49:39
It's very, very easy to for because it's it's kind of very
49:41
easy for them to to understand. Man, that's fascinating.
49:45
And I find it so cool how you think about the incredible ROI
49:53
when you're looking at it as like, hey, we saved or we
49:57
increase the average sale price or whatever, the average value
50:02
per customer by this percent and it equals this many dollars.
50:08
And then you say, and I did that with a team of 20 people or 50
50:14
people. And so the ROI on that is 40,000
50:19
times for whatever you put in. Yeah, you know, the cost of
50:22
those twenty people was well worth it.
50:26
I think it is very interesting, right?
50:28
Because what I've seen, what I've seen in in particularly
50:32
working in Microsoft is that now thanks to Opening Eye, there is
50:40
a lot of companies that are really understanding that, that
50:45
we are in a new industrial revolution, which is a
50:49
revolution of the eye. But I've been there, I've been,
50:52
I've been selling to customers where when, you know, AI was
50:58
like this magical thing that, you know, nobody really
51:02
understood, right. So it's very interesting how
51:08
the, how important it is that the key decision makers, they
51:13
have an, A, a strategy and, and actually, you know, like I, I
51:19
don't, I don't work with them with the sales organisation in,
51:22
in, in Spain, in Microsoft, but I, you know, I know many people.
51:27
So if you ask these, these people like, what is the, like,
51:31
what is the difference? Like what?
51:32
Why two years ago, like most companies were not interested in
51:37
AI and suddenly now many companies are really thinking
51:42
about AI and having a strategy and it's all about trust.
51:49
It's all about that. You know, two years ago they
51:54
CEO's of these companies were not convinced.
51:57
They were not convinced of AI, right.
52:01
And I think all this and, and something very interesting is
52:04
that all these CEOs, typically, they are all enough to have seen
52:10
the Internet revolution because the Internet revolution was 20
52:15
years ago. So it's not that it's not like
52:18
100 years ago Saturday. No, no, it's 20 years ago.
52:20
So many of us are all enough to have seen this.
52:25
And you've seen companies that they didn't get into the
52:29
Internet and many of them failed, right?
52:33
But you know, it happened the same thing 100 years ago with a
52:37
steam engine, with a steam engine and a lot of companies,
52:41
they were into the, you know, the steam engine and and
52:43
electrification came and all these companies went away.
52:46
But you know, people haven't lived that, but we have lived
52:50
the Internet. So now, only 20 years later, we
52:55
have another revolution, which is I.
52:56
So I think now people are like, OK, I'm not going to make the
53:00
same mistake that people do 20 years ago with the Internet.
53:05
Now I'm going to get into AI and you like from an SC was actually
53:11
it was thanks to Chad Deputy, thanks to Chazy beauty, people
53:15
really started to see a I as a useful solution.
53:21
And and you know, that actually was what convinced the decision
53:27
makers. And then you know when, when you
53:29
have the buy in from, from the CEO, from the decision makers,
53:34
then top down is much easier to to actually start implementing
53:39
these air solutions. Yeah.
53:41
Then the budgets all of a sudden open up and we gotta get
53:44
involved in this and find out what's REI strategy.
53:47
I really like that. Well, man, this has been great.
53:51
I wanna end with one question because I know that we're
53:54
talking about LM and we're talking about ChatGPT and tying
53:58
that into recommender systems. Have you seen many people using
54:02
LMS with recommender systems or is that just over engineering it
54:07
making it too complex? I know there's a lot of papers
54:10
that come out that try and incorporate the two and I just
54:15
wonder, like out in the wild, is it being used?
54:21
Have you seen it? Have you tried to use it?
54:23
Yeah, yeah. So, yes, absolutely.
54:25
So I think there are many ways to combine Rico and elements.
54:33
So you can think of as. You can think of elements as
54:37
just another machine learning algorithm and one of the authors
54:41
of of the recommenders open source library that actually,
54:44
you know, we we donated to the new foundation recently from
54:48
Microsoft. And in a similar way that we
54:51
have, you know, gradient boosting trees or or linear
54:56
models or we also have a transformer based architectures
55:01
or LMC that you can use with, with recommendation systems.
55:06
So it can they can be used just to power these algorithms.
55:11
Another approach is to have a chat bot that is like a
55:18
personalised chat bot. So basically, and when imagine,
55:24
imagine when, when you go into something like, you know,
55:28
Netflix or, or you know, or even an e-commerce, for example,
55:33
right? You're looking for movies,
55:34
you're looking for products that you could chat via voice or via
55:39
text and say, you know how I would like to buy some product,
55:45
right? And then based on this
55:47
conversation, you can actually have and based on your, your
55:52
history, your history of purchases and, and, and your
55:56
preferences, you can have the answer some recommendations by
56:02
by this recommender that actually is is is an LM combined
56:08
with, you know, some other techniques.
56:11
But the other day is, is basically just another chat bot
56:14
that is personalised. And The thing is that they
56:18
interface is different, right? Like you, you have the If you
56:22
think about something like Netflix, for example, every row
56:27
that you see in Netflix is a recommender.
56:30
So there that there is no conversation.
56:32
But for example, I've seen a lot of use cases of on on search
56:37
that the certs experience is being changed to some kind of
56:43
chat bot in a similar way that Google or Bing is kind of being
56:47
moved to a chat. Who experience the search
56:52
experience in an ecommerce probably is going to be moved
56:55
through some kind of table experience as well.
56:58
Fascinating to think about that. Yeah.
57:00
And it does feel like those use cases are more prevalent when
57:05
you don't need that super fast rendering time.
57:08
And you can kind of fudge it because those LMS aren't
57:12
particularly fast. They aren't known for being
57:14
fast, at least not these days. So, man, I know we wanted to
57:20
talk all about you donating recommenders to the Linux
57:23
Foundation. I think we're going to have to
57:25
save that for the next conversation.
57:27
Yeah, so. It's super cool to talk with you
57:31
about robotics and how if you can avoid using ML and robotics,
57:37
you probably should. Also the football use cases of
57:41
your past life and using those real time analytics and how
57:45
we're seeing more ML in that. And then the different styles of
57:49
recommender systems, the batch real time hybrid and of course,
57:56
the ROI on recommender systems, which I think is probably the
58:00
most important piece here. I'm going to be quoting that one
58:03
for a while. So thanks again.
58:05
Yeah, thank you, thank you. Thank you for the for this.