Latest / Agentic Conversations (formally mlops.community) / Democratizing AI // Yujian Tang // #163

Transcript

Democratizing AI // Yujian Tang // #163 · Agentic Conversations (formally mlops.community)

0:00
My name is Eugene Tang right now.
0:02
I'm a developer Advocate a Zyliss.
0:04
I'm actually drinking coffee with milk right now, but when I
0:09
go to coffee shops, my favorite thing to order is two shots of
0:13
espresso with sparkling water over ice and a little bit of
0:16
cream. What caramel shots?
0:19
Yeah, usually caramel shot two shots of espresso over ice with
0:24
sparkling water. Yeah, we're getting water in the
0:28
coffee. Yeah, yeah, yeah.
0:29
It's Actually this load of coffee?
0:31
Yeah, it's really good. If you put a little bit of a
0:33
caramel syrup and like some like cream or like milk in there,
0:35
it's my job. There's not a special name for
0:40
that. I don't know if it's a special
0:42
name for it but this is what I've been ordering for the last.
0:45
I saw it. I saw it at this place called
0:47
espresso Vivace or something like that in Seattle, and then I
0:50
haven't seen it. I the other places but I go.
0:52
And usually if I order it like, they'll have it, like I don't
0:55
think Starbucks does it, but some places some of many like
0:58
locally-owned coffee shops here and Seattle.
1:00
They have sparkling water and so will make it for you.
1:04
Hello. Hello everyone.
1:06
Welcome back to the ml. Ops Community podcast, I am your
1:10
host Demetrios. And today, I'm flying solo
1:13
again. Y'all are probably thinking that
1:17
people are starting to not like me or I must not have any
1:21
friends because I've been doing a few of these solo podcast
1:25
recently and it is not true. I'm just going to refute it
1:29
right now. I do have friends and I could
1:31
ask them but I've actually been Been liking doing this solo
1:35
thing, I am gonna be straight with you right now because I
1:40
always would have co-host, come on because I was always insecure
1:45
that I was not able to go sufficiently deep enough in my
1:49
questioning to actually provide value to you the listener.
1:54
However, now I tried it and I feel like I was all right, I
2:00
feel like I could go deep and I know a few things now.
2:03
I mean Let's be honest, it's been three years.
2:05
I've been doing these, at least twice a week for the last three
2:09
years. So hopefully, I picked up
2:10
something along the way, but as always, if you have strong
2:17
thoughts on, if I should be having co-host come, so they can
2:23
go even deeper than I'm able to go.
2:25
Please let me know in the comments or in the reviews or
2:29
all that fun stuff. I love hearing from you all.
2:32
It is a pleasure being able Able to do this.
2:35
I have the most fun interviewing all of these, great minds and
2:39
speaking of great minds. Let's talk about our guest
2:42
today. Eugene Tang, Who currently is a
2:46
developer Advocate at zeile is zal.
2:48
Is you probably know because Vector databases are blowing up
2:52
there. Exploding on the scene right now
2:54
and Zell is, is the parent company.
2:57
That keeps Mill vez in. Check mil veces.
3:00
One of the ogee databases vector.
3:04
Bases out there and they are built for scale.
3:07
They are absolutely just beasts. You've got some really big
3:12
companies that are using them and these companies they they
3:17
got scale. All right, so let's let's talk
3:20
about that. Now in our conversation today,
3:23
we talked about Vector databases, we talked about the
3:26
democratization of AI in this LM world and how Victor databases
3:31
play in that. And we also Mentioned, all
3:34
different kinds of ideas around the best ways to fully utilize
3:39
your vector database. If you are using it in
3:41
conjunction with a good old large language model.
3:45
Now there is some very hot topics like chunking and what is
3:50
it? A dense Vector representation,
3:52
sparse, Vector, representations. And so we go into that a little
3:55
bit and we also just talk about the Liz and what they've been
4:00
doing and how they have now. Novus light.
4:04
And this is, since we are releasing this today, there's
4:07
also zilla's Cloud. So there's easy ways for you to
4:11
get up and running with a vector database.
4:14
And if you want to choose the Liz or Mill, this it is very
4:18
easy. And I got to give him a shout
4:20
out because they were kind enough to sponsor this episode.
4:23
And because they are sponsoring this episode, we are able to
4:27
continue to bring you some absolutely amazing content in
4:32
the ml Ops Community, it helps Helps us a ton for them to be
4:36
sponsors because we get to do stuff like have meetups all
4:41
around the world, even if meetup.com is costing me an arm
4:45
and a leg. Seriously, if anybody knows any
4:47
better alternative to meetup.com, please tell me I've
4:50
tried Lumen but you don't get the whole like notify people
4:53
when you have an event. I mean meetup.com, seriously,
4:57
how do you sleep at night? That's what I want to know.
4:59
How do you sleep at night? Anyway, let's get into this
5:04
conversation about Vector databases before I fully derail
5:07
it by talking about meetup.com. Anyway, if you can just share
5:15
this episode with one friend who you think would enjoy it,
5:21
alright, let's get into man, what is this?
5:31
You tried a hundred different snacks in the last two years.
5:34
So wait, what are we talking to snacks like chips or like
5:38
guacamole? Is that a snack?
5:40
Huh? - like single packaged?
5:43
Like Goods that are like package for like single single single
5:46
person? Like consumption kind of like a
5:49
like Justin's like peanut butter cups.
5:52
Yeah, like those things or like the Kawaii tote k0i a, I don't
5:57
even know how to pronounce it like the the drinks or like
6:00
energy balls. Yeah, yeah.
6:02
Yeah, yeah. Or like the there's like these
6:04
like cocoa truffles to have some sort of Coca lavender, honey
6:08
stuff in them. They're like pretty my God.
6:10
Yeah that sounds amazing. Yeah.
6:12
They're they're actually they're they're okay.
6:14
There's your record on Tick-Tock your response to so yeah.
6:17
A regular eating these. I have it on the couch
6:20
somewhere. I just I just got Too lazy to
6:22
continue doing it because I was like okay like it just can't be
6:26
on so many social media platforms and well, I got a new
6:29
phone I'll have to take talk on my new phone so I was like all
6:31
right, I don't want to be on Tick Tock.
6:32
So I'm just going to really get rid of it.
6:35
Yeah, it can definitely take up a lot of your time.
6:39
So you're working is Al is right now, your developer Advocate.
6:42
I mean, you've been doing all kinds of cool stuff.
6:45
With Vector databases, tell me a little bit more about your
6:48
trajectory and how you got here. Yeah, I guess I guess I can just
6:53
start kind of at the beginning or after I graduated college, I
6:56
was working at Amazon for a while.
6:58
I worked on the auto ml team so I did, I built like this like
7:01
the infrastructure side for machine learning models.
7:04
And then I left Amazon to kind of start working in startups.
7:08
I started a couple of my own like companies.
7:11
The first one that I started, I like created this app.
7:14
It was like around, like some sort of like data aggregation,
7:16
all kind of stuff. I got a few thousand users on it
7:19
but I never really figured out how to monetize it.
7:21
It this, I shut it down. And that was when I started
7:24
working in startups and I've been to a bounce to a couple
7:28
different startups, kind of doing this kind of developer
7:30
Advocates, kind of stuff. This is my first role as like an
7:33
official developer Advocate before he was like, like full
7:36
stack engineer or some stuff like that.
7:38
And then or like technical consultant during something like
7:41
that. Basically, like I decided, I one
7:43
point that I kind of enjoyed this kind of content creation
7:45
stuff, and I started like writing my own blogs.
7:48
So, last year, I wrote a Blog that got like, 200, Thousand
7:51
views last year. Nah and then, yeah, I someone
7:55
reached out to me as I was like shutting down like my second
7:59
company because you know, I was like okay I figured out how to
8:02
monetize this one but no, like product Market fit.
8:04
So not really making like a few hundred dollars a month.
8:07
So I was like, all right, and go do something else.
8:09
Yeah. Or then they were like, hey,
8:10
there's this company is working. This really cool space and I
8:13
looked into Vector databases because I'd heard about Vector
8:15
databases before, but I never really knew.
8:18
I knew a lot about it and then as I poked into it, I was oh,
8:21
this is like something that I think really has a lot of
8:25
potential to be like an integral part of stacks especially for
8:32
well especially for the large language model applications and
8:35
then kind of just seeing like the other applications of it.
8:38
For anything that requires semantic, similarity is just
8:40
like a very easy way to kind of be searching through those
8:44
documents and stuff and saw like this would be a really cool
8:46
experience and I'm at the team and I was like wow, these people
8:49
are all really smart and then Show me the architecture and I
8:52
was like take a look at this and say oh this is like super cool.
8:54
And so I was really excited to just kind of like it on board
8:58
and here we are and those two companies that you started.
9:02
The first one was around data aggregation, was the other one
9:05
around data in some way to it was around.
9:08
It was a natural language. Processing API was like a
9:11
text-based like processing API. So I was already like in the
9:14
space of doing like this kind of like text-based kind of like
9:17
stuff with like natural language processing and so it made a lot.
9:21
Lot of sense to me, when like Factory databases, were
9:24
explained in the context of semantic search.
9:27
Yeah. You had been stewing it at for
9:29
the past couple of years and probably felt a bit of the pain
9:33
that Vector databases can solve and and solve for y.
9:38
So let's talk about that. A little bit for the uninitiated
9:43
vector databases have blown up in the past.
9:47
I would say six months basically since chat GPT came on the In,
9:52
they've been the unsung hero of this whole generative, a I wave,
9:57
why is that? I really think that I'm like
10:00
prior to this it's not like vector databases weren't being
10:02
used. They were being used for some of
10:05
the more traditional use cases like the semantic search stuff
10:07
and or like image search or product recommendations.
10:11
But kind of the the wave of LMS has really brought up Vector
10:16
databases into more of a mainstream, kind of topic, I
10:20
guess. And the real reason I think for
10:22
that is just because there's so many data problems with lme's.
10:27
Lme's are so we'll take U BT 33. The door 3.5, whatever.
10:31
For example, right? It's only trained up until
10:34
certain time, so it's only train till September of 2021.
10:37
So if you in 2023, you want to be using an llm app in your
10:43
back. End is like, you PT 3 or 3 .5,
10:45
or some like that, you're only getting access to data that.
10:51
Almost like a year and a half out of date now.
10:53
And the only way that you can kind of inject, you can the only
10:56
way you can work with these with modern data is that you inject
10:59
the data. And the other thing is that are
11:00
like Enterprises, they want data that use, they want to be able
11:06
to interact with their data and so they need some way to To do
11:14
to do that. Basically, you need some way to
11:15
inject this data layer and that's really where Vector
11:18
databases have been really popular.
11:20
Is that they allow you to To inject them up-to-date data,
11:25
inject your domain data and if you are work, if you're
11:29
interacting with chat, ubt or GPT, whatever a lot Ellen's a
11:33
lot, it's going to be costly and there's going to be some things
11:36
that our FAQ kind of stuff especially if you're working on
11:41
a large system. Let's say you're at a call
11:42
center, you're going to have customers who are going to be
11:46
asking some of the same questions.
11:47
And so if you have some sort of chat Bots and a lot of customers
11:52
are asking, The same question you're going to want to Cache
11:55
some of those queries so that you can return back your answers
12:00
without having to Ping the LM again.
12:02
So these are like the three main use cases that I see better
12:05
databases being used in for lme's and I think that's why
12:08
they become super popular. It's just because if you're
12:10
building an llm at that scale, there's almost no way that
12:15
you're not using about your database.
12:17
Yeah. Yeah.
12:19
And this is something fascinating that I think about a
12:21
lot. That trade-off of. when you use
12:28
a vector data base, and when you just try to stuff everything
12:35
into the prompt and how you look at that because I know that
12:40
throb expect context. Oh, yeah, you're able to throw
12:45
so much more into the prompt and you just raised the great issue
12:50
is that, hey, it's not only the stuff that you're putting into
12:52
the llm that you're grabbing from the vector database.
12:55
But also, when it comes out of it, you can go and save it in
12:58
there and cash it in there for the next time that you asked
13:02
that same question. In right.
13:03
But how do you see that? Because I think that's something
13:06
that is coming up more and more. Okay, where's the role of the
13:10
vector database if we can just put millions and millions of
13:15
documents into a prompt eventually?
13:19
Yeah, that's I mean, that's a good that's a that's a good
13:21
question, but I think the main thing there is to do you really
13:23
want to be spending a hundred thousand tokens every time you
13:27
talk to a topic or every time you talk to Claude, I don't
13:29
know, I think it's pretty. Can you do that?
13:32
Yes, but I think it's a cost issue and for this lme's I mean
13:35
Vector databases is help solves part of that issue but what you
13:40
really need also are other tools or Frameworks around it.
13:43
Some of these a, I like Asian building Frameworks like Auto GP
13:46
t, or a llama index or length chain, are really offering some
13:50
Tooling around that to make sure that your prompts are reasonable
13:53
or that you're only sending like, the right amount of info
13:56
to your prop. So, I was working with Lane
13:57
chain recently and Ling, Shan has this thing where they limit
13:59
the number of tokens. So you send to to the prompt to
14:02
kind of keep your costs down, but I think the main my main
14:05
kind of got around that is just you probably don't want to be
14:09
using 100,000 tokens every time you talk to Claude, or you're
14:11
going to only talk to it like three times a month, right?
14:15
Yeah, you're going to have to have a damn good reason for
14:19
talking to Claude. If you're going to be doing
14:22
that, yeah, it makes complete sense.
14:24
It does seem like cost can Skyrocket if you are hitting
14:28
that limit continuously. And right so there is there is
14:32
something interesting too that you were mentioning before is
14:35
how it's almost like you have these large language models.
14:38
And we talked with NEOS at cohere while back and she was
14:43
mentioning how How difficult it is if you want to make a large
14:49
language model, understand something from the present, like
14:54
someone died. Yeah.
14:56
And you have to show them so many different examples of this
15:03
person dying to update the knowledge that a large language
15:08
model has, every time the world around us changes and it feels
15:13
like, that's why Databases can be slapped on top and you can
15:18
say all right, cool. Let's let's look at what we have
15:22
in the vector data base first and then we can go and query the
15:26
large language model and one area that I love to explore with
15:31
you and I'd love to hear what you've been seeing is these
15:35
newer models that are coming out that have been open source than
15:38
their smaller. Okay.
15:39
It feels like you can lean on a vector data base, a lot more
15:43
than we originally thought. It doesn't have you been seeing
15:46
that or how have you been, like, noticing the performance and as
15:50
opposed to, I mean, obviously GPT for is a beast in its own
15:54
and it does its own thing and it's very hard to find something
15:58
in the open source world right now.
16:00
That is as good as GPT for, right?
16:02
Yeah, but I feel like, what is going to start happening more
16:08
and more, is this idea of hey smaller models leaning on Vector
16:12
databases more and we're Asking it these crazy complex things
16:16
were just asking it these things that we know it can do, right?
16:20
Yeah. I think that at the heart of the
16:23
at the heart of machine learning models in general and like why
16:25
is it so hard to retrain one of these models while you have to
16:28
show so many examples is that large language models are still
16:32
neural Nets. They're all statistical models.
16:34
Right. They're all using some sort of
16:36
their initialize with some sort of ways, they're given this
16:39
data. And the data, really, this is
16:41
why they call them training neural networks, right like and
16:44
apps. The data.
16:45
And so, when a neural network, for example, I think the the
16:48
you, the, the point that you brought up about having to show
16:51
a lot of examples of The Queen Is Dead, is like a interesting
16:55
point because we as humans, like we hear oh, the Queen of
16:58
England. She like she's past we kind of
17:00
immediately like okay, she's gone.
17:03
I think that's because I guess this is a bit more philosophical
17:06
but like humans had kind of have this like inherent built-in,
17:11
maybe feeling or knowledge of like time and like death.
17:14
And machine learning models. Don't really, I don't think they
17:18
fully grasp the human mortality and so we as humans, we hear
17:22
that. We're like, okay, that's fine.
17:24
But you tell the language model and it's just it's a statistical
17:26
model. So it says, well, based on my
17:29
data, here's 3 million points to say, the queen is alive.
17:33
And here's one that says the queen is dead, so I think the
17:35
queen is alive and like the whole like fine-tuning thing.
17:38
This is the problem with fine-tuning is you if something
17:40
like that is a major update or if Microsoft buys opening II or
17:44
Some like major update like that, it's going to need a lot
17:47
of examples and that's why the whole like vector database thing
17:50
is interesting, right? It's like why you how you can
17:53
have this kind of like data cache data store.
17:55
They're just sits in between the the user and the the llm and
18:00
really what you use. The L M4 is just understanding
18:04
what the user is saying to parse the question again, to reformat
18:10
the query, our reform at the response in a way that makes
18:13
sense to the user in like a conversational kind of like
18:16
sense, right? You wanted to be like a chat box
18:20
so I wouldn't be surprised if really what we end up seeing as
18:25
like the LM stack is just that we can we'll have some sort of
18:28
framework that it's like very easy to switch out these
18:30
language models because Is Chad cbt3 a lot better than Chad.
18:34
Should be two yeses, Chad. CBT for a lot better than Chad.
18:37
Cbt3. Yes.
18:39
But is it something that we can really tell the difference of
18:42
comes to most like Enterprise use cases?
18:46
I don't think so. I think that like, for if you
18:48
have a chatbot that you want to just chat naturally with maybe
18:52
Chad zucchini for out for Chad, cbt3 by W.
18:55
But in terms of like for like most Enterprise scale, use
18:58
cases, what are people going to be doing?
19:00
They're going to be asking questions about about some of
19:03
the data, they just wanted to be able to formulate some sort of
19:07
Chatbot type response that makes sense like full sentences
19:10
paragraphs and a 3.5. And three can really do this and
19:13
vicuna can do this and like llama can can do this and I
19:17
think most of the the open source models have reached the
19:20
point where if you get them right.
19:23
If you use them correctly they do pretty much match like three
19:26
or 3.5 and there's been a lot of research papers around this.
19:30
But one thing to say here is that also it is difficult to
19:33
judge the How good a large language model is because the we
19:38
only have there's no quantitative like metrics for.
19:41
It's all qualitative, it's all, oh, I'm reading this sentence
19:43
here, it does. This make sense to me.
19:44
Like that's that's that's pretty much all there is to it.
19:48
So I think that like the I think there will be more like small
19:52
language models, but it is interesting to see.
19:54
There's a lot of this new like language model development and I
19:57
really do think that having effective database and the LM
20:00
stack is going to be pretty pretty integral like anybody
20:03
who's kind of building this. Off into the future, is probably
20:05
going to be using some sort of vector story.
20:08
Well, so break down, how that works, what does it actually
20:11
look like, in practice, when you're using a vector store, you
20:14
mentioned how because my understanding is that you can
20:19
actually just know you break down how it works.
20:21
I don't want to tell you what my understanding it, okay?
20:25
So let's say so I'm just going to I'll just cover like the
20:27
architecture of OSS chat which is like one of our like, like
20:32
demo apps that we use that lets you chat with open source 2.
20:34
Documents. So basically what you do is if
20:37
you want to build one of these apps has you go and you grab all
20:40
the documents that you need and then you pipe the documents you
20:43
need and you as the upike them into vectors database in this
20:47
interesting manner. So you break them down and check
20:49
them obviously. And what you do is you actually
20:51
send the documents to chat CBT or cunha or a lawn mower, or
20:56
whatever, Bard, whatever open source, whatever, LM you want.
20:59
And what you do is you tell the LM like, Here's some
21:04
documentation. What are some questions that I
21:06
could ask about it or something like that?
21:07
That might not be the exact prompt, but it'd be something
21:09
like that. So now you have the question
21:10
space and you have the answer space and what you do is you
21:13
store both of these and together and so each entry into their
21:17
Vector database is going to include a question and an
21:19
answer. And now you have a way to query
21:23
for the questions instead of saying.
21:25
Okay? So if I'm a user and I only have
21:26
the docs and I want to say something like, can you show me
21:30
how? I don't know.
21:31
XYZ. How does Torch Vision work.
21:33
And I want to ask pie torch about it, it's going to be very
21:36
difficult for it. So maybe I want to say, how do I
21:38
build a CNN in with with torch Vision?
21:42
We using XYZ modules, it's going to be very hard for the vector
21:48
representation of just the answers to give you a good
21:52
response. So you have to store the
21:54
question space as well as the answer space.
21:57
And so first once these once we once these documents are sent to
22:02
tragic Beauty in that Generates these questions these and bet
22:05
these entries are put into the database that includes the
22:08
embeddings for the questions and the answers.
22:10
And then when the user comes the user says I have this question.
22:15
Show me how I can build a convolutional neural network
22:18
with pie torch. Then what it does is it says
22:20
okay. Well out of these questions
22:22
which one of these questions was generated from which one of
22:25
these documents generated a question that response to this
22:29
and if I was like the closest question and then it says okay
22:33
well this here's this tutorial on how you can generate a
22:36
convolutional neural network and then the outputs all the code
22:38
for you. So the the thing is that you
22:41
have to kind of Store, both the question and the answer space
22:46
and you just use it as like use the vector store as your first
22:50
place. Kill query.
22:51
So if you so one of the things that is that you'll also see is
22:56
if you ask a question about pie torch, let's say we're talking
22:59
so the way OSS chat works is it can predict which Which open
23:05
source Library you're asking about.
23:06
But if you only have pie charts documents and you ask a question
23:08
about, I don't know, hugging face, or whatever.
23:11
It's not going to be able to give you a correct response.
23:14
And it'll default back to chat gbt and we'll see what it says.
23:18
But most of the time that's up to you as like a user in your
23:23
production, use case, whether or not you want to default back to
23:26
the LM or if you just want to say we don't have that
23:28
information in our data. Yeah so one of these use cases
23:33
that you mentioned Or with the Enterprise and another think
23:36
it's becoming more and more popular.
23:37
Is the idea of we have a knowledge base and we have
23:42
everything in a million different places, but we can
23:46
pipe all of that into a vector database.
23:49
And we can have it on hand so that if people want to query
23:53
different questions about how this project is going, or how
23:58
this product was created or who was involved with it, excetera
24:03
Etc. That can happen.
24:05
And the way I understand it, is you pipe, all of that data, from
24:08
all these disparate sources into the vector database and then
24:13
your, your chunking it out, you're sending it to your large
24:17
language model of choice. Asking for the questions that
24:22
can be asked around that you throw that back into the vector
24:25
database. But at some point and if I
24:30
understand correctly, you also are going to get these questions
24:33
that Then asked. So then let's say there's
24:38
there's a question that I want to know was John working on this
24:43
project when Henry was not on the team yet, right?
24:48
And that seems potentially, it doesn't need large language
24:51
models to do especially because they could hallucinate that
24:55
really easily and just tell you whatever.
24:58
So how have you been seeing those kind of situations where I
25:03
have a question, a very specific question.
25:05
I go into the vector database. The question hasn't been asked
25:09
yet and then what do you do from there?
25:13
And then how do you like maintain the validity of those
25:16
answers? Yeah, so the actual, so the
25:20
visor database doesn't actually you don't actually have to store
25:23
questions that have been asked. You actually asked chat CBT or
25:26
whatever, it was, whatever, LM you have to come up with a set
25:29
of questions. And so, I think that in that
25:32
case, perhaps the LM has not come up with a question.
25:35
Was John working on this project before Henry and joined the
25:38
team? But you know, once you have one
25:40
of the things about well I can only say that my love is has
25:44
this. I don't know.
25:44
I don't know. Other databases that well so I
25:46
can't say exactly what they have or what they don't have but Mill
25:49
this does allow you to do this kind of metadata search around.
25:52
Okay, let's say we have a filter and you want to say I only want
25:55
information about this project from these dates then you can
26:01
pass that in and then it would be up to Up to whether or not
26:04
the LM can parse that correctly and that's something that is
26:09
kind of. That is like a more up to the
26:13
users implementation and how the, how the user is able to
26:16
implement that whether or not that is able to be answered
26:20
easily. Or whether or not you're going
26:22
to get some sort of more generic response, but I would say that
26:26
like you would just have to Think about some of the filters
26:31
that you may need to apply on your data and allow the owl and
26:37
to kind of like no, this kind of stuff, it is very unlikely that
26:40
you would see something. So like, for example, with that
26:43
question, exactly. It's very unlikely that you
26:45
would see something like This team, this was done when Henry
26:52
had not yet joined the team, but it's possible that you see
26:55
something maybe in the docks. And maybe this means that you
26:58
don't have to do the filtering that maybe, if you have
27:00
something in the documentation that's team members, who worked
27:03
on this project were John Barbara Sam, I don't know.
27:07
And then the what the doc could, what the language model could
27:11
understand from. That is the query, these
27:12
documents and look for ones where Henry is not mentioned in
27:15
the team name. But yes I mean like that's a
27:17
that's a that's a tough. That's a tough question, right?
27:19
That's like something where if you are building the LM app
27:22
that's kind of up to you to decide how you want to handle
27:26
these things. Where the user may ask a
27:28
question that the llm doesn't know how to parse and that
27:33
hasn't hasn't been asked. Yeah, I get it and the I guess
27:39
the other big question for me. Always.
27:41
It comes back to when how can we set up some kind of process or
27:48
rules or whatever to where we don't need to use the large
27:53
language model unless we absolutely need to because I
27:56
think it's always going to be a better scenario if we don't
28:00
considering that it can go off the rails or if we can get the
28:05
information. Just directly from the database.
28:07
It's going to be faster and cheaper and all of that.
28:10
So like how can you set up? Those potentially just those
28:14
rules. I imagine it's not going to be
28:16
hard-coded, maybe there's actually people.
28:18
There's, there's researchers out there that are creating papers
28:22
on this very topic, but I wonder about that a ton.
28:26
How did set up rules around using Vector data bases and like
28:30
an owl a map as okay. Yeah.
28:32
I mean, it's really like how to make sure that you Don't over
28:36
use the LMS because you don't always need them, right?
28:41
Yeah. Yeah, I mean, I think that well,
28:45
the way that the way I kind of see that working is just then
28:48
you would, you would say, okay, we only want, if we want you to
28:51
query the vector database first and if the answer is not within
28:58
the certain distance. And so okay, this is where we
29:00
get kind of games like factor in beddings, right?
29:01
So back to embeddings, are these numerical representation?
29:06
Of some sort of data and they're usually taken from the second to
29:09
last layer of your model. And one of the things that the
29:15
way that vectors are indexed is based on what the back door and
29:20
bedding actually is and then there's two ways there's
29:22
multiple ways to measure distance but the two ways that
29:25
we do. It, our inner product, which is
29:27
basically cosine, similarity, or L2 Norm, which is the, the L2
29:31
Norm of two vectors, I would say you would basically say here's
29:35
It's query. Are you finding anything within
29:38
a certain distance state of the virtual database?
29:40
Are you finding something within a certain distance of this and
29:43
that distance metric is a parameter for the user to tuned
29:46
for their user to understand, like what makes two queries too
29:51
far apart? Because what you can really do,
29:53
what you normally do with this is you say, give me the top
29:55
three results or give me the top five results or give me the top,
29:58
whatever results and then you look at those results and you
30:01
say these results are Like this result is the right one and
30:05
maybe it's the second result that's returned.
30:07
Maybe it's the first one. Usually, we just say the first.
30:09
We just assume the first result, the closest one is going to be
30:11
the right one. But if you have like certain
30:14
things where you want to say, if the vector doesn't, if the
30:16
vector because of the vector database is never going to not
30:19
find a solution. As long as there is a vector in
30:21
the vector data base. Hey, Lazlo here, if you're
30:24
serious about Emma laughs, you hit subscribe right now.
30:33
There's only one vector, it will return that Vector.
30:35
Every time you will you have to set some like parameter that
30:38
says, if the vector is this far away, then then it doesn't count
30:42
then it's not close enough to our query vector.
30:44
And that is actually up to how well, you know, the data.
30:50
What do you mean by that? How well, you know, the data.
30:52
So if you. So if you want to say there's a
30:56
certain Vector distance that is too far from the vectors.
31:01
The vector query, you can kind of like look at some of the
31:04
distances between your vectors and understand.
31:07
Okay, these two vectors are, let's say 1,000 units apart and
31:12
that is too far to be To be related.
31:15
Because maybe one of them is like, apples are pollinated by
31:22
birch, trees in the Pacific Northwest and the other one is
31:25
like Kobe beef is raised in Japan or maybe the other ones
31:30
are just came up with this off. The top of your head, that was
31:32
incredible. I love the creativity there.
31:34
Yeah, thanks. Yeah, I mean, these are good
31:37
though. So they are kind of say, okay?
31:39
Yeah. That's why I was saying, okay,
31:40
maybe the second one should be something like this.
31:43
Northern California is better than Southern California.
31:46
Yeah. Maybe the other one should be
31:48
something like the derivative of, e of X is always 0 of X.
31:50
OK. There you have two completely
31:52
unrelated topics and you say this is where I draw the line
31:55
but you could also draw the line at the food.
31:56
When you say what About beef, 10 apples.
31:59
These are two unrelated to, to, to say that this is a correct
32:03
result, Legend has know what your data looks like and know
32:07
like how far apart is two unrelated for you as the the
32:10
query. ER, yeah, I see.
32:13
So as The Courier you're the one and correct me if I'm wrong.
32:18
You're the one that are setting up these these rules on what it
32:23
returns if it is not far or if it's too.
32:27
If our then you're going to say don't return that because it
32:30
doesn't make any sense. Right?
32:32
Right well I guess not too might not necessarily be the person
32:35
who's running the queries who sets that up.
32:37
It's probably the person is setting up the app and I'm just
32:40
assuming that these are the two of the same people but that's
32:43
not always the case. In fact, I guess that's usually
32:45
not the case but you know the person is setting up the app.
32:48
Should understand like what? Understand that at a very basic
32:51
level. Yeah.
32:53
Okay. And so then when it is through
32:55
far that's when you Say go to the large language model highly
33:00
over simplifying. This aren't I?
33:02
I mean yes but, yeah, I mean yeah, that's basically what it
33:05
is is if it's too far, you say, go to language model or you say
33:07
return. Like we don't know her, I see.
33:12
Alright. So one thing that I was thinking
33:16
about to as you were mentioning, all of this good stuff is How?
33:25
The vector databases, like they're great for certain use
33:30
cases as we've been mentioning, but they also are maybe not
33:35
great for other use cases. And so, what, when would you
33:39
advise someone not to use a vector database?
33:42
Yeah, I think that if you have something where your primary
33:47
differentiator is something that is can be stored in like key
33:51
value pairs, then you don't need a bunch of database.
33:53
And in fact I think that if You are working with data that isn't
33:58
that doesn't need any semantic similarity.
34:00
You don't need a vector database.
34:01
So for example, let's say you're working with your labeling cans
34:05
and you want to say this can is lime flavored sparkling water,
34:09
okay? So now you have two kinds of key
34:11
value pairs, right? Flavor, lime water, type
34:15
sparkling and then you want to filter based off of am I looking
34:19
for sparkling water? Am I looking for Stillwater?
34:20
Am I looking for lime or grapefruit or apple, or whatever
34:24
you want to say? That's like something where
34:26
there's no need for a vectors database like just go to key
34:29
value store and also it wouldn't really fit of actual database
34:32
that well because widths with text you actually want the text
34:37
to be. Well I guess it depends on the
34:39
the and the model that you use to generate your vector and
34:41
beddings. But most models that we use to
34:43
generate a vector, inviting us are trained on written text in
34:47
the form of like sentences or paragraphs.
34:49
And so if you have something like Flavor: lime, Water: still,
34:58
it won't understand that. As well as, if you said
35:01
something like, this is a can of lime flavored sparkling water.
35:05
So if you have your data in a different form than complete
35:09
sentences, A it's going to be I don't think there's a lot of
35:12
good models for finding the right and beddings and then be
35:15
even if you do find the embedding Soylent about to
35:17
database, you're wasting your time.
35:19
You could have just used a key Value Store.
35:22
Yeah. It's not the right tool for the
35:26
task right right. Right.
35:27
It's not the right tool for that task.
35:28
Yeah and so one other thing that I hear people talking about
35:33
quite a bit is elastic search and elasticsearch feels like it
35:39
could be you. Through this but not quite.
35:42
Have you seen people talking about that and like, how do you
35:46
navigate? Hey, should I be using
35:49
elasticsearch? Do I need to bring on the vector
35:51
database? Can I just get away with what
35:53
I've got? Yeah, I've seen people talking
35:56
about this as well, but to be honest, I don't have a lot of
35:59
experience working with elasticsearch so I'm not
36:02
entirely sure. Yeah, I can't, I can't really
36:04
say. All right, no worries then So
36:08
let's go into this idea of where you have been seeing generative,
36:16
AI apps, what are some common pitfalls that you have been
36:21
seen. So so let's go into this idea of
36:25
what different pitfalls you have been seeing in generative, AI
36:30
apps are Well, given the space where I work.
36:34
Most of the pitfalls I see in generative.
36:36
AI apps are just like data. And like I said, like I'm like,
36:40
most of them are, what I just see is that they don't have the
36:42
right data or what I've seen a lot on stack.
36:45
Overflow recently has been people, like saying things like
36:49
so this one's actually I've seen on stack Overflow and read it.
36:52
If someone was like, Hey I have the CSV data and i-i've actor
36:56
eyes, Dan. I put an adventure database and
36:58
oh actually someone came to Mill this office hours with this
37:00
question, too. There are I'm having I have this
37:03
data and I factorized and I put it into a database and it's not
37:05
returning the right responses to me, what's up with that?
37:08
What's going on? And then, so we took a look, we
37:11
took a look, into their into their into the code with them
37:13
and into the, the data as well. And what we found is that, for
37:17
example, like I was saying earlier your CSV data it's not
37:19
stored in full sentence like format and so you see, you have
37:25
these problems like It's got a bunch of special characters in
37:29
it. /, R /, + /t, / you, these these whatever.
37:33
And then also, there's a bunch of commas that just separate
37:36
different independent clauses. And so, when you have something
37:38
like that, it's difficult to vectorize for machine learning
37:42
models in general, and then it doesn't work well in in a vector
37:47
database, or any any kind of it data base, really.
37:50
So, one thing I see is people kind of like have dirty data, we
37:55
or they don't have the right data to Insert.
37:58
Or one thing that I think is, is interesting that I haven't seen
38:01
a lot of people solving is really like, how do we How do we
38:07
make sure that our data in our Vector database is continuously
38:12
up-to-date as well? And like you can build a
38:14
pipeline for that, you can build batch answer for that, so that
38:17
you can do event based processing for that.
38:18
But there's a bunch of there's, there's a bunch of kind of
38:21
people who are just saying what I see a lot of built-in the lmf
38:25
space right now. Generally, I apps is like people
38:27
who are just like, oh, like here's chat.
38:30
So I saw this one is like, chat UT for your PDF documents where
38:34
they throw a PDF into an OCR and And then, they like to rise
38:37
that. And I've seen this multiple
38:40
times, I don't know what they're doing under underneath the hood.
38:42
I don't know if they're if they're getting the the, the
38:45
question space and having you question the question space, or
38:48
if they're only storing the answer space.
38:50
But, you know, I see a lot of things that are just like,
38:52
here's how you query, your own documents, or for example, like
38:55
Auto GP T, that one was, like super popular.
38:58
And I have not I've met one person who's been like, oh, I
39:03
was actually able to get this to, to do something like, great
39:05
to do something useful for me. And someone said that, they got
39:08
it to build like a very Like react react site.
39:11
That like have a spinner on it and I was like, Oh okay, that's
39:14
cool. This is like the first story of
39:16
anybody saying that they've done something without a gbt that's
39:18
worked for them. I try to get out of GPT to do
39:20
some research for me and I was like, Find Mii news from the
39:24
latest month last month of AI and it went back to the 2022.
39:29
And I was like, this is not really, not really what?
39:32
I need little suspect. Yeah.
39:35
Yeah. Yeah right.
39:36
Oh, but I thought was hilarious. And I don't think anybody else
39:41
laughed but it was a long the lines of yeah.
39:43
So I asked Auto GP T to cook me, a carrot cake.
39:47
And next thing somebody from taskrabbit is knocking on my
39:51
door with the latest DVD from Space Jam because like, what is
39:58
going on, sup? Yeah.
40:01
It says here, I'm there with you.
40:02
This is the rabbit. Yeah, Bugs Bunny is in space gym
40:05
and then you've got Bugs, Bunny likes, carrots, and all It just
40:09
went way off the rails and taskrabbit and all that.
40:12
So, yeah, I think I've heard a lot of people Echo that
40:17
sentiment, but again, we digress.
40:20
I didn't mean to derail you on this idea.
40:22
I think the the greater theme Here is and it just goes back to
40:27
something that we say Time and Time Out on this podcast, is you
40:31
got to be intimate with your data.
40:33
Yeah. You gotta go and no, it, the,
40:38
the really Lee top performers they can know though that data
40:43
so well and then get those insights from it because they're
40:47
just so very familiar with what is going on in the data.
40:55
Yep. Yeah, knowing the data is
40:57
definitely, I mean, I feel like anybody who's worth it ml for
41:01
long enough kind of comes around.
41:03
Everybody is all these. Not like when you first start,
41:05
you like oh these models are like so cool.
41:07
We got to get like the right model and then, as you kind of
41:09
get into your like, oh wait, what was the data that we train
41:12
this model with? And then you're like, oh, we
41:14
didn't include anything about bananas.
41:16
And so the model doesn't know anything about bananas and stuff
41:19
like that. And it's interesting to kind of
41:21
see that in the same way that it's, I mean, it's kind of like
41:24
how people are as well. He kind of see this like an
41:27
Anthem for mantha. From Worf lme's acting like
41:32
people because they don't have like, the they don't have the
41:35
data and you if you don't give them the right, the data in the
41:38
right form, it's just like people, if you don't, if you
41:39
don't give them the right date in the right form, they don't
41:41
understand. And so, it's very interesting to
41:44
kind of like see that. And it's very interesting to see
41:45
that as like, a problem that we're kind of like facing right
41:48
now, 100%. Yeah.
41:50
So dude, talk to me about Mill, this I know that you all have a
41:55
ton of cool stuff going on and there's no this Which is the
41:59
open source project, right? And that is Liz which is the
42:03
beefy. Enterprise version.
42:06
Is it like a managed novice and then there's also Mobis light
42:10
and I know you all just I think somebody was it Frank.
42:14
He presented an MLs Community meet up in San Francisco and it
42:19
was all about GPT cash, so you all are doing a ton of stuff and
42:23
then there's there's a list Cloud to, right?
42:25
Well, yeah. There's there's a ton going on
42:28
there. Break it all down.
42:29
For me and what you're seeing people using each one for ya.
42:34
So I mostly use Mill, this light this.
42:36
So there's Mill this which is the open source Vector database.
42:40
And the great thing about, I think the really really cool
42:42
thing about Mavis for me is that it operates really well at
42:46
scale? And when I say it's scale, I
42:48
mean like if you're working with 500 million vectors kind of like
42:51
scale. Yeah.
42:52
And huge players that are using Mill this to.
42:54
I know that I've seen like the crew uses Novus and there
42:59
There's, it's that kind of scale.
43:01
Yes. Yeah, like Walmart and eBay and
43:03
they're just, they've got huge amounts of data.
43:06
And I think that one of the things that when they, when they
43:09
first showed me, the architecture diagram from Ylvis,
43:11
it looked pretty complicated and it kind of is three different
43:14
kinds of like notes that you spent up as a query, no, data
43:17
node, and index known, they all handle different things.
43:19
And the query, no handles. How you query the data, the data
43:22
handles actually, what's going on with the data and in, and
43:26
then, in S3, or whatever persistent stories you used.
43:29
There's all these like segments, so none of us has these things
43:31
called five segments and they grow up to 512 MB and then once
43:35
they're 500 megabytes, they're sealed off.
43:36
They're not touched any more. They're just sort of persistent
43:38
memory. And one of the, I think, I think
43:41
one of the reasons why I'm Elvis works so well at scale is
43:43
because of these segments, and it's really interesting to kind
43:46
of like see how it works is. Let's say you have one terabyte
43:49
of data, you don't want to re-index all of your data.
43:51
Every time you add something to it, right?
43:54
And one of the nice things and also, if you search one terabyte
43:58
of data, that's Just the amount of calculations are going to
44:01
have to perform is really high. So when you have that splinter,
44:03
512 MB segments, what happens is actually we parallelize each
44:08
search for the segments so that it's actually possible to
44:13
basically have you're basically getting like a Much much, you
44:19
get much faster queries on large amounts of data and it kind of
44:23
as it scales, your performance gets better and better relative
44:26
to having single block chunks of data.
44:28
So I think that's one thing that's really cool about it.
44:30
Well this itself is usually run. If you are running Mill this in
44:36
like an Enterprise, you're probably going to be running it
44:38
on some sort of server some sort of sets of servers so you can
44:40
kind of like horizontally and vertically scale and so if you
44:43
want to run it locally usually people use like Helm or
44:47
Composed, something like that. But the nice thing about Mill,
44:50
this light, and this is what I think is actually really cool.
44:53
Is it kind of like, does this is job of democratizing vector
44:56
databases by making it more accessible.
44:58
And so I wrote a piece recently about Vector databases and how
45:02
we're democratizing Vector databases and like the three
45:05
like pillars that I have for. That was like you provide like
45:07
educational material that actually teaches people about
45:10
what our Vector is, what, our Vector indexes, what is the
45:12
importance of using? The right factors and stuff like
45:14
that. And then you have another pillar
45:16
of Making making an accessible because Vector databases used to
45:20
be. Well, they're pretty complex in
45:22
general and so they used to be things that were pretty much
45:25
available only to large players, like Enterprise people and now
45:29
with things, like, no, the site you can run it directly in your
45:31
notebook. You don't even have to be like a
45:33
software dive. You don't have to understand,
45:35
you don't have to understand, like Docker or like, kubernetes
45:37
or anything like that. You don't even have to know what
45:39
an images. You can just say, pip.
45:41
Install Mill, vez default server dot start, and you've got a
45:45
vector notebook, you've got to factor database, write in your
45:47
Look. And so now, like data,
45:49
scientists data analysts people who could actually be using
45:52
Vector databases now have access to this, right?
45:55
So I think that's cool. And then the last thing is,
45:57
obviously, like the evangelism aspect, we just go and you like,
45:59
tell people like, hey, this is cool.
46:01
Technology, should go check it out, right.
46:03
But so, yeah, I think the, no, the middle I think is really
46:06
cool. I write a lot of jupyter
46:08
notebooks, I use a lot of just I just it's easy to kind of like
46:13
get these mini projects up and running like llama llama and XO
46:16
like Lane chain or anything else like that.
46:18
Can you can run in your vector notebook.
46:19
And then you can your jupyter notebook, and you can throw it
46:22
on coab and it's like, easily accessible to the public.
46:26
So, I work a lot with Nova slight, initially, when I, when
46:29
I first onboard it, I worked with Mill.
46:30
This like the Standalone Mill, this on a server version, which
46:34
is fine. You just you have Docker and
46:36
then you spin it up. But no, the slide is just really
46:39
easy to use and it's just really nice to have And then Zyliss.
46:42
Zyliss, Cloud recently has released this like free feature
46:45
where you now have free, Zyliss Cloud up to half a million.
46:51
I think vectors and it's it's kind of like management of this.
46:55
Like, you don't have to worry about scaling, you don't have to
46:58
worry about posting it on. Whatever poster using ec2, or
47:04
you'll have to worry about spinning up and spinning down
47:06
pods and stuff like that. So that's kind of what Zyliss is
47:08
that's what those cloud is. GPT cash was like a project that
47:13
we built that actually came from OSS chat, which is the one that
47:18
allows you to talk with the open source software GPT.
47:22
Cash was the caching layer for that?
47:24
That was like, oh someone's asking a lot about the same
47:27
questions about pie torch that people.
47:29
Keep asking about torch Vision or people keep asking that torch
47:31
audio. Now, let's just cash some of
47:33
this as well, so we don't have to Ping about that.
47:36
So GPT cash was kind of like that thing that was just like,
47:39
Hey, we're cashing Not a bunch of queries, all your FAQ and you
47:43
can save as much money doing this and here's how you do it.
47:46
So, yeah, that's pretty much like, what, what to listen and
47:49
Zyliss have been up to. I'm really excited to see like,
47:52
a bunch of, like new releases on those Cloud as well.
47:54
People in the middle of his slack, ask me for things like,
47:57
oh, is there a dynamic indexing? Like our Dynamic schema is, can
48:01
we change up things? And like now, like you can you
48:05
can just throw in like a Json as your schema a little except that
48:09
you don't have to go in. Your own like right now, there's
48:12
like a field scheme of a collection schema and then like
48:15
collection and you have to put the field schemers into the
48:16
question Steamers and so the questions and it.
48:18
So, actually, you don't have to do that anymore and then there's
48:21
like, role-based access control, which people have been painting
48:24
out on Mill. This, at least like three times
48:26
or four times in the last couple weeks.
48:28
Everyone loves that. Especially once you start to get
48:30
into actual Enterprises cases, that's like a go no-go type
48:35
thing. Yeah, we basically we had
48:37
someone who like literally came to us and was like, we've
48:39
decided to go. With nervous because you guys
48:41
are the first one to implement our Zyliss because you guys are
48:43
the first one to implement role-based Access Control
48:45
station and I was like, oh, very interesting.
48:48
This is like not something that you know as like a as like
48:51
someone who doesn't necessarily have to work with Enterprise
48:53
data all the time. This is like an interesting like
48:55
point. It's got that sock to compliance
48:59
also I don't know, I think so, I don't know.
49:02
I just know this is like just something that people came up to
49:05
me and they were talking to me about and in the slack and I was
49:07
like, huh. Okay.
49:08
That's cool. Yeah, that's that's interesting.
49:10
Something that I would have directly related a thought
49:12
about. Yeah, it's huge for some
49:16
companies. I mean me and it's a showstopper
49:20
if you don't have that HS. Yeah yeah I remember when I was
49:24
at Amazon, like the whole I am policy was, was just there was
49:27
just a lot you had to do a lot with the whole I am thing.
49:30
So, yeah. Yeah.
49:31
That data access, man. That's a whole nother beast in
49:36
this and I've heard horror stories to about how How
49:41
somebody got a job. And then, six months later, they
49:44
still didn't have access to the data that they were supposed to
49:48
be working on, and yeah. Yeah, that's that's I mean,
49:54
that's just yeah, it's terrible practice.
49:55
It's also really relatable. Like, I've been at, I've worked
49:58
at places where it's just, we want you to build this thing
50:02
that requires you to train it on the data.
50:04
And then I was like, okay, where's the data?
50:07
Well, software does don't have access to the production data
50:10
data and I was like, well, what do you can?
50:12
I can I get access what you want me to do?
50:14
I'm not really sure. Yeah, I see that.
50:16
That's like a thing, like mid-sized companies, like big
50:19
companies. It's very kind of weird.
50:21
That's morning. That's more like a cultural like
50:24
work culture, like company culture, kind of thing than
50:26
anything else. That I'm not really sure like
50:28
how we could go about fixing these kinds of things.
50:31
Yeah, there's definitely people try and I mean, I think that's
50:33
what the whole data mesh crew is yelling about.
50:39
And how do you gotta own your data data products, and treat
50:43
your data. Like, it's a product and you
50:45
service the producers, and the consumers, all that.
50:50
So the producer service the Rumors or whatever.
50:53
That's my limited understanding of it, just from talking to the
50:56
different evangelist in the data mesh area.
50:58
But I do know that there's there's some pain around that
51:02
and the bigger the company is the the worse.
51:04
It is. So dude, just a quick side, note
51:07
on GPT cash. What is this something that you
51:10
can just slap on or do I need to be running Novus with it?
51:15
I think you slap it on. I mean it uses Mill.
51:17
Vez. Oh, give me a background.
51:19
Use dilys. Okay, but yeah.
51:21
You Needs. If you're going to run a
51:23
standalone you and you'd like a standalone server to pain.
51:26
Okay, yeah yeah yeah yeah. So this is all super, super cool
51:31
stuff man and I'm wondering like anything else you want to
51:35
mention before we jump what's new in your world that we didn't
51:40
get to talk about. I just want to say like we're
51:44
seeing a lot where the new in another like AI Renaissance for
51:49
lack of better words and I'm very excited to To, to be part
51:53
of it. I'm very excited to see more
51:55
people get access to this kind of stuff.
51:56
I'm hoping that we are able to provide like the educational
52:01
material that people need to get started with this.
52:03
And yeah, that's pretty much it. I'm pretty pretty happy about
52:06
this. Unlike a totally unrelated note,
52:08
very happy to have the MLF Seattle Community getting
52:11
started as well. And oh, yeah, we gotta give a
52:14
shout-out to the Seattle Community.
52:16
You all did meet up week ago, I think.
52:18
And so and yeah, it's growing and it's Get up steam, I would
52:23
say. So I am very happy to see that
52:25
happening to and you're leading the charge on that one.
52:27
Yeah, yeah, yeah. I'm very interest.
52:30
I love the community stuff. I love getting my hands into the
52:33
technical stuff and building stuff, but I also loves to sing
52:35
people come together and talk about things they're interested
52:37
in, especially when it's a. I and I've been, and I've been a
52:40
nerd about this for a long time now.
52:41
So I'm very happy to see other people talking about it.
52:44
Yes, awesome. So also you mentioned that
52:47
you're doing a office hours, right?
52:49
Yeah. In the middle of this community,
52:51
when is that what? What's the deal there?
52:53
Oh yeah. So we have these office hours
52:55
with me and Phillip there. Every Tuesday at 1:30 p.m.
52:59
Pacific time. There are primarily the
53:02
questions are primarily being answered by Philip to be honest.
53:04
I'm just there to kind of like, facilitate an answer like some
53:06
basic stuff, but a lot of times people will ask questions that
53:09
are much more like how do you use Mill vez at an Enterprise
53:12
scale and then Phillip has a lot more experience than I do with
53:15
that. So he'll answer though, this is
53:16
probably blah blah. This is probably all blah and
53:19
you'll see kind of like we see a few people come every week.
53:22
Really mmm. Therefore usually nice usually
53:26
about three or four people. I don't think I've seen more
53:28
than like seven awesome. Yeah, they get some quality time
53:33
with you all the answer, all their questions, their hearts
53:36
desire will leave a link to those office hours in the show
53:40
notes in case anyone wants to drop by and chat with you.
53:43
And I think that's it for today, man.
53:46
Thanks for coming on here and talking with me.
53:48
Okay, awesome. Sounds good to me tres.
53:50
It's great to chop. This is Skyler.
53:55
I lead machine learning at Health rhythms, if you want to
53:58
stay on top of everything happening in ml.
54:01
Ops, subscribe to this podcast now.
54:05
Now, now, now now now Now.