Latest / Agentic Conversations (formally mlops.community) / Data Governance and AI // Alexandra Diem // #212

Transcript

Data Governance and AI // Alexandra Diem // #212 · Agentic Conversations (formally mlops.community)

0:00
Hold U Before we get into this next episode, I want to tell you
0:03
about our virtual conference that's coming U on February 15th
0:08
and February 22nd. We did it two Thursdays in a row
0:13
this year because we wanted to make sure that the maximum
0:17
amount of people could come for each day since the lineup is
0:22
just looking absolutely incredible.
0:25
As you know, we do. Let me name a few of the guests
0:29
that we've got coming because it is worth talking about.
0:33
If you got Jason Louie, we've got Shreya Shankar, we've got
0:36
Drove, who is product Applied AI at Uber.
0:40
We've got Cameron Rook Wolf, who's got an incredible podcast
0:43
and he's Director of AI at Reba Engine.
0:46
We've got Lauren Lockridge, who is working at Google also doing
0:52
some product stuff. Oh, why is there so many product
0:54
people here? Funny you should ask that
0:57
because we've got a whole AI product owners track along with
1:01
an engineering track. And then as we like to, we've
1:05
got some hands on workshops too. Let me just tell you some of
1:08
these other names just for a moment, you know, because we've
1:12
got them coming and it is really cool.
1:14
I haven't named any of the keynotes yet either, by the way.
1:18
Go and check them out on your own if you want.
1:20
Just go to home dot ML OPS dot community and you'll see what
1:24
we've got. Tungi, who's the lead researcher
1:27
on the deep speed project at Microsoft.
1:29
We've got Holden, who is the open source engineer at Netflix.
1:34
We've got Chai, who's leading the AI platform at Uber.
1:39
You may have heard of it. It's called Michelangelo.
1:41
Ohh my gosh. We've got Phazon, whose product
1:44
manager at LinkedIn, Jerry Lewis, who created good old Lama
1:49
Index. He's coming.
1:51
We've got Matt Sharp, friend of the pod, Shreya Rajpal, the
1:55
creator and CEO of Guardrails. Ohh my gosh, the list goes on.
1:59
They're 70 plus people that will be with us at this conference,
2:07
so I hope to see you there. And now, let's get into this
2:10
podcast. Hi, my name is Alexandra Dean.
2:13
I am the head of Cloud Analytics and Machine Learning Operations
2:18
at Encelia Insurance company in Norway and I like to drink my
2:24
coffee black. What's happening gang?
2:27
Welcome back to another ML OPS community podcast.
2:30
I am your host Dimitrios and I just got off this call with
2:35
Alessandra and what a call it was.
2:39
She is doing some really fascinating stuff when it comes
2:42
to enabling the teams at her company.
2:45
Her team basically just is like the SWAT team for every other
2:49
data team at the company. They go in there and as she put
2:53
it, they are trying to make themselves redundant.
2:57
So their whole idea is how fast can we get in and get out.
3:02
It's like the extraction team almost.
3:04
I don't know if SWAT team is the best metaphor, but when you
3:07
listen to it, you'll know what I'm talking about.
3:09
O They go in, they figure out the MVP, whatever that needs to
3:14
get done to make sure that happens and the data product
3:18
gets shipped. They do it and then they get
3:21
out. They leave it in the hands of
3:24
the team that asked for whatever it was to get built.
3:29
And they also are making sure that all of these different
3:32
teams understand cross functionally what other teams
3:37
are doing. O oh, we saw that the accounting
3:40
team needs something like this and it just so happens that our
3:44
sales development team did something along these lines.
3:47
We could probably generalise what the learnings are and the
3:52
problem is here in order to say that we can solve this problem
3:58
the same way. So if that doesn't catch your
4:01
attention, the other piece that I think might be of interest to
4:05
you is how she talked about bringing software engineering
4:11
best practises to the data teams and the data science teams.
4:16
That is something that we talk about a lot.
4:19
You've probably heard it being said on LinkedIn or in the Milos
4:22
community and how useful it is. She mentions this.
4:26
She talks about how much her game was up levelled because of
4:33
learning the software engineering, best practises of
4:36
clean code and justice, getting ready for production, and how
4:40
she makes sure that the teams that she works with when they go
4:43
in there as that SWAT team, if we're going to stay with that
4:46
metaphor, they're able to have code that's ready for production
4:51
and they're able to not get stuck in that exploratory data
4:56
loop that some of us are familiar with.
4:59
So hope you enjoy this episode of the ML OPS Community Podcast
5:07
and right now I've got a new little segment for us which I
5:12
like to call. Let's read what some of our
5:15
users think about the MLPS community.
5:18
This is a segment that I am going to read off some of the
5:23
comments that you all leave in the ML OPS community, YouTube or
5:28
Spotify or all that. The only catch is some of this
5:32
shit is not nice. So let's go through it.
5:35
It's almost like reading mean tweets.
5:37
Here we go. O we've got dev, Yuns commented.
5:44
What a waste of time. Please get more qualified
5:47
interviewers and interviewees Thank you Deb Yens.
5:51
Good thing I go to therapy and and that stuff does not hurt me
5:56
anymore. Next up, Johnny, Ronnie
5:59
commented. You are a hack.
6:03
Learn how to play the guitar. I can't believe that I spent
6:08
time watching this, noted. Last but not least, we've got Mr
6:16
Scooby Dooby saying this was an utter waghmare of a talk.
6:24
Two thumbs up to that. So if you all are interested in
6:28
letting me know what you think of these talks and podcasts, you
6:33
know where to find me. Leave a review.
6:35
I will make sure to read it off in the next episode.
6:41
We'll see you on the other side. Where to start, Where to start,
6:50
Where to start? Where are you in the world?
6:51
By the. Way I'm in Oslo in Norway.
6:54
OK, cool. Are you originally from there?
6:57
No, I'm German and I've but I've lived in.
7:00
I've lived a little bit around the world and have now settled
7:06
on Norways. I've been here for six years
7:07
now. Yeah, I've heard beautiful
7:09
things about the fjords. Is that how they're pronounced?
7:14
Yeah, that's that's correct. Yeah.
7:16
Yeah, they're super nice. So we don't have that those kind
7:19
of views in Oslo. We have to travel a bit to get
7:22
those, but it's always worth. But yeah, I've kind of.
7:24
Since I've moved here, I kind of lost interest in travelling
7:27
abroad that much. Yeah, so I'm actually in Germany
7:32
right now. Yeah, funny enough, I am where
7:36
you hail from, and I'm right outside of Frankfurt.
7:41
Say OK, close. Are you from this region?
7:45
Where you where you from in? Germany.
7:46
Quite Sam from them back. OK, OK, cool.
7:49
So. Not too S, yeah.
7:51
Southeast. Yeah, totally, very cool.
7:55
So I I'm in the middle of the countryside.
7:57
Yeah, I like to make jokes at the Germans expense because my
8:02
neighbour, I wave at them every single day and I am hoping like
8:09
I'm hanging on. I've been here for four years.
8:12
I'm hanging on to this little sliver of hope that one day they
8:17
will wave back to me. Ohh, I hope that happens.
8:22
I've got some people, I should say that they noticed me right
8:27
when I moved into the town, in the town of like 100 people,
8:30
right? And so there are some neighbours
8:33
that are full on, like they wave to me now first and they're on
8:38
board, and then others they're holding out and they do not wave
8:43
at all. They just stare at me, they hug
8:45
me and it's like, why is this guy waving at me?
8:48
I think the best way, the best way to engage with Germans who
8:54
you don't know from before is by finding something to mutually
8:57
complain about. Together, that's it.
9:01
I gotta figure out what we can complain about and then I'll be
9:04
on their level and they'll take me in and invite me over to
9:07
dinner. Exactly.
9:09
Yeah, man. That's great.
9:11
Excellent. Well, I think we should probably
9:14
start with a bit about your background.
9:16
I know you have a lot of experience in applied
9:20
mathematics and statistics, but I want to hear from you like
9:24
what's your story? How did you get into tech?
9:27
Yeah, yeah. So as you said, I started out
9:31
with applied mathematics. So actually I started out with
9:33
bioinformatics, which basically is statistics for medicine.
9:39
And then I started. And then I decided to do a PhD
9:42
in applied math, also applied to medicine in Southampton,
9:47
England, because I was convinced that I was gonna be an academic,
9:50
I was gonna be a professor, and so I did the whole.
9:54
Well, how did that dream go away?
9:57
Yeah, yeah, it's. It went away because because of
10:03
the temporary nature of per stock positions.
10:07
So I had one per stock position right after my PhD in England,
10:12
and then I Brexit because I didn't really feel that the
10:17
political situation was moving in a direction that I I thought
10:22
was so productive. Wait, so you you left Britain
10:27
and you Brexit? Yeah, it did.
10:30
It did leave because of Brexit, Yeah.
10:32
And then I moved to Norway and took another, took a pro stock
10:37
here and that I was lucky enough to get because of, yeah, the
10:42
network that I had with other researchers.
10:45
And then that was a temporary position for two years.
10:49
And then at the end of that position, it took a long time
10:51
for them to tell me whether I get not that she is or not.
10:54
And it just never. Even though I got another
10:57
position, it never stopped feeling like there was a sword
11:01
dangling over my head about the temporary nature of the
11:06
position. And I I knew that if I wanted to
11:09
be a professor, I would have to move country at least once more.
11:13
But I really liked started. I really started liking Norway.
11:17
So then I became a consultant, and that was basically my
11:21
entrance into into tech. And because I was the first data
11:26
scientist at that consultancy that they employed, I had to
11:31
work on software development projects first to be profitable.
11:37
And then I realised, ohh, well there's there's a lot of methods
11:41
in software development actually that we should we should apply
11:45
in data science. And I started doing that with my
11:48
then data science clients and then there was A and then there
11:52
was this position at NCD at the insurance company that I'm at
11:57
now before head of cloud Analytics and Envelops.
12:02
So leading a team that would exactly, precisely do that.
12:04
So applying software development methods into the data space and
12:11
yeah, needing that sort of that change within data.
12:15
So presumably this first consultancy job that you got,
12:19
they really needed a data engineer or a software engineer,
12:23
and they hired a data scientist. And then they threw all of this
12:26
software engineering at you and you were like, well, I guess I
12:29
could do this too. Yeah, I gotta give them a little
12:33
bit more credit. So they were really working on
12:36
on getting proper data science contracts with me.
12:41
But just because they didn't have any experience in that yet,
12:45
it took a little while. And so they asked me if I could
12:49
do if I could work on software development in the meantime,
12:53
with the promise that they would still actively sell me to data
12:57
science projects and be able to pull me from software
13:00
engineering whenever we yeah, got the we got appointed.
13:05
And then that skill set in turn made you stronger as a data
13:09
scientist. Exactly.
13:11
Because at the beginning I was thinking, ohh, OK, I guess I'll
13:14
do that because then at least I have something meaningful to do,
13:18
but then it turned out to be a great asset.
13:22
And and what made you like Norway so much?
13:24
That's just me who's never been to Norway.
13:27
I'm on videos and seeing all the incredible nature that they
13:31
have. Is that what it was or what?
13:33
Yeah, that was a big, big factor.
13:35
So I really needed to have snow, proper snow, back in my life.
13:39
And so I've been. I've been downhill skiing since
13:42
I was four years old and so my parents would always take us to
13:46
the to the Alps once a year as kids.
13:50
And while I was living in England and also the other
13:53
countries, there was no one, uh basically that I could go skiing
13:56
with. Um.
13:58
It would have been a very, yeah, um, complicated trip to do so.
14:02
And so then there was a decision between basically Switzerland
14:06
and Norway. Yeah, Swiss Switzerland isn't
14:10
bad too. Those are two nice choices.
14:11
I know, yeah. Yeah, great problems to be
14:14
having. Where do you want to exactly.
14:18
That's very cool. So there is something else
14:20
before we move on that I wanted to touch on, and it's your
14:25
contributions to Alzheimer's and debunking a few of the common
14:32
approaches. Can you break that down before
14:34
we get into all this consulting and ML OPS and the software
14:38
engineering and data and all that?
14:39
But I just want to know about Alzheimer's, something near and
14:42
dear to my heart because my grandma actually passed from
14:45
Alzheimer's and I think a lot of people probably know somebody
14:48
that has Alzheimer Alzheimer's has gone through that.
14:52
What did you do and what would you contribute?
14:55
As I was one theory that I debunked.
14:57
Um. But yeah, so basically
15:00
Alzheimer's is a very unnatural uses of of ageing and it's the
15:08
result of a failure to clear waste products from your brain.
15:13
So there's this particular protein called amyloid beta,
15:16
which each of us produce all the time.
15:19
You produce it right now. I produce it right now, but
15:21
because our waste clearance mechanism work, we're fine.
15:26
And once those, if those mechanisms start failing, then
15:29
that's when that protein starts accumulating.
15:31
And when that protein starts accumulating into larger chunks,
15:35
then it starts degrading both your neurons and your synapses,
15:40
which is what then eventually causes this cognitive decline
15:44
that we see in patients. So basically what I was starting
15:48
to work on was a theory of how how this protein is normally
15:55
cleared from the brain. Because the problem is once you
15:57
see symptoms, um, it's most likely way beyond the point
16:03
where you could do anything about it.
16:05
So really, prevention is what's in my group and we were aiming
16:10
for. And so there was this theory
16:12
that said that the way all the arteries pulsate in the brain
16:18
would be driving this clearance mechanism.
16:21
And so a lot of energy and also mice was spent on trying to show
16:27
how this mechanism would work. But then I used equations and
16:32
computer simulations to show that basically this theory
16:36
doesn't hold, that just the the pulsations themselves they are
16:41
not, they're not strong enough to drive this mechanism.
16:45
And that there is a most likely another contributing factor from
16:50
a tiny muscle cells within the artery walls that have to whose
16:54
frequency of contraction instead could could be the driving
17:00
factor. But the biggest, because the the
17:02
problem in science always is almost the is that we can't
17:05
really prove things a lot of the time, but you can disprove
17:09
things by basically just showing a counter example.
17:14
And so then that means you can you basically try to always you
17:18
come up with a theory and then you try to destroy it.
17:21
You try to disprove it. And for as long as you can't
17:24
disprove it, you can kind of assume, OK, maybe it's true, but
17:29
then once you have been able to disprove it, then you can adjust
17:34
your theory and work towards something that might be closer
17:38
closer to the truth. So you did all that and they
17:41
still wouldn't give you a job as a professor?
17:45
Yeah, it takes a bit more research, takes a bit, takes a
17:48
bit more research contributions than that to become a professor.
17:51
So then after Alzheimer's I've basically moved to studying the
17:55
hearts with the same methods Ohh well and trying to figure out
17:58
how basically how profusions of the blood supply to the heart is
18:05
negatively affected by heart attacks and then later on heart
18:10
failure as a as a long term consequence of heart attacks.
18:15
And you were using machine learning in these different
18:18
research examples. Yeah, exactly.
18:21
And so basically there's one of the biggest problems in these
18:25
with these models is that you need to you need to find, yeah,
18:31
you need, you have a lot of parameters in these models for
18:34
which you basically can't take any measurements directly.
18:37
And So what you would do is large parameter sweeps using a
18:42
supercomputer, and then you use Bayesian methods for uncertainty
18:47
quantification to try and convince doctors and other
18:53
collaborators that would do models are predicting, or what
18:57
you're saying is actually, um, useful.
19:02
This is neat stuff. O that was the almost like the
19:07
before and that's what you brought into the consulting
19:10
game. And then you realised, well now
19:13
I need to start doing more software stuff.
19:15
What were some what are some things that you now recognise
19:20
when it comes to software engineering that are gigantic
19:24
level ups for you? And if you do not know them, as
19:27
a data scientist, you're basically digging your own
19:31
grave. I'm and maybe I'll start.
19:33
If you don't know git, then what are you doing?
19:37
Other stuff though that you feel like you've learned?
19:39
I know there are some quite vocal people in the MO community
19:43
Slack about like you need to think about clean code, you need
19:48
to think about production code, You can't be just going and
19:51
exploring Jupiter notebooks. Granted this a lot of this has
19:56
changed since the advent of Elms, but I would love to hear
20:00
how you think about it right now and what things you're teaching
20:04
the rest of your team, because I know that is your advocating for
20:08
your helping others understand the value of these software
20:12
engineering practises. Yeah, exactly.
20:14
So there's two that you just mentioned.
20:16
Uh, Clean code? And then also what was the
20:22
second one? Now yeah, product code.
20:24
Ohh yeah, production. Yeah, bringing code into
20:27
production, those hit the nail quite well on the end.
20:32
So basically what we often see with data scientists is that
20:35
they're really, really good at writing Python scripts, but
20:39
they're not good. And they've never been taught
20:41
usually how to collaborate with code.
20:45
So what often happens, and also, but also how to basically write
20:48
code so that they can collaborate with themselves in a
20:52
way later on. So when they're when they're
20:54
supposed to look at something that they've written six months
20:55
ago? Yeah.
20:57
Yeah. And so these, these are the
20:59
things that no one has taught them, but that would that I that
21:04
I find really valuable to teach them.
21:06
Because that means that now once a person goes on holiday or
21:11
leaving a company, then you don't have this catastrophical
21:15
effect of a model not being maintainable anymore.
21:19
And actually what's been a real, real asset from my time as a
21:24
researcher into this, into also teaching and getting production,
21:30
getting code into production, is this emphasis on hypothesis
21:35
testing. So you basically come up with a
21:37
hypothesis and then you write some code and you write the
21:40
minimum amount of code to basically test your hypothesis
21:44
and your assumption. You put it into production, you
21:46
test it, and then you iterate over that.
21:51
It's so funny that you mentioned that because I literally last
21:53
week was talking to somebody about this a a partner, and I
21:58
was mentioning how another person in the community very
22:02
much was an advocate for that. And it's been something that has
22:05
been floating around off and on in the community for I think
22:09
like a year, year and a half, two years maybe by now, which is
22:13
exactly that. Like get something to production
22:16
as fast as possible. So you can know where the
22:18
bottlenecks are, You can know where things are breaking down,
22:21
you can know which data streams aren't actually working or which
22:24
data is bad quality, all of that.
22:27
You're not going to have an easy time figuring that out until you
22:31
get to production. Not that it's easy when it gets
22:33
to production, but at least you will have a better idea of what
22:37
is working and what is it. Exactly.
22:39
And then also you'll be able to figure out whether you're
22:41
actually building the right thing.
22:43
There's a there's a difference in there's a difference between
22:46
building the thing right and building the right thing.
22:48
And I think we are often focusing too much on building
22:51
the thing right and too little on building the right thing.
22:56
Yeah, and that is. That goes back to one of the
23:00
first meetups have you ever had in 2020, where I remember we
23:04
talked to Dan Sullivan and this quote stayed with me for all
23:08
these years. So whatever.
23:09
Four years. Now he mentioned how nobody on
23:13
the business side cares about your F1 score.
23:17
Nobody cares if your accuracy is 98% versus 97 if it totally
23:24
misses the mark. Exactly.
23:27
You and that is exactly what you're saying.
23:30
It's like you have to build the right thing, not so this thing,
23:33
right? Ohh, it's good to do both
23:35
ideally, but we need to really focus on building the right
23:38
thing, and So what are some ways that you recognise if you're
23:43
building the right thing as opposed to the thing right?
23:46
Yeah, so that's the biggest one, especially this hypothesis
23:51
testing approach. I also actually fell into this
23:53
pitfall when I had my first data science science project as a
23:56
consultant. And then basically the problem
24:00
was that there wasn't there. We my company recently gave it
24:03
away for free as a tests, um, to a client that we have a lot of
24:09
that we had a lot of other projects with.
24:12
And so then a group that didn't know what they wanted their data
24:16
scientist for was thrown into the was basically thrown at my
24:20
feet and I was supposed to try and figure out how what we can
24:25
use and what we can do for them. And so in the beginning it's
24:29
very easy then to basically get stuck into data exploration
24:33
forever and forever and you could do data exploration, yeah,
24:36
for a long, long time. But once we started switching
24:39
over to hypothesis based testing and just talking to the customer
24:46
more and involving them very closely, then we actually
24:51
started being able to generate some value for them.
24:54
For the Long story short basically is, is that we always
24:58
involve stakeholders from the very beginning and we spend a
25:02
lot, we spend a lot of time at the beginning also trying to
25:05
come up with hypothesis with several hypothesis that we can
25:09
test that the the owner for the model and also the business
25:15
value would think, yeah would see as valuable.
25:19
Yeah. And that was my question.
25:20
Is the customer you say the customer?
25:23
Is it just another person in the business or is it an end user
25:27
from the product itself? Yeah.
25:29
So for us it's often the the owner of a, yeah, let's let's
25:36
say for example an end solution within our company.
25:39
So we have, yeah, if if I use as an example, a model factory for
25:46
generating sales leads for our customer's customer service
25:50
advisors, then the the owner of that would be the sales, the
25:54
sales centre. So those people managing the
25:56
sales centres because they are the ones that rely on these
26:01
leads that our models that the models generate.
26:03
And it's also a little bit more complicated with us because my
26:07
team for example doesn't earn any any of the models or any of
26:12
the solutions that we help develop.
26:14
We're we're basically a pure enabling team.
26:18
So we all, we enter all of our models are earned by analytics,
26:23
analytics teams that sit in the business units.
26:26
So for example, either in private insurance or in
26:29
commercial insurance, so they would earn the models and we
26:33
basically enter temporary collaborations with these teams.
26:37
So usually two of my team will work together with two of the
26:41
analytics team and then together they will work on a, yeah,
26:47
usually a minimum viable product, first of a solution
26:50
together and then depending on how much time there is left,
26:53
they will improve on it. And then we during this
26:58
collaboration, my guys will teach the analytics team how to
27:03
write clean code, how to collaborate with code and so on
27:07
and so forth. So that then the analytics team
27:10
is able to earn and maintain their own solutions.
27:13
And so we then basically pull out and work with the next team.
27:19
So the value add there is you making sure that something
27:23
there's some tangible deliverable that gets done.
27:26
Exactly. You're the team that goes in and
27:28
says like, there's no messing around here.
27:31
We got to get something out the door so that it's working, it's
27:34
shipped, and then we're iterating on it as fast as.
27:37
Possible. Yeah, exactly.
27:40
So we basically our our biggest goal is to make ourselves
27:46
redundant every single time we work with an analyst.
27:50
And do you feel like the last couple years have made your job
27:56
significantly easier because of the tools that have come onto
28:00
the market? Or is it a little bit like you
28:04
don't pay attention to all that noise and you know what you're
28:07
doing, You know you've got a great workflow that hits, then
28:11
it hits the mark and so let's just go with that.
28:15
And yeah, so that's a, it's difficult to exactly answer the
28:19
question for me because I've only been in exactly that job
28:21
for a year and before that I was a consultant.
28:24
But it's for sure and the the tools that we have have
28:30
available in the cloud nowadays make make this process a lot, a
28:35
lot easier. So for example we've we've built
28:39
a cloud data platform based on data bricks and so that means we
28:44
can in one platform we can, yeah, our analysts can basically
28:49
do everything from exploration and experiments to production
28:54
ready machine learning operations and also
28:57
orchestrating, orchestrating the other code and what they might
29:02
need and then also their visualisation reports and what,
29:06
yeah, whatever else they might need.
29:08
Yeah, I think about that quite a bit when it comes to the tooling
29:11
that's come out on the market because of speaking to a few
29:16
people at these bigger companies that we all think, oh wow,
29:20
they've got the most advanced ML and AI on the market.
29:24
And you realise that because they've been doing ML since 2014
29:30
or 2016, they're actually a little bit legacy.
29:34
Unless they're investing a lot of time and energy to continue
29:39
and stay up with the greatest and latest trends and tools that
29:44
are coming out that make your life a lot easier.
29:46
Because there has been a lot of learnings since 2016 on how
29:50
things can be way easier. You can just think about Spark
29:56
and how many different competitors there are to Spark
29:58
and how difficult Spark is to learn.
30:01
And there's a lot of people that are trying to streamline that
30:04
and get you 80% of what you need from Spark.
30:09
For those use cases that don't need all these bells and
30:13
whistles that Spark can give you and it's 10 times easier, right.
30:18
And so in that case, it's almost like I I think a lot about that
30:22
today. How much different it is if you
30:25
are starting today versus starting in 2016?
30:29
And how, if you're one of these gigantic, fancy, big name tech
30:36
startups that we've all heard of in Silicon Valley, we think that
30:40
wow, they're so amazing, but really they're probably sitting
30:44
there and they're like ohh, we might be hanging on by a
30:48
shoestring, you know, unless you're adding a whole lot of
30:53
resources, which let's be fair, like a lot of them are because
30:57
they see the business value in it, they've seen the business
31:00
value in it since 2016. So they recognise that they
31:03
should be putting it a lot of time and energy and resources in
31:06
it. But that's something that for
31:09
me, you know, I remember in 2019 when I was going around and
31:13
going to conferences and trying to talk to people and they would
31:17
tell me how they were doing experiment tracking with excel
31:21
sheets sheet and that was how they would track their
31:25
experiments. And it's like wow, now there's a
31:28
million tools out there that you can track your experiments with,
31:31
right? And even in 2019 there was,
31:34
there was ML flow rates and biases in that, but it wasn't a
31:37
standard practise. And so kind of where I'm
31:40
bringing it back around to what you're doing and teaching these
31:44
analysts on the different benefits of having clean code
31:49
and production ready and then being able to bring that to to
31:53
production as quick as possible, it does seem like there are
31:57
tools out there that can greatly benefit and streamline that
32:01
process. Exactly.
32:02
And actually, you're also touching on another aspect that
32:07
my team is responsible for, which is basically finding out
32:11
in these, in all of these tools that they are finding out which
32:16
one should we use. Because that's the other
32:19
challenge, right? When there's so many tools to
32:22
choose from, then I think that compels people, paradoxically
32:27
even more so, to fall back onto something that feels very
32:31
familiar, like Excel and and so that's where you that's where I
32:36
think you also need basically the sprawl of an enabling team
32:40
that's looks at the the available text stack and then
32:45
comes up with a recommended one. Real quick, some words from our
32:51
sponsors Ellis and then we'll be right back into the programme.
32:54
Are you struggling with building your retrieval augmented
32:57
generation AKA RAG applications? You are not alone.
33:03
We all know tracking unstructured data conversion and
33:07
retrieval is no small feat. But there's a solution available
33:11
now. Introducing zelizer Cloud
33:14
Pipelines from the team behind the open source vector database
33:18
mill this This robust tool transforms unstructured data
33:22
into searchable vectors in an easy manner.
33:25
It's designed for performance. Zellis Cloud Platforms Pipelines
33:32
Zilis Cloud Pipelines also offer scalability and reliability.
33:37
They simplify your workflow, eliminating the need for complex
33:41
customizations or infrastructure changes.
33:44
Elevate your rag with high quality vector pipelines.
33:47
Try Zelis Cloud Pipelines for free at soliz.com.
33:53
You can find the link in the description.
33:54
Let's get back into this podcast.
33:56
For your company so that basically you can then build
34:00
also a network of super users who will be able to help their
34:04
peers. Umm, so you can also scale the
34:07
approach. It doesn't mean that everyone
34:08
has to use the same tools, but at least there is a there's a
34:12
recommended set of tools if you don't know where to start, if
34:16
you're not sure what to do, or if you don't care what you use,
34:19
then you know at least there is a yeah.
34:23
There is a resource available for you to help you get started
34:26
and also probably many examples of things that other people have
34:32
done in your company that are similar to your problems, which
34:35
is another thing that we work with.
34:38
To make sure to get that out into the open and as opposed to
34:42
keeping it in a silo. Yeah.
34:44
So basically it's making basically it's making sure that
34:47
we don't invent the wheel and you every single time because so
34:53
what's what's really what I think my company is doing quite
34:57
really right is to earn the data and the analytics models in the
35:03
domain, in the business domains. And so that means we have about
35:06
20 different analyst teams in the different business, the
35:11
divisions. And so because the day is only
35:14
so and so long, right, they don't necessarily know what.
35:18
So people who people who work in pricing don't necessarily know
35:21
what people in developing their private insurance products work
35:26
on and vice versa. But because we go and work with
35:31
the different analyst teams and then go to the next one into the
35:35
next one, we can see where there's synergies between
35:39
different problems that teams are facing and then we can tell
35:43
them all. But this other team in
35:45
commercial, they've actually had this exact same problem and
35:49
together with them we solved it in this way.
35:51
So we believe that that would also be a suitable solution for
35:54
you guys. And how have you seen the
36:00
analysts take to becoming basically data scientists,
36:07
machine learning engineers types in a way?
36:11
And so the ones that we've worked with so far and we're
36:14
lucky enough still that we can pick and choose to work with the
36:18
ones who are really keen. So they they really enjoyed it
36:21
and they actually we had an internal conference in September
36:26
and I sent basically one of the teams that's implemented a model
36:32
factory for these sales leads that was talking about earlier
36:35
And they basically opened their presentation by saying yeah, we
36:38
feel like we're more software engineers now than we actually
36:42
are data scientists, but they seem to enjoy that that's you
36:47
kind of role quite a lot. So they're they're doing they're
36:50
doing great. And then there's other, there's
36:53
other analyst teams that we've actually worked on pure
36:58
basically software kind of yeah solutions with so not not so it
37:04
doesn't necessarily have to be machine learning when machine
37:07
learning is overkill, who also who also seem to enjoy their
37:12
just their basically being able to build more automated software
37:18
based on their data so that then they can actually they actually
37:22
basically release or yeah they actually have more spare time to
37:27
start thinking about smarter things that they could do with
37:30
their data instead of basically debugging and maintaining all
37:33
the time. Yeah, you're freeing up their
37:35
time that that makes a lot of sense.
37:38
And it's funny that the analysts are now considering themselves
37:44
software engineers. I know that for me, the first
37:46
time I like, submitted a PR, changed all of my profile photos
37:53
to be like I am now hacker man. Don't call me Demetrius.
37:57
I You better watch your data. You better watch your systems
38:02
because I am hacker man. I know now I can use git.
38:05
I am ready to take on the Internet.
38:09
So I I feel where they're coming from in that regard.
38:13
And so it feels to me like you've got this enabling
38:17
function, but you also have a bit of and R&D function of going
38:22
out there figuring out a who's doing what and how can we enable
38:28
them and make sure that they're, they know they have exposure to
38:32
what the other team is doing. So that if there is overlap we
38:36
can get them talking and get them using the same tools or
38:41
trying to solve it in the same way and reduce that time to
38:46
production and reduce that time in in exploratory mode.
38:50
But then the other thing is you're going out there and
38:53
you're saying, look, there's a million tools on the market.
38:57
There's a million ways that we can solve that.
38:59
And and that's not even we didn't talk at all about AI and
39:04
LM's yet. But I imagine that totally threw
39:08
a little bit of a wrench in all this stuff you've been doing and
39:12
what people are asking for. So you have to go out there and
39:16
you have to say here's some tools that we would recommend.
39:19
This gets our stamp of approval. We're already paying for them
39:23
with this team over here, you can just join and we'll buy
39:26
another seat by imagine that's kind of how the conversation
39:29
goes. And so you're the one that's
39:32
constantly tasked with exploring the newest, greatest, latest
39:37
things on the market and then making the business case for
39:41
bringing them on and actually either supporting them if
39:45
they're open source or paying for them if they're managed.
39:48
Yeah, so funny. Funny you mentioned also, so
39:51
that's pretty pretty much right. And funny you mention elims
39:55
also, I think it's. I think it's only a matter of
39:58
time before my title will be renamed to Head of AI and
40:02
develops. That's a that's a big title
40:07
right there. Let's just better come with a
40:10
salary bump. That's all I'm.
40:11
Saying but it's also I mean it's also fun that people like to
40:16
basically feel enthusiastic about these new technologies and
40:20
new ways of using machine learning.
40:22
So we've we've implemented 2 reference cases now, but are
40:30
basically in. One of them is in production in
40:33
a way that we tested with a specific pool of test users and
40:39
the other one is still in development but has been yeah,
40:46
it will be close to testing, probably within the next Sprint.
40:50
Because I imagine there is a pool towards some self-serve
40:56
motions O instead of you're going in there and you're doing
41:01
things homegrown. It's like there's this SAS
41:02
product that will get us again maybe 90% of the way there or
41:07
it's specifically built for analysts and we can use that.
41:11
Have you tried to look at that or those types of tools as
41:15
options? Do you feel like siloed, or is
41:18
that something that you're currently using?
41:20
Yeah. So that's a that's also one of
41:23
the projects we're currently running again with the same, the
41:26
same analysts or data scientists who have developed the, the
41:31
model factory for sales leads. Because originally that was
41:36
written, all of it written in code and then we together with
41:40
them or mostly them wrote it all themselves.
41:45
But now they and we were expecting maybe that they have
41:51
would have a sort of ownership to that code and that would
41:55
prevent them from wanting to try, from wanting to try out
41:59
already IT solutions because that's also something that we
42:03
often see with analysts that they they have a little bit more
42:06
ownership to their codes than is healthy I would say.
42:11
But in this case it was then we said Oh no, we.
42:13
So we've seen that basically feature store in Databricks
42:17
would basically would give us the opportunity to probably get
42:20
rid of 90% of the code that our model factory is made of.
42:25
Can you guys help us do that? And so now two of my guys, um,
42:31
are working on that. And so when it works, we
42:35
hopefully have gotten rid of about, yeah, 90% of the code
42:39
that doesn't need to be maintained anymore.
42:42
Yeah, exactly. And then you can ideally
42:45
generalise that to other parts of the business that could find
42:49
use from that. Exactly.
42:51
But this is, so this is a group that we're quite lucky with that
42:54
they are so enthusiastic about this.
42:57
Yeah, about this way of using tools.
43:01
I know there are also other teams who do have who have spent
43:05
significant amount of time, many years and build build tools for
43:12
doing the pricing or whatever it may be.
43:15
And so those will be a bit more of a challenge to convince to
43:21
try the other approach. But I think this kind of change
43:26
management is also a very fun challenge.
43:30
Yeah, because you're not. It's funny because it's very
43:33
much on the human level. Then you're not really able to
43:38
make logical arguments. It's more like they have their
43:43
nostalgia for the way that things are done.
43:45
It's comfortable and that is the way that we do it.
43:50
Why would we want to and reorganise our processes?
43:55
It's going to take us a long time.
43:57
We're going to have to, it's going to be painful, then we may
44:01
not do things right. There's so many reasons why you
44:05
don't want to change, but if you're stagnant then that's your
44:08
your worse off in my opinion. Exactly.
44:11
But we, and we shouldn't also forget that to basically
44:15
acknowledge the amount of effort that teams have put in into
44:19
building their solutions. And that, of course is if then
44:23
suddenly someone comes along and tells you we don't need 90% of
44:26
your code anymore because we have this product that does it
44:29
all for you, That's not that's not easy to handle.
44:35
What's eight? Yeah, what's that?
44:36
There's a term for that where you've, like, spend so much
44:39
money on something that you've got to see it through.
44:42
It's. Yeah, what is it?
44:45
It's something like. Cost fallacy.
44:48
That's it. The sunk cost fallacy.
44:51
So it's almost like that with the code like we've already put
44:54
so much effort into building this.
44:57
We're not going to just give it to some verticalized.
45:03
And I mean one thing, the other part that I was thinking about
45:06
is, yeah, feature store still to me feels like it's not quite a
45:10
fully verticalized SAS type thing.
45:12
Where I was thinking since LM's have taken the scene and they
45:17
every SAS tool now has AI capabilities.
45:21
You have your HR department or you have your legal department
45:24
or you have your sales department.
45:26
Now that says, you know what? We don't need to create our own.
45:34
Let's take a chat box for lack of creativity on my side.
45:38
And let's say we don't need to create our own chatbot with all
45:42
the data and train our model and get something out real quick
45:45
because we just can buy this off the sell off the shelf solution
45:50
and it'll get us 90% of the way there.
45:53
Have you seen people starting to demand that or have you seen the
45:57
other thing? The other piece of it is you've
46:00
got someone that says, oh you know what would be really nice?
46:02
If we had some API calls out to Open AI and it went to GPT and
46:08
it made our lives easier in this these like 5 different workflow
46:12
steps and they're now they're they're including AI into their
46:17
business. But it's not necessarily like
46:20
what you're used to doing where it's a full overhaul.
46:23
You're going in there, you're saying, hey, if we're going to
46:25
do this, we got to get something out.
46:27
We want to be from soup to nuts. We want to be in control of it.
46:31
It's more like, I don't want to say half assed.
46:34
It's just like the when you talk MVP, it's the ultimate MVP
46:39
version. Yeah.
46:41
So we're a little bit both both sides and in the middle of it
46:46
because so the the our biggest problem currently with the, yeah
46:53
with with the generative AI solutions within other products
46:58
is that they are usually the the distributors are usually quite
47:05
thin on on data governance. And so it's difficult basically
47:11
these are difficult for us to purchase at the moment because
47:14
we have very sensitive data about the entire Norwegian
47:17
population and even more so on the part of the population that
47:22
our customers that we cannot. We we can actually not just go
47:26
and buy these solutions before, um, they have put more meat on
47:31
the table in terms of data governance, what they do with
47:34
our data then and where it gets sent to, we do have.
47:39
So we were out pretty early with requesting access to the GPT
47:44
models locked within Asia because that's our chosen cloud
47:51
service provider anyway. And so we are basically the
47:55
solutions that we work on at the moment are actually very
47:58
specialised chat bots for very specific tasks that can that can
48:06
make a lot of our internal processes more effective.
48:09
So for example, the one that we've just in a way or finished
48:15
enough to basically test it with test users is a chat box.
48:19
That's business. Developers who don't know how to
48:22
write SQL can use to then basically query our databases
48:27
with natural language and so they can then get information
48:32
about, yeah, aggregated information about our customers
48:35
back to make decisions instead of having to wait for an analyst
48:39
to answer them. And the other one is a kind of
48:43
RAG application as a retrieval augmented generation where we
48:48
try and improve a, yeah, the search for our customer advisors
48:54
so that when they talk with people on the phone, they can
48:57
search in our internal documents lives in, but they're actually,
49:02
yeah, Azure also has out-of-the-box AI such solutions
49:09
that we're basically comparing to our self implemented RAG
49:13
version. So there's a lot of learning
49:16
also to be gained from basically implementing these solutions at
49:19
least once yourself and then. But we always try and advocate
49:25
to use these cloud native solutions, um, for the actual
49:30
production case. Yeah.
49:32
There is a lot of question marks when it comes to those
49:37
verticalized associations that you were talking about because
49:40
of the big unknowns when it comes to so where's my data
49:46
going and what? What is it doing and are you
49:48
training your model with it or are you just sending it to Open
49:53
AI? How does that look?
49:54
And we were talking about this last week in our build versus
49:58
buy roundtable and the folks from Quantum Black were saying
50:03
how you can get different. You can be on a spectrum almost
50:07
of I don't care about my data and where it goes and who
50:10
touches it versus and versus like next step might be I use
50:15
like an open AI, but I make sure that or a third party API
50:20
service provider, but I make sure that they're not training
50:25
any models with that data that's coming through.
50:27
It's my data. It stays with me supposedly like
50:30
you've got to kind of take it at face value that they're going to
50:32
be actually doing that because who knows what they're doing
50:35
behind the scenes. But that's what they say, right.
50:37
And you would hope that they follow the law or follow their
50:42
own word and and then you say like, no, I need 100% control.
50:47
So I'm going to get everything in house and I'm going to do it
50:50
with my own models. And I do like this use case
50:53
though that you're you're talking about, which is keeping
50:57
the AI stuff happening internally so that it's almost
51:03
like the stakes are a little bit lower.
51:05
Like this person, the business development person who's doing
51:08
text to SQL, if everything goes haywire and the model totally
51:12
hallucinates, it's not like it's going to ruin a customer
51:16
relationship and potentially put you out millions of dollars.
51:19
Exactly. Yeah, like the hallucination is
51:22
OK there. You're not going to blow
51:24
anything up, any relationships, any brand, you're not going to
51:28
have really bad repercussions. Exactly.
51:33
And we, we have a lot of processes that we can that we
51:36
can, yeah, make a lot more efficient with using generative
51:41
AI before we even before we even begin to directly let them talk
51:47
to our customers. And we have this, we have this
51:51
principle at my company which basically says that everything
51:54
that we do needs to be able to tolerate the front pages of the
51:58
yellow press. And so it was the following.
52:02
That principle was very clear for us from the beginning that
52:06
we would want to focus on internal processes first.
52:09
But also of course that means that basically then we can use
52:14
our best customer service operators to basically work on
52:19
those cases with those people that really need them.
52:23
So there's so there will be more so that we will be able to
52:26
deliver basically more personal, yeah customer service by just
52:33
making everything else internally as efficient as
52:35
possible and then automating whatever can be automated.
52:39
Yeah, yeah. So how do you prioritise what
52:43
use cases you're going after? It feels like you've got a whole
52:47
spectrum of things that you can tackle, whether it's enabling
52:51
other teams, it's optimising current things that are there.
52:55
And you mentioned how you're trying to just make yourself
52:57
redundant. But then you've also got this
53:00
other piece, which is, well, we could do a lot with AI in
53:04
quotation Marks and do that for the internal team and optimising
53:09
the way that the internal processes happen.
53:13
What do you have a rubric on how you go about things?
53:18
And luckily I've needed one that much.
53:23
Yet it's yeah, for lack of a better word.
53:26
So I have to use a buzzword and say I do it in a in an agile
53:30
way. But basically it just yeah, so,
53:34
so far, fingers crossed it stays that way.
53:36
But so far it just so it just worked out so that basically
53:41
teams were ready to be able to work with us when we could, when
53:47
we could start working with them.
53:48
Other teams might have had to wait a little bit, but then we
53:52
give them, we give them an approximate sort of start time.
53:57
But one important principle is also to never basically promise
54:00
how long a project is going to take because you never know.
54:02
And then it's also just. But I also use basically what my
54:08
guys want to work on as a guidance.
54:11
So I I'd like to keep happy team that works on interesting
54:15
projects, but then in between an in between 2 interesting
54:20
projects. Then I feel like it's nice to
54:22
squeeze in a project that's maybe slightly less interesting
54:27
from the technical perspective, but a huge gain in terms of
54:31
business value, like a minimum effort kind of maximum result
54:36
kind of project. And that feels a little bit more
54:39
like a chore maybe, but it would be really important to do.
54:43
So it's like it's about balancing basically who works
54:45
and what. And then I also try to always
54:49
rotate out who in my team works together so that we can spread,
54:55
yeah, all the skills that we have amongst everyone.
55:00
So I hate to break it to you, but I think if you're going for
55:04
that head of AI title on LinkedIn, you gotta have some
55:07
kind of rubric. You gotta be holding how This is
55:11
how we do AI at my company. Here's the formula for it and
55:16
maybe some acronym. We got an acronym that you can
55:20
set team, make sure the team is happy.
55:23
So I just noticed that like that change management.
55:27
Kind of like what some people may consider a little bit more
55:31
fluffy, that kind of stuff. And whenever you see someone
55:35
talking about AI, I imagine you've seen the meme where it's
55:38
like, if it is written in Python, it's machine learning.
55:41
If it's written in PowerPoint, it's AI here.
55:45
You know, if it's like, we'll figure out some kind of a rubric
55:52
or some some acronym that would be flashy.
55:55
And then you can change your title, you can have the head of
55:58
AI, change management, all that fun stuff.
56:01
But the other thing that I was thinking about when you were
56:03
talking about this, to come back to the point after that little
56:07
sidebar, was around how you scope these different projects.
56:11
Because as you were saying, like, oh, there might be a
56:14
relatively low effort thing that can cause huge value for the
56:19
business or there might be something that is very valuable
56:23
and it's probably going to be a little bit more involved.
56:27
I know that from years of talking to data scientists and
56:32
machine learning engineers, scoping out data projects is not
56:37
quite as easy. I mean, scoping out software not
56:40
easy in general, like when you add data to the mix, it just
56:43
adds that extra layer of complexity.
56:45
Have you figured out ways to at least contain scope creep and
56:52
then the amount of time it takes to do things?
56:56
Yeah. So that's an that's a very
56:58
interesting question as well actually.
56:59
So the last with the last couple of projects, it so happened that
57:04
we were able to deliver the solution quite significantly
57:09
under time because it turned out that the tools we were using,
57:14
yeah, made made some of the jobs so much easier than the original
57:20
solution that we were trying to modernise had.
57:22
So that was that was quite fun but that makes my job funnily
57:27
enough that makes my job harder because it means I need to find
57:30
new projects faster. But anyway, so to actually
57:35
answer your question, we basically we focus on lots on
57:39
the minimum viable products to production approach.
57:44
So for example, we have, we just started a project with a team
57:49
that works on the profitability prognosis for all our insurance
57:54
products. And so in the way they usually
57:57
would have packet A project and they would have basically done
58:02
all their data transformations for each product 1st and then
58:06
all the modelling for each product next and so on and so
58:09
forth. And then the visualisation for
58:12
each product last, whereas we are saying we're taking one
58:16
product and then we're taking it from end to end.
58:20
So that basically in the shortest amount of time you
58:22
learn how to do the entire solution end to end.
58:25
Then we will take the next product and probably will will
58:28
try and pick the product that is going to be the trickiest there
58:35
and then we will we will work on that end to end.
58:39
So if we then are out of time in terms of what we've allocated as
58:44
time from the beginning no worries because then you know
58:47
everything that you need to do to finish this you don't need us
58:49
anymore and or we can if we have we can try and allocate more
58:54
time but ideally they weren't need us anymore and they will be
58:57
able to finish the rest. But it's always this purge,
59:02
this, this focus on end to end solutions from the start, from
59:06
day one. Yeah, that helps us.
59:09
It's almost like you're tasked with going from like one town to
59:15
the next, and the way is not clear.
59:20
You may have to go through the jungle, you may have to go over
59:22
mountains, you may have to go through some pretty gnarly
59:26
terrain. And you're going to figure out
59:29
what is the way that we go. So you can map out that way.
59:33
And then once you have the way mapped out, you've cut through
59:37
the jungle, then you're like, OK, now let's bring all the
59:39
supplies, let's print, make it a little bit more difficult so
59:43
that we can really start to maximise the maybe you, you can
59:49
build a highway, shoot, if we're thinking big, she say, no, we
59:52
don't even want a trail anymore. We're going to set up a super
59:55
highway and we're going to make sure that all of these supplies
59:58
get from town A to town B in a fast fashion.
1:00:03
Yeah. But the point then is that we
1:00:04
basically only built the highway if we actually end up needing
1:00:07
the highway. So we could if we're only if
1:00:09
we're only shipping very small number of products, we don't
1:00:14
need more than a trail maybe. So the way I usually I usually
1:00:19
try and visualise it is especially when your customer
1:00:23
wants a car, or they think they want a car, then you're going to
1:00:28
give them. You don't.
1:00:29
You're not going to give them a wheel first or a set of two
1:00:32
wheels, but you're going to give them first in the skateboard
1:00:35
because you can't get from A to B with the skateboard.
1:00:37
It's going to be, it's going to be slower, it's going to be
1:00:39
painful, but you're going to get there.
1:00:42
And then in the next step you're getting them a bicycle and then
1:00:46
in the next step you're getting them a motorbike.
1:00:48
And maybe by the time you've given them a motorbike, they've
1:00:51
actually already figured out we don't need the car.
1:00:53
Instead of basically having first delivered a set of four
1:00:57
wheels and then having delivered a chassis, and so you actually
1:01:01
able to, you're actually able to deliver the entire business
1:01:05
value from the from the minimum, the actual minimum viable
1:01:10
product. So the skateboard.
1:01:12
And in a way, that's still painful, but it works.
1:01:17
And then you take it from there. OK, that is a much better
1:01:22
metaphor than I was putting together there.
1:01:24
And it makes complete sense on how you are helping people
1:01:29
understand that you've got this tool that is going to be useful
1:01:36
and it's going to get you what you need.
1:01:38
And then if you encounter that it is not living U to the
1:01:42
standards or your needs and your requisites, then you can
1:01:47
upgrade. Because we got this fancy E bike
1:01:49
over here that will get you there a little bit faster and we
1:01:52
can make sure that that's all you need.
1:01:54
And then if you say you know what, that's still not what you
1:01:57
want as opposed to going for the car right from the beginning,
1:02:02
You can step up incrementally. And I do see what you're saying
1:02:07
now it it makes something clicked in my head, which it
1:02:10
took me a little bit longer than I actually would like to admit
1:02:14
on how you're able to do things much faster that way because
1:02:21
you're not having to go through the intricacies of building a
1:02:25
car. You're able to.
1:02:27
It's almost, it's like you dumb it down to a skateboard and then
1:02:31
that works. That's great.
1:02:33
We're out of here. Yeah, exactly.
1:02:35
So the the the principle from software engineering that we
1:02:38
actually try and use the most and the the sentence that we use
1:02:41
the most is You ain't gonna need it.
1:02:44
That's right. Yeah.
1:02:45
So you have that hanging on your wall or somewhere?
1:02:48
Ohh, maybe I should, yeah. And then you can just show
1:02:51
people that point your finger to it.
1:02:54
Exactly, yeah. Yeah, that is really cool.
1:02:58
Alright, last piece of information that I want to bring
1:03:02
up is that you not only do all this, but you got a friend that
1:03:08
was part of the Tour de France and was on a team that went
1:03:14
through. And yeah.
1:03:15
He was a chef for the Tour de France and all right, so.
1:03:19
For one of the teams that. I was going to call him a doper,
1:03:22
but again, getting through this, who knows at the end one and we
1:03:28
can What is this for you? You have a restaurant on the
1:03:31
side. Yeah.
1:03:32
So actually you know, well actually now we had to close it
1:03:37
because, but we had it for a couple of years.
1:03:39
Yeah. It's basically the idea came
1:03:42
from we have this we have this cycling group that writes very
1:03:47
early in the morning at 5:40 AM called Oslo Dawn Patrol and now
1:03:51
they're actually are dawn patrols in in many other cities
1:03:55
in Germany has one and killed by the way as well.
1:03:58
And so basically what we would do after that so we'd be we'd be
1:04:01
done writing at like 7:00 AM and so we'd be going we'd be wanting
1:04:05
to go eat breakfast together and so we would go to usually a cafe
1:04:10
and then you'd be able to get a cinnamon bun and a coffee.
1:04:13
But we wanted a place where we could have proper breakfast and
1:04:17
that's how this idea came along and then it developed into an
1:04:20
entire restaurant. But then unfortunately some a
1:04:25
group we brought in as investors, they invested into
1:04:32
too many locations at the same time and went bankrupt.
1:04:37
And so when we were looking for a new investor to join us, this
1:04:41
new potential investor instead turned around and made a better
1:04:46
offer to our location and then snatched our location away from
1:04:49
us. Ah, no, yeah.
1:04:54
It's a difficult business. Cutthroat.
1:04:58
Wow, yeah. It was not expecting that.
1:05:00
No, but it was really a fun experience to to work on that.
1:05:03
But yeah, the restaurant business is difficult so.
1:05:07
Wow. Wow.
1:05:08
Incredible. Well, I guess that's all then,
1:05:13
Giant. It just shows you life, will
1:05:16
show you where you need to focus.
1:05:18
And it seems like you are better off focusing on enabling the
1:05:24
teams at your company and helping them get things into
1:05:27
production and leverage the data and machine learning and
1:05:32
analytics capabilities that are out there.
1:05:34
It's been incredible talking with you.
1:05:36
Alessandra, thanks so much. Thanks for having me.