Latest / Agentic Conversations (formally mlops.community) / Data Engineering in the Federal Sector // Shane Morris // #223

Transcript

Data Engineering in the Federal Sector // Shane Morris // #223 · Agentic Conversations (formally mlops.community)

0:00
Hey, what's up? I'm Shane Morris. I'm
0:01
a senior executive advisor for Devus or anything on
0:05
the board at Dada X someone advisor for a
0:07
little ETL company called mage as well And I take
0:10
my coffee with two wooden spoons of sugar And then I
0:14
have a milk frother that I use and so I have a very
0:18
fancy milk frother So a lot of froth milk and
0:22
sweet and a French press. It's
0:24
my coffee What is up MLOves community? We
0:27
are back for another podcast Today with mr.
0:31
Shane Morris I've been really
0:33
enjoying talking to him because He's one of those
0:36
guys that I found on tiktok and then I started
0:39
following on social media and just See him as a
0:44
Multidimensional human being without getting too woo -woo
0:48
He's got many different lives that we get into
0:51
here He was a music manager before he got into
0:54
data engineering worked for the government We went into
0:58
some of the qualms some of the challenges of working
1:00
for the government as a data engineer and getting
1:03
design documentation and getting new tooling through
1:06
in These design specs or these projects that you're
1:10
building out in government operations and how that
1:13
differs from being in the private sector And we also
1:17
talked a whole lot about Data X the company that he
1:21
is an advisor for and this crazy Programming language
1:25
which I will not spoil at all and I will let him
1:28
explain a hundred percent My biggest takeaway that I
1:34
thought was the best piece of Information is how
1:41
bullish he is on data engineering It lines up a
1:46
hundred percent with what I have been feeling and a
1:49
lot of people have been saying the more LLMs we've
1:53
got out there the more AI we've got out there the
1:56
more important data engineering is going to
1:58
become and You made some really funny analogies Which
2:03
I will hold off on saying because I do not want to
2:06
spoil anything. Let's just get into it with
2:09
Shane Huge thank you to everyone out there listening
2:13
and if you like it, you know what helps a ton is
2:17
hitting that Follow button if you're on Spotify and
2:22
of course if you are listening on YouTube or
2:24
watching It would mean the world to me if you Go
2:29
ahead and comment. Let us know what you thought
2:31
of the episode Let's get into it with Shane We are
2:41
recording this for the second time because the
2:43
first time the recording and the audio didn't go
2:45
through but Since we started recording or since
2:49
that first time that we were recording until now
2:53
You got a new job and when we last talked you
2:56
were like, oh, yeah, I literally just left today I
2:59
left the job then the next day I saw in LinkedIn you
3:03
congratulating yourself, which I thought was
3:05
hilarious post but we can get into that later and So
3:08
I think man one of these things is that you put
3:12
yourself out there and the surface area you cover is
3:14
So large that of course you're gonna get a job in
3:17
two days Yeah, I think the success I had in getting a
3:20
job was Going places where I didn't belong even though
3:25
I Just kind of showed up randomly So the reason I
3:29
was able to get job quickly is because I met a
3:32
guy on tiktok because he had heard about me through
3:35
a net app event net app as a storage company for
3:39
you guys who don't know and He ended up in my
3:42
fantasy football league And so when I was a free
3:45
agent, you know, we were all data science dudes and
3:49
machine learning engineers Which by the way is an
3:51
absolutely vicious fantasy football league devil a
3:53
bunch of dudes who studied science and a fantasy
3:56
football league and yeah, so within 24 hours I had
4:01
talked to him and 24 hours later I had an offer that
4:04
I liked and so I know that it could be a tough
4:06
job market But I think if you're willing to go places
4:10
where you don't necessarily think you're gonna be doing
4:13
anything They're just to meet other people in
4:14
adjacent markets like I don't do storage stuff I
4:17
mean kind of and not really but that's that's
4:19
the reason I ended up, you know getting the position I
4:21
got exactly that man It's all about putting yourself
4:24
out there in that way like getting out there just
4:27
hanging out That's why we do the local in -person ML
4:31
Ops community meetups But also I think one other
4:34
thing that you do really well is you talk about
4:37
your experiences on LinkedIn you you could talk about
4:41
some really awesome experiences on tick -tock
4:43
and that's actually how I found you Back in the day
4:46
and so I'll tell this story again because I find
4:49
it to be Incredible it was the biggest pattern
4:53
interrupt I think I've ever had on tick -tock and
4:55
that's saying a lot because tick -tock is one of those
4:58
places where you get pattern interrupts but it
5:01
went from me scrolling tick -tock and then landing on
5:05
one of your Videos and you were talking about how you
5:09
were a music manager and I was looking at the video
5:14
And I was like yeah that tracks like you had the
5:17
gear on you have a grill for everybody that is just
5:19
listening You've got some gold teeth. You've
5:21
got tattoos. It was like yeah music
5:23
manager. That's that's alright And then at
5:26
a certain point in the video you went from talking
5:29
about how you were managing Travis Scott or panic at
5:32
the disco And then you were like yeah, so I just
5:34
spun up some VMs And then I started sending traffic
5:37
to it and it was like wait what did he say VMs?
5:40
And then you were like and
5:41
this data pipelines and stuff and I was just like
5:43
no Is he is he talking about like stuff that I
5:47
actually talk about on a daily basis? And
5:49
then I went deeper down the rabbit hole and I got
5:52
to know who Shane Morris is and it's like wow multi
5:55
-dimensional here Yeah, it's it's kind of interesting
5:58
because most people don't associate associate music
6:03
management with data pipelines or data
6:05
integration But back in the day, you know, it was
6:09
really the Wild West of data pipelines and being
6:12
able to spin up VMs and game SoundCloud and other
6:14
services to create fake plays Because you could
6:18
fake it till you make it on the internet and it was
6:20
especially easy If you were someone like me who was
6:23
industrious and willing to spend $30 a month on cloud
6:26
services to have your VMs running And so as soon as
6:30
I realized that SoundCloud didn't have anything getting
6:33
in the way of me, you know Getting on mechanical
6:35
Turk and having them create accounts for me and then
6:37
log again, right? I was like wait a minute
6:40
like I can have a thousand SoundCloud accounts. This
6:42
is great. You know, this is all I need
6:44
and Yeah, I think it's funny now looking back as
6:48
people are like, what does music management have to do
6:49
with data? Well, I mean music is an
6:51
inherently digital asset, you know, you're streamed
6:55
your accounts your analytics You know your e -commerce
6:57
sites are ticketing This is all digital management at a
7:01
very high level And so the reason I was successful in
7:03
the music industry is because so few of my peers
7:07
Were as advanced as I was, you know from a software
7:11
sophistication level You know, we had people who
7:13
were just getting used to maybe like google analytics
7:15
back in maybe 2012 or 2013 Just be like how many
7:18
people are coming from north america? And
7:19
I'm like, how many people am I faking coming from
7:22
north america? Let's find out, you know So I
7:24
had a lot more I guess I had a lot more fun than
7:26
some people did but a few people got angry at me.
7:28
They were like, hey, it's not
7:29
okay You broke the rules. I'm
7:30
like, there are no rules. Yeah,
7:33
well show me where it says that Show me where it says
7:35
the rules and I will Show you a lie. Yeah,
7:39
that's for sure There's so many good stories that you
7:42
have from this and I would love to go down the rabbit
7:45
hole of just like You were working at a website that
7:48
again when it comes to data You could get really
7:52
granular with the data because how mature you were
7:54
back in the day So you could know a Which bands
7:58
were popular where they should go and tour and If
8:03
if I'm remembering this correctly different places
8:07
like Coachella or Lala Palooza would reach out to
8:10
you and be like, who should we get for next
8:12
years? Uh festival. Correct.
8:16
Yeah. So really early on at ear
8:18
milk, you know, this is You have to remember ear
8:21
milk had a streaming music server before SoundCloud
8:24
existed So we would upload our own music we had our
8:25
own analytics dashboard Because when you're hosting
8:28
content on a music blog You want to post what's
8:31
popular because then people will come back and you'll
8:33
see, you know, where the fans go Now by doing this,
8:36
we also started hosting our own small events in Atlanta
8:38
and San Francisco, Toronto Or as the Canadians say
8:41
Toronto strange way to pronounce it but uh, you
8:45
know, we would host our own events and then talent
8:47
buyers for you know, ag and live nation and some
8:50
smaller promotion groups would say Hey, Shane, you
8:52
know, we're booking this band. We
8:54
need to get support act, too Do you think would be
8:55
a good draw locally? I'd be like, oh these guys,
8:57
you know, because according my analytics, there's a lot
8:59
of bands of indie band x in southern california
9:03
whenever it is and so You know, I you know, I've got
9:05
so many strange stories like uh, there was this
9:08
band called foster the people we booked their
9:10
first ever show Uh, I booked geese's first ever
9:14
show in atlanta, georgia. Um
9:16
Obviously, you know, I got involved working with Travis
9:18
Scott, but you know, we were like Uh, the first
9:21
ever for a lot of people. Um,
9:23
I remember ellie golden playing, uh, beer pong My
9:26
sister taught ellie golden to play beer pong because
9:28
we booked her in um, I love this to outside of
9:31
santa barbara And so she was you know, kind of a
9:33
small little indie electronic act and you
9:35
know, we got her I want to I want to say we paid
9:37
her $500 for the first show. Of
9:39
course she blows up So, um, you know, it was cool
9:42
being on the front end of that analytics because we
9:44
could see this huge upward growth trajectory But the
9:46
fans hadn't really figured out how popular some of
9:48
these artists were yet And so yeah, we had talent
9:51
buyers who were happy to be like Your please help
9:54
me and that's why I made so many connections because
9:56
I probably could have sold that information But I was
9:58
trying to be friendly and helpful. Uh,
9:59
so it helped me later down the road Wow, dude. And
10:02
so you were the epitome of like, I knew this band
10:05
before they were cool. Yeah Yeah, I mean, I you know
10:09
going back I remember we had this festival called
10:10
music in the middle in 2013 and uh, looking at
10:13
that That headline flyer. We
10:15
were the first, uh, you know, I guess promoter in
10:18
that sense because we were promoting our own show But
10:19
you know, we booked the 1975 right when they were
10:22
coming out I remember they had just put out this
10:23
single called chocolate and we were the first one and
10:26
if you go back to their original Reissue they just
10:28
sort of reissue of the stuff on their website and
10:31
you can see that original tour they did in 2013 and
10:33
it says Ear milk festival for the first date It was
10:36
because we flew in and landed them like the reason
10:38
they were able to land here is we paid them an
10:39
Exorbitant amount of money to get here. I
10:41
remember that and uh, you know, jeez. He
10:43
was their tv girl I remember flying out to la
10:45
tv girl is a big band now. I
10:47
was hanging out. I think it's just a one -man
10:49
band I don't know if it's like lead singer situation,
10:51
but anyway, I remember hanging out with him in
10:53
his apartment I want to say was an echo park or
10:55
something And I was like, here's this dude. He
10:57
was a great artist But nobody knew tv girl was
10:59
and then now of course tv girl's huge So there's so
11:02
many of those to first where I would just like
11:04
randomly run into these artists before they got
11:07
huge and today it's kind of funny because I'm still
11:10
on a first name basis with a lot of them and a lot
11:12
of their management Because they're thankful that I was
11:15
giving them their start when no one else would
11:17
because we had our own events We could throw air
11:19
milk shows in Atlanta And so I'd be like, I'm gonna
11:21
book you, you know, or our monthly and they would show
11:24
up and play shows It was it was a really awesome
11:25
time to be a music blogger Dude, so music blogging is
11:30
one of those things that sadly it feels like it has
11:34
kind of gone out the window with the spotify
11:36
recommender systems but I feel like Do you have a
11:40
playlist on spotify where you you've compiled all of
11:44
the people that you've worked with over the years
11:45
I have like a blog house playlist Yeah, so we called
11:48
it the blog house era And I was it's interesting you
11:51
bring this up because I was just talking to my
11:53
buddy, brendon shout out to pizzas on tiktok brendon
11:56
He's a pizza account, but he's also just an all
11:58
-round cool dude, but he and I were just talking
12:00
about the blog house era because it doesn't it did
12:05
I think it had to exist in that perfect moment in
12:07
time where it was Decentralized as we say we
12:11
didn't have spotify at apple music It was de
12:13
-centralization before the blockchain was cool, bro
12:15
And so, uh, you know, we had this fragmented
12:18
ecosystem that allowed us to be independently
12:20
successful um, and You know it was cool because the
12:24
music I feel out of that era Was strangely unaffected
12:28
by what was algorithmically cool And I've had
12:31
discussion with the artists about this where I think
12:34
sometimes they chase a sound that they think is
12:35
currently cool Rather than defining themselves and I
12:38
think because you could still be hyper regional You
12:40
know music from the bay area sounded like bay area
12:42
music You know, houston rap sound like houston rap it
12:44
land out of sound memphis out of sound and now
12:47
Because you have these like top rap playlists top pop
12:50
playlists Some of the the regional idiosyncrasies of
12:54
music do get lost to the bigness of it and I do
12:56
have very strong opinions about this Yeah, but and
12:59
that's so funny. We'll link to the playlist that
13:02
you have because I feel like that is treasure trove
13:06
of great jams and It is so true how you can just
13:10
like lose yourself chasing the streams And because
13:14
basically it's all gone on the spotify right now The
13:17
other thing that I I think is fascinating and this is
13:20
not going to be a conversation 100 percent
13:22
about music streaming and how to be a music manager
13:26
Uh, we're going to get into the data aspect of it
13:28
because you went and you worked for like companies
13:32
that Related with the government slash government
13:35
companies and so I want to go into That whole thing
13:38
and the headaches and pros and cons tradeoffs and all
13:41
of that But before we do I think there was something
13:44
that stuck with me When you talked about how much
13:49
money is in technology versus music Right and I
13:54
couldn't get that out of my head like once I heard
13:56
it. It was just like oh dang like
13:59
Musicians are chasing crumbs compared to Like tech
14:04
companies and so can you reiterate that for us?
14:06
Yeah, so this is the the analogy
14:08
that I always try and draw people so people understand
14:11
how much money is in music versus how much money is
14:14
in literal hardware so Last year spotify made $17
14:20
billion it was the first time they were actually
14:22
profitable right also last year apple made $17 billion
14:30
selling airpods Just the airpods that's it that
14:34
doesn't include their computers their phones Just
14:36
their airpods and and so when you break down what
14:39
your expenditures are and I learned this when I was
14:41
working at sunny music Nashville, what do people
14:43
spend money on well? You get your streaming revenue,
14:45
which is pennies. It's nothing Uh, you got some
14:49
ticketing revenue. You got some merch revenue
14:50
You'll get your publishing, you know if you manage to
14:52
get your song commercial movie whatever But there's
14:55
just not that much money that people actually spend,
14:57
you know If I look at a high value fan the the
14:59
taylor swift fans the rabid fans They'll buy those
15:02
tickets, but taylor swift is the extreme outlier
15:05
most, you know concert tickets are that maybe 40
15:07
to 50 dollar range You'll spend I don't know 12
15:10
dollars a month on spotify So on a good year, you
15:13
know assuming you're you know Unmarried single person
15:16
or maybe you're married no kids and you can actually
15:18
go to concerts You don't throw a range for the
15:20
babysitter You might go to four five six ten concerts
15:23
a year if you're really having on at one festival
15:25
But even then that's not even a car payment. That's
15:28
nowhere close to what you're probably paying for
15:29
your rent or your mortgage And so it's just a tiny
15:31
amount of money that people spend on music And so and
15:34
the music business you are fighting for people's the
15:38
smallest amount of revenue and the smallest piece of
15:40
pie Versus, you know the reason the main reason I
15:43
got out of working at music is is I had a
15:45
friend who was working Advanced the shoe company
15:48
and he was like hey, can you do a little six month
15:51
project? And it was literally my entire
15:54
annual salary at sony, but in six months of work. I
15:56
was like, wait a minute I could double my pay Selling
15:59
shoes, but then I was like wait a minute. I
16:01
pay a lot more for shoes than I do for music Makes
16:03
sense. It makes total sense, man. And
16:05
it's so wild to think about that especially
16:08
because The merch it just makes me think like next
16:12
time I'm at a concert I definitely got to buy merch
16:15
just to support those artists I mean the merch
16:18
the merch is huge, you know when we look at you
16:20
know, your your markup You know because you're buying
16:22
hoodies or t -shirts or whatever in bulk Yeah, 35
16:26
to 40 shirts pretty steep But you gotta remember they
16:28
don't have to pay shipping when they're at a concert,
16:31
you know Because the logistics kind of come with
16:32
them and they do pay for shipping, but it's landed,
16:34
you know And so yeah, they might be making 30 bucks a
16:37
shirt By selling merch with their shows versus if you
16:39
buy it on the website or amazon You still got to
16:41
pay the fulfillment. You got to pay amazon their
16:43
fee. They make a lot less So I
16:44
would always say please buy buy merch at the shows. It
16:47
definitely is a higher profit margin Yeah, and
16:49
even now nobody's selling CDs, right? That
16:52
was I So I was making music and going around
16:56
touring we could say from like 2012 to 2020 when the
17:03
pandemic hit and Right up until the end there. We're
17:09
still selling the occasional CD We had them it wasn't
17:13
that big of a draw, but After 2020 man after the
17:17
pandemic nobody's buying CDs. Yeah,
17:19
nobody's buying CDs I buy vinyl, but I only do it
17:21
for the reason everybody else does to be pretentious
17:23
when I have company. That's it Yeah Exactly so it
17:28
makes it even harder. It's like yeah, and just makes
17:32
it so hard, but That's not why we're here. Yeah,
17:35
we'll pivot right now and talk about this idea of
17:40
you went and you were working for The government
17:43
we could say and it's really hard doing stuff in
17:46
the government And I know we went into so many
17:49
different ways and challenges that it's hard I
17:53
would love to kick off the conversation on just getting
17:56
a design document okayed When you want to get
18:00
something through for a project Breakdown that
18:03
process. Yeah, so about I spent two years
18:06
and nine months working for booze Allen Hamilton which
18:09
is a large federal and defense contractor and booze
18:13
Allen is unique because there's booze Allen in
18:15
deloitte and gd it And guide house lied us center
18:19
federal. They all kind of operate in the
18:20
same space And so if I would like to change
18:25
someone's it architecture, you know any government
18:27
entity It's not like in the commercial world. Why
18:30
can't you say we find this more efficient and we might
18:32
save 17 % on cloud cost whatever the the reason
18:35
behind it is You have to go through depending upon
18:38
the agency There's like ato processes, which means you
18:41
have to get an authority to operate is your software
18:43
fed ramped What level of fed ramp is it? You
18:47
know and then there's also a stick hold ramp mean fed
18:49
ramp There's a fed ramp low fed ramp moderate and
18:51
fed right high and it's whether or not you can put
18:53
it into classified environments So it's it's
18:57
your level of classification that's allowed And so
19:00
there's a lot of classified environments in the
19:01
government You know and some of them are places
19:04
you wouldn't think like nasa has classified
19:06
environments for instance You know there's a public
19:08
space agency, but they still have you know skiffs
19:11
that they have to operate operate out of sometimes
19:13
And so, you know, whether it's do d work or nasa
19:16
work health related work sometimes you have to have
19:19
fed ramped software and That's incumbent on the
19:22
software company to submit their software through a
19:25
fed ramp process And so you can find software that
19:28
you want to use sometimes But if it's not fed ramped
19:30
you have to find an alternative So you're a
19:33
little more limited in your tool set and what you can
19:36
can't use Which can be a little annoying because you
19:39
have to there's workarounds in that case And even just
19:43
saying all right, we're going to use the newer
19:45
version of python Would be something that you would
19:47
have to get checked out Yeah, so anytime you want
19:51
to upgrade something you you typically get locked in
19:53
a version Because they want to make sure that there's
19:56
no security holes like everything in the government
19:58
is hyper security oriented and so even if I want to
20:02
go from you know, some python buster 3 .9 to
20:05
whatever You know, I guess 4 .0 whenever it arrives.
20:07
Uh, I'm not I can't just go
20:09
do that I can't just update the newest version
20:11
even if I believe that it's going to be more
20:13
secure and they're advertising all these bug
20:14
fixes Etc. You still have to go back
20:17
through an entire security process to say will it
20:19
break anything? And there's you know a testing
20:21
process we go through it's once again It's not just
20:24
as easy as it is in a commercial environment where
20:26
you know, you're making the best decision It's that the
20:30
government has to say yes And we agree to this
20:32
because there won't be any effects that we're not
20:35
expecting And the people that you're Having to make
20:40
the case to are not necessarily deep in the
20:43
weeds on this I imagine No, often they're not and
20:46
it doesn't mean that they don't have good intentions.
20:48
It's uh, it's just that many
20:50
times what you'll find is the people who are
20:52
administering Uh, government programs. They
20:55
might have phds and a specific type of earth
20:57
science, right or they're uh, you know A colonel or
21:02
a general and their degree was in biology, right?
21:05
These are people who are educated
21:06
just not specifically at it There aren't that many
21:10
people of computer science degrees that go and work
21:14
as military officers or as You know gs -15s or scs is
21:18
and the simple reason is pay if you have An
21:23
advanced degree in computer science or a year or
21:25
somebody who is a software engineer It doesn't really
21:28
make sense to stay in a government position given
21:30
that The pay is not there and at the grant that they
21:32
have the retirement programs and some federal benefits,
21:35
but It makes more sense to be in the private sector
21:38
Which is why I don't bemoan anybody who decides
21:40
to you know exit the military after six years
21:42
and and then decide to go in the private sector
21:44
there's a Rather unfortunately, uh, a real
21:48
talent gap in in the federal government right now
21:51
Where we actually need a ton of talent. So
21:53
if you feel driven and you're a public servant
21:54
you're motivated that kind of thing I highly encourage
21:56
you to go, you know, either, you know Become a
22:00
military officer get a go go work for nasa or no or
22:03
the department of health There any of these other
22:05
departments that desperately need people with computer
22:07
science degrees So when you were doing stuff You got
22:14
security clearances. Did you get to get classified?
22:18
Yeah, yeah, so I did get so as
22:19
part of the process I became cleared right and
22:22
there's an entire process where You fill out an sf
22:25
86 and they investigate your entire life and
22:27
everyone you know and your family and all your
22:29
neighbors Uh, it's uh, it's interesting to say the
22:33
least And uh, because you know on your sf 86 you
22:37
need to be honest about your past and you know I
22:39
was like, hey, yeah back in the day when I was in
22:41
california. I you know smoked like a
22:43
fire, you know and uh, so I had a long weekend
22:47
talking to my investigator going through all the video
22:49
games that I was playing at any given time and You
22:52
know, I was what you do this week. I
22:53
went to colorado and went skiing You know, just you
22:56
know kind of going through it and so finally I I had
22:58
to go through an adjudication process and
23:00
write a Write on a thing saying I promise I will
23:03
never ever smoke as long as I'm working with the
23:05
government And so I get I can honestly say I haven't
23:08
smoked since I wrote on that piece of paper because
23:10
I do take the job seriously Um, and I still
23:13
am a contractor now with my new job. So
23:15
I have to you know, make sure to play by the rules
23:18
Yeah, yeah, yeah, so this is this is fascinating. I
23:22
wasn't gonna say how the fuck did you get the
23:24
clearance? But I was I think they did a
23:28
good job as they were that thorough I imagine if there
23:30
was anything to find I hope that they would have
23:33
found it Yeah, seems like you're clean as well. Yeah,
23:35
the the the thing is you can have an interesting
23:36
background But you can't so remember when you're
23:39
thinking about Clearable jobs or this is for
23:41
anybody who's maybe listening this and it's
23:42
like and can I get a clearance? um
23:45
Typically you need to be us born That's either
23:47
there's a few people out there that are canadians
23:49
that you know got married maybe but for the most
23:51
part It's a five eyes nation. So
23:53
Canada, Australia, New Zealand, England Um, and
23:56
maybe you got married to somebody and at the most
23:59
you're going to get a secret clearance there
24:00
Sometimes you go to top secret if you're ex
24:03
military and you're a five eyes nation But what
24:05
they're really looking for if this like makes sense
24:08
is that you can't be compromised You can't have
24:10
a ton of like outstanding debt gambling debt Um, they
24:14
don't want people who have like a background of
24:15
domestic violence or violence They're looking for
24:18
people who are pretty stable financially secure
24:22
No, good credit score. I think they do a credit
24:24
evaluation So they're just basically trying to make
24:26
sure that you're responsible with money And in a way
24:29
that you know somebody was to come to you, you know
24:31
with a heavy russian accent and you know offered you
24:34
$40 ,000 you wouldn't be inclined to take it because
24:37
you don't need the money Uh, and I think that's the
24:39
one of the biggest things is they look for financial
24:41
security But also u .s. Background
24:43
and And then they do look for foreign travel as well
24:46
my background is a little interesting because My
24:49
wife's family is from Baku, Ajah, Pai, Shad and they
24:51
were like, so what's it? You
24:52
know, what's your wife's family like and I'm like,
24:54
um They speak russian their house is way too hot. I
24:57
think they're kind of cheap You know, they should run
24:59
the air conditioning a little for What do you
25:02
want to know eat funny food? I
25:03
have no idea that that's so cool and and again this
25:07
just Confirms my point of like the more that I get
25:11
to talk to you and get to know you is the more
25:13
interesting it gets And so now you you moved on from
25:18
booze booze allen hamilton, which It was great. What
25:23
are you doing these days? Yeah,
25:24
so I was like I was unemployed for 48 hours and
25:28
I got an offer from a small business called devis
25:31
out of arlington And so, you know in the government
25:35
contracting world, you're either bigger or small and
25:37
I decided to go to an sba So i'm doing that and then
25:40
i'm doing some consulting on the side One for a
25:44
company that I know you're familiar with called mage,
25:46
which is a etl and data pipeline tool So I
25:49
befriended tommy and i've just been kind of helping
25:52
him navigate some of the startup process a little
25:54
bit So he and I text back and forth, you know pretty
25:56
much all day every day now Um and then uh, I never
26:00
see yeah, dude the guy never sleeps. He's
26:01
a machine. It is absolutely crazy um And
26:05
so uh, I've been i've been talking to the mage just
26:07
trying to you know help out tommy because I love
26:09
their product And then i'm also on a board now with
26:13
a company called decision zone. They've
26:14
got a product called data x And I was really
26:17
thrilled that they invited me part of their board
26:19
Because I wanted to for a long time I just couldn't
26:22
get it through the approval process at booze allen
26:24
because they felt It might be a little conflict of
26:26
interest and I was like, I'm not gonna force it. I
26:29
can always do it down the road Um, you booze on the
26:31
big company. They've got a lot of investments
26:33
So if they feel like me being on the board would
26:35
compete with one of their investments I understood
26:37
they kind of explained it to me that way. Yeah,
26:39
one of the up there's upsides and downsides of
26:41
working for a Super huge company and I you know, I
26:44
called up sarah eaton. She's
26:46
uh, this wonderful woman. She's
26:47
got a phd for meals She's absolutely brilliant. I
26:49
was like, sarah. I got some great news. I
26:51
can be on your board and so uh, She was like, yeah,
26:53
so I said send over the paperwork in like 24 hours
26:55
later You know, I did and um, I've been having great
26:58
meetings with them today. We've
26:59
been You know mostly strategizing but anyway,
27:01
data x is really cool And it's a really nerdy thing
27:04
and I'll try and distill it the best way I can so
27:07
Data x is the distributed autonomous decision agent x
27:10
being you the person the thing the entity So it is
27:13
built upon this programming language called repeat And
27:17
so repeat started at stanford in 1996 and I
27:20
actually met the two professors that started it
27:22
And so repeat was then compiled for the first
27:24
time. I think they compiled it in
27:25
java in like 2003 and then it sat and did nothing for
27:30
20 some odd shares because it's meant for super
27:34
parallelization and um basically If you think
27:40
about like javascript being like single threaded where
27:41
you know, you could only do one process at a time
27:44
repeat as a programming language that in like I
27:47
want to say inverts the Classic model view
27:50
controller triangle, but it gets rid of your model and
27:53
your model goes into your controller language you
27:55
actually program your business logic into your
27:57
controller and so When you think about how programming
28:01
languages typically work It's a very different way
28:04
to think about how Programming works in
28:07
general. I mean me trying to wrap my
28:08
head around it as somebody who's Learned multiple
28:10
programming languages and I think I understand
28:12
understand syntax across a couple different This one
28:15
has been like next level for even me to understand
28:17
Which is why I think the adoption curve has been
28:19
pretty steep, but with all that said, I'm hoping that
28:22
in the next You know year I can make it more
28:25
accessible. I'm good at all the secret
28:26
plans we have But I do really believe in what
28:28
they're doing and I think they happen at the right
28:30
time given the need for a lot of autonomous systems
28:34
that we have right now with like a The agi need
28:37
across the horizon Yeah, talk to me more about
28:39
that. So then there's this wild
28:41
programming language and data x Leverages that
28:45
correct product. What what's the product that
28:48
they have? That's the autonomous Uh, I'm not
28:51
sure I fully understood. Yeah,
28:53
so dot x is their patented product that allows it to
28:56
be deployed on IOT devices and devices so it's it's
29:01
chipset based more than it is language based and so
29:05
um, give you an example when we talk about Say in
29:09
the defense world now use the the defense background
29:11
because I can speak to it For like class 2 uas
29:14
systems right now the war between russian ukraine
29:16
involves a lot of drones Small class 2 commercial
29:19
drones for the most part too. And
29:21
so right now they require a human pilot The ideas in
29:24
the future autonomous agents, uh, could be
29:28
Piloting these so they can kind of self pilot self
29:30
correct and in a war zone We have casualties things
29:32
getting shot down. They can complete their mission.
29:34
This could be for surveillance.
29:35
This can be for Understanding
29:37
sygant if you've got a sygant need There's a lot
29:40
of different ways that you can maximize coverage of
29:41
drones if they're making their own decisions
29:44
autonomously based upon the model built into their own
29:46
controller rather than You having to have a human
29:49
pilot making decisions about what happens next It's the
29:51
best process but it could also be a wonderful an
29:53
industry where you have to have a lot of different
29:55
sensors anywhere You've got a sensor Uh, it could be
29:58
useful Because you want to make sure you know things
30:00
are happening in sequence or if there's an issue on
30:02
one side that you can correct So that's where the
30:05
paralyzation comes in that's where it shines Is you've
30:08
got a ton of sensors and you need to be processing
30:11
all that data super fast. Yep
30:14
I see so I I think I think it was a language
30:19
and a product that was in need of The compute and
30:24
then also the infrastructure. I
30:25
mean in 1996 when they were doing this people were
30:27
connecting with 56k dial -up modems This is one of the
30:31
funny things that happens at academia. They
30:32
think about the hey, can we do this? Not
30:35
whether they should right the old drastic park thing
30:38
like could you versus should you and And I think
30:41
in 1996 you've got a bunch of stanford professors who
30:44
are really intelligent people And they're like
30:46
we're going to make our own programming language
30:47
that can do distributed autonomous decisions And
30:49
it's it's going to work on iot devices, which by the
30:52
way in 1996 barely existed All right, let's give it a
30:55
minute to thank our sponsors of this episode
30:57
weights and biases Elevate your machine learning skills
31:01
with weights and biases free courses including the
31:04
latest additions enterprise model management and llm
31:09
engineering Structured outputs choose your own
31:13
adventure with the wide offering of free courses
31:16
ranging from traditional mlops llms and CICD to
31:21
data management and cutting edge tools That's not all
31:26
you get to learn from industry experts like jason
31:29
lewy jonathan frank shreya shankar and more All those
31:34
people I will 100 % vouch for they are incredible
31:38
friends of the pod Enroll now to build better models
31:41
better and faster and better and faster and just
31:45
get your Education game on check the link in the
31:48
description to start your journey now Let's get back
31:52
into this and and so they were just doing it because
31:54
from an academic academic perspective it's probably
31:56
seemed interesting And now finally I think that it it
32:00
has a a market need whereas for the first 20
32:03
some odd years of existence it didn't That is so wild
32:07
think about the I wonder if they just had a strong
32:10
vision or it was just like That whole random thing
32:15
like what am I gonna work on? Well,
32:16
this seems like it could be cool You know I I
32:21
talked with both herstrell and jj. So
32:24
those are the two kind of progenitors Both the
32:26
stanford professors And if they hear this I'm gonna
32:31
make sure I'm saying this the next way you're met
32:33
somebody who's like really smart And you can tell
32:36
they're thinking super deep things and they think on a
32:38
level you don't but at the same time Uh, a lot of
32:42
academics kind of fall victim to this is they
32:44
They think so much about the theory of how you
32:47
might do it and what you might do and the getting
32:49
there that they don't think about the How might someone
32:52
use this? Right, and I think a lot of what
32:55
You know that's just the strength of academia with
32:57
with commercial sector, right? Where
32:59
the academics figure out some really strange use
33:02
cases that maybe don't even need to exist yet And then
33:05
they find a place to be later and that's really one
33:07
of the strengths of like academia is that You know
33:09
there's been plenty of medicines that are developed
33:11
that way that like, you know We're kind of
33:12
piggybacked on another different medicines, you
33:15
know Scientific need and then for 10 years there
33:18
was no anything behind it in poof. All
33:19
right By the way, then now we can regrow your hair or
33:21
why because we did it on mice 10 years ago So not
33:24
that you have the need but I do it's where I'm
33:26
wearing a hat Yeah, that is so funny man, and that
33:32
is it lays dormant until Somebody dusted off the
33:36
shelf they'd grab it and go like Oh, maybe we
33:41
should try and use this. All
33:42
right. Let's see if that can Scratch
33:45
our itch here. No huh, so So then I I want
33:50
to keep talking a little bit more about the idea of
33:53
like How It is working with the government and
33:59
getting something pushed through I think we've
34:01
mentioned beforehand how you When you do get something
34:05
pushed through you like become an advocate to try
34:07
and help others Understand that all right now we can
34:12
use this new tool. It's been like clearance. What
34:14
what was it called fed? Fed
34:16
Fed ramp. Yeah, that's the word. That's
34:19
the word of the day right there. So
34:20
it's been fed ramped No, I want to let everybody else
34:23
know that it's possible for you to use this in your
34:26
design documents Yeah, so well, I'll give you a good
34:28
example like in the past I want to say two years data
34:31
bricks was fed ramped right and so for a long time
34:34
data bricks is in the commercial space Everybody
34:36
knows what data bricks was and then data data bricks
34:38
gets fed ramped right and so, you know Databricks has
34:41
their entire federal sales arm now um, and so I was
34:44
working on a navy human resources project, which is
34:47
Think about what the human resources is but just for
34:50
you know, the navy And you know, we had the ability
34:52
to put data bricks into our stack for you know The
34:55
classic medallion architecture and then now
34:58
that I've got this ability to put it into the stack,
35:00
right? I have to go to you know,
35:04
our stakeholders or our gs -15s and our ses and say
35:07
Guess what we can now build application level
35:09
dashboards So you can see recruiting and accession
35:12
information and understand who the better recruits are
35:14
And we just pull the sales force information in and
35:16
here's how we're going to do it and they're like
35:17
What you know like so they don't really know what's
35:20
happening You're just you're speaking whatever to them
35:22
and so you have to really like say hey This is how
35:26
another commercial entity used it here So the
35:28
government can use it because usually they go if
35:31
you tell them a story about how it was used
35:32
before they're better But if I try and explain, you
35:35
know application level versus, you know Quote
35:37
-unquote single source of truth, etc. They're
35:38
lost But because once again, they might have a
35:41
degree in biology or mechanical engineering smart
35:43
people just not it people and so You know data
35:48
bricks is one of those big victories that I looked at
35:50
on on the navy human resources piece because We
35:52
had good leadership chat out to admiral cheeseman.
35:55
Uh, we had good leadership that
35:56
was able to Uh know that there was a need for it
36:00
And they were willing to listen to the experts and
36:02
then you know, I that was what was you know One of
36:05
my my happier moments because in that case on
36:08
n16 Admiral cheeseman, I don't think he had a
36:11
background in it But he was very open to new ideas
36:14
even though he didn't always understand exactly
36:16
what you know was happening And he didn't get into the
36:18
nuts and bolts of coding it He knew that you know,
36:21
the navy human resources had a specific need to
36:24
modernize And he was willing to say all right,
36:27
let's use elation. What's data governance? How
36:28
does this help? Fine do it, you know, like it
36:30
seems like the best best use case use commercial
36:33
tools in the navy space Get an ato for it.
36:36
Whatever it takes. I
36:37
want to be modernized and I worked on other projects
36:39
where I won't name Uh, where maybe their leadership
36:42
wasn't is ready to change and didn't understand it
36:45
and felt like They were uncomfortable with it
36:47
because it was something they didn't understand And
36:49
so in my experience the leadership in government,
36:52
um, that is most Capable of change Just trust the
36:57
professionals and understands that sometimes commercial
37:00
industry has good solutions that are working government
37:02
And they're willing to just even though they don't
37:04
understand it Let those products be used in a
37:07
government space and then those people shout out to
37:09
admiral cheeseman are the people that I think, you
37:12
know Allow for the best organizational change It's
37:16
very much like you are doing a bit of like dev
37:19
rel or evangelizing more developer evangelism or just
37:24
evangelizing the tool and having to show that Or
37:29
knowing the strategy that works is taking a parallel
37:33
Situation and saying here's how it was used on this
37:37
project or with this team And so if you think about
37:40
the possibilities we could do here Then we can
37:44
incorporate it in this way or that way exactly like
37:47
At least take an organization like walmart,
37:49
you know, my buddy, uh, you know ben works over
37:51
there so we kind of talk shop a lot and uh Walmart
37:54
you don't have to sell a product like data bricks to
37:56
their leadership, right? They're
37:58
gonna know and understand it. They've
37:59
got you know an e -commerce need they're selling stuff
38:02
and so Obviously we want the thing that's going to
38:05
maximize our profit and the government because sometimes
38:08
you don't have that profit motive You have to say
38:11
even though you've never used this product before
38:14
Uh, maybe data bricks can help us solve this
38:16
recruiting need or whatever it is I'll put better
38:18
understand this so we can get a dashboard out of
38:20
these or whatever it is because the government, um
38:24
Yeah, because the government's not profit
38:26
motivated and I think a lot of the things in the
38:27
commercial industry Are very much does it increase our
38:31
profit in a fortune, right? There's
38:33
a very linear if I use this this product Then we
38:36
will make more money or save time or whatever and
38:39
the government it is Or especially in the military.
38:42
It's how does it help me
38:43
complete my mission better? You
38:44
know, it's all it's a different type of language
38:46
So in the in the uh, say the n16 space their mission
38:50
in my navy hr Is going to be getting more officers in
38:53
retaining officers longer Making sure that it's
38:57
easier for officers to move their families making sure,
38:59
you know If you're relocating someone that
39:01
they're Uh going with their family in a timely manner
39:03
or you know, just like dealing with the the people
39:06
side of things But it's mission oriented. It's
39:08
not dollar oriented I don't know that sometimes hurts
39:10
people to hear because we're paying, you know the
39:12
taxes on it But it's they care about the mission
39:15
direction and so you have to change the way you
39:17
frame Um anything you want to deploy as mission versus
39:21
profit It's so funny you say that because it is so
39:26
Often that on this podcast we talk about ROI and you
39:32
have to Continuously be analyzing what the ROI of
39:36
anything you do is and so going into The military and
39:41
having it be more about less about ROI and more
39:44
about can this help me Get to get from a to b or
39:49
complete my mission as you're calling it Is a
39:53
totally foreign way of thinking about it And it's
39:56
it's kind of the the weird part is it's it's it's
39:59
sort of the same in many cases Especially like from
40:01
an HR perspective, right? Because
40:03
when we think about HR tools Retaining talent costs
40:06
money, right when you're if you're going to advise to
40:09
what you to adopt a work day for instance Uh
40:12
retaining people caught, you know is is retention is
40:14
something they think about a lot institutionally And
40:17
so the navy thinks the same way but for a
40:18
recruiting mission, right? They're
40:20
thinking broadly about every navy recruit. How
40:23
do we get more officers in? How
40:25
are we getting more people who just graduated with a
40:27
degree and um, you know biology or really tough to
40:31
fill positions? You know for navy doctors navy
40:33
nurses chaplains Navy surface warfare some of
40:38
these positions are inherently harder to fill
40:40
they're called billets And so it's it's tougher to
40:43
fill some of these billets. And
40:44
so when i'm going in i'm not saying hey, we're
40:48
Helping you save money and retention for recruiting an
40:50
ad budget, right? I'm saying it helps you
40:53
complete your mission because you're going to
40:55
have more officers that stay in for 20 years
40:57
versus leaving after six And so when you're saying
41:00
I would like to deploy Databricks because it's
41:03
going to do x y and z the real thing is I'm
41:05
deploying Databricks because it's going to help you
41:07
complete your mission by helping retention accession
41:10
Recruiting the the things that they actually care
41:12
about because that's the way that they're graded as
41:14
officers And so it just you just have to change
41:17
the way you talk about tech because it's not money
41:20
for them that the military is not a money making good
41:22
organization Yeah, that is fascinating. Uh,
41:25
there are places where It aligns and then other
41:30
places where potentially it doesn't align. So
41:32
you have to be very clear on What the strategy is
41:37
going into it and if the Juice is worth the squeeze.
41:42
I think is the other piece
41:43
like is it going to Be worth it for me to try
41:47
and champion this and we have to go through a very
41:50
rigorous process to actually get it through Or are we
41:54
good enough as it is like how much of a Bonus is it
41:59
going to get us to completing that mission?
42:02
It's true like I You know, I've
42:05
had so many Peers that I worked with especially when
42:09
I'm you know on the aforementioned in 16 Navy
42:10
human resources project where We were doing things
42:14
that we knew would be difficult to explain to the
42:16
clients like I'm sure you're familiar with elation
42:18
The the data governance tool. It's
42:19
fairly widely used Uh, you know, we had to get a lot
42:22
of our clients up to speed on what data governance was
42:25
and you know managing metadata and understanding
42:27
data lineage Uh, to people who don't live in that
42:30
world Like sometimes it's difficult to explain why I
42:33
need why do we need to pay the license for this?
42:35
We've never had it before like
42:37
these are new tools And so when you have an
42:39
organization that's existed without any kind of you
42:41
know data governance tools for forever Right and then
42:44
you say now we've got this new thing that you need to
42:46
pay money for you gotta Sometimes bring it down to
42:50
this very like third grade level like this is how
42:52
data governance works And you sit in the room with
42:55
once again smart people but in they don't live in the
42:57
nuts and bolts of it I think even for people like
42:59
you and me When I try and explain, you know Data
43:03
governance to people and and metadata management and
43:06
data lineage sometimes it can seem like a little
43:08
overkill even in my sense Right, but I know that I
43:11
need to do it because if I don't do it my life is
43:13
hell So like I I have to explain it that way just
43:17
trust that I'm doing this because if I don't I'm
43:19
gonna hate my life in three To five years and
43:21
and let's cut the way look at it That is another
43:24
piece that I think you called out is how you
43:27
don't get to have that feedback loop right away
43:30
Right you you have the feedback loop in years as
43:34
opposed to days or hours Which is really weird when
43:37
it comes to software. Yeah Yeah, I mean most of these
43:40
contract vehicles are going to be you know, any
43:42
between three and five years I love option periods
43:44
and you know and so You know you get these these
43:47
contracts and you've got to implement a lot of it
43:49
because usually the contracts will have a lot
43:51
of Performance, you know objectives and so you're
43:54
dealing with five year cycles. It's
43:55
here. Here's the front end development
43:56
We're going to do here's going to do sales force
43:58
and service now And you've got everything you're doing
44:00
and these are you know, sometimes 150 200 300 400
44:04
person Contracts are fulfilling. I
44:07
mean so this is a significant contracting force
44:09
on this organization that's designed to do modernization
44:11
And it it happens in really every part of our
44:14
government It's it's it's a really good time to be in
44:18
big data in the government because there are so many
44:21
pieces of legacy software I mean once again, you have
44:25
to be incredibly patient because some of this stuff
44:26
is old And it you know the the refactoring can be
44:31
tedious and annoying But if you're a patient person and
44:34
you're willing to undertake it, it's it's good work
44:39
And it's it's good teamwork like it teaches you a lot
44:42
because like I said before you're working with a set
44:44
of of constraints that aren't usually something you
44:48
would see in in a commercial area So you
44:52
probably got really good at putting things into
44:55
metaphors. Yeah Yeah, yeah, the the
44:58
metaphor game is strong here How did you I was
45:01
curious to know like how did you explain data
45:04
governance to me like explain it to me like I'm
45:06
five, right? What does that actually sound
45:09
like to somebody who is outside of the world? So
45:12
yeah, so when I'm trying to explain data governance,
45:15
I use um Uh like a hospital is like a
45:18
metaphor, right? Um and a different part of a
45:21
hospital You have different departments. You'll
45:24
have maternity. You're gonna have care for senior
45:26
citizens You might have an emergency department, right
45:29
at all parts of data in that hospital Are gonna
45:32
need some consistency to them, right? Because
45:34
let's say you're born in one department of the
45:36
hospital in maternity But assume you're going to go
45:38
to the same hospital your entire life There's certain
45:40
things that you're going to need to carry with you and
45:42
the reason you want things like data lineage and data
45:44
governance Is so I know that if I had a certain
45:47
allergy and I go from being born into pediatrics
45:50
that all my data is going to move with me That all
45:53
the the the correct tags are going to be there that
45:55
I'm going to know what the doctor was That's the
45:57
lineage of the doctor that treated me. I'm
45:59
going to know when I was there I need to know when
46:00
that data was changed and my weight changed. Okay,
46:02
that's data lineage, right? And
46:04
so then I can draw a picture of me the person
46:07
going through a healthcare system But you know that
46:10
data lineage could exist differently for a lot of
46:12
different organizations for the manufacturer of say an
46:15
f 35 That's going to be supplier lineage so you can
46:18
look for defects, right? If there's a part defect where
46:20
did it come from what supplier when when was it
46:23
made? So now if I've got an issue
46:26
where an f 35 is having a mechanical error And then
46:29
another f 35 as a mechanical error. I
46:32
can say hey wait a minute According to our parts
46:34
suppliers We can trace the data lineage back to know
46:37
that they had the same supplier. They
46:39
made the same date at the same time That's a
46:40
commonality, right? And so, you know, it's it's
46:43
understanding how Every every organization can have
46:47
an impact to it But understanding how data
46:49
changes when it changes who changed it What all the
46:52
rest users are allowed to make changes like hospitals,
46:54
you know doctors versus nurses some people can
46:56
change things Some people can't right so there's
46:59
levels of users And so I think if people think of
47:01
data metadata managed data governance and the way that
47:04
you might design the most efficient hospital data
47:08
infrastructure, that's the way that I try and explain
47:10
data governance to people Damn that is awesome. That
47:14
is a great way of looking at it and It paints the
47:18
picture for someone who is like, yeah, I've been to a
47:22
hospital I I understand that I would want when I
47:26
go to the hospital I want my doctor to know about my
47:29
peanut allergy Yeah, and I even want my doctor to
47:32
know if I go to a different hospital or a
47:35
different doctor You know, like that stuff should go
47:37
with me wherever I go And so in that case you want
47:41
that that data to live With the the different item
47:45
Have you asked chat gbt for like give me a
47:51
Metaphor about how I can explain this. Yeah,
47:53
I I'd say what's kind of interesting is I experiment
47:55
a lot with different LLMs I started messing around
47:58
with anthropic and Claude Claude for I think or
48:01
maybe it's Claude three or forget Um, and I like try
48:03
and see I do the same prompt to see what response
48:06
differences I get One thing that I do believe that
48:10
large language models fail at is applying analogies
48:14
and metaphors to people's lives because There's a
48:17
certain type of lived experience that a large
48:20
language model can't tokenize effectively because
48:22
once again They're just trying to predict the next
48:24
thing that sounds correct Whereas if you've lived,
48:28
you know, uh, It hit your life and you've been to a
48:31
doctor and been to a hospital You can say hey,
48:34
this is how a medical record is changing like I
48:36
get that it makes sense You know, I've I understand
48:38
data governance now like sort of you know, you've
48:40
don't know me wrong Elation is still a bear to learn
48:42
but like if you wanted to You could you could
48:47
understand the concepts and and I think that llm still
48:49
Fall dramatically short when I'm trying to say create
48:52
an analogy I encourage you because you know chat gbt
48:55
loves hallucinating Try and get chat gbt to chat gbt
48:59
to come up with an interesting analogy And it
49:01
won't it is terribly analogies Yeah, that's that
49:06
is a huge insight because of the non learned learned
49:12
experience or non lived experience And how it's
49:15
almost like the only way that it's going to know
49:17
about that is if there are enough training data samples
49:21
on the internet of analogies being created for
49:24
certain things and analogies are inherently so abstract
49:29
That it's hard to make them hold up And not and
49:34
you're towing that line between like hallucination
49:36
and quality analogy Yeah, abstraction is is inherently
49:42
Artistic in nature like you have to think in a way
49:45
that isn't directly about what you're talking about
49:47
And I do this a lot, you know when I'm talking about
49:49
my students about code, right when I'm trying to
49:51
like Explain what they're doing, you know in python
49:54
or sequel I have to give them another example that
49:58
they're using their real world to say oh Think
50:00
about this way because like to me and I always kind
50:02
of come back to this A coding is just giving a
50:04
computer direction step by step directions The people
50:06
who do the best at coding in my opinion or the most
50:08
efficient just understand how to give people
50:10
directions in the dumbest possible way you know, so
50:14
Um, you know, you remember you're you're giving a
50:16
logic board really what a computer is you're giving a
50:19
logic board Directions and it's got to be step by
50:21
step or maybe not step by step or you know Putting
50:24
directions in groups and then executing and hoping
50:26
that you're really really good at explaining
50:27
directions So it's just why marriage has always been
50:29
challenging for me when I'm navigating in the town with
50:32
my wife. I digress So let's talk a
50:36
little bit about your course because I think it's
50:38
super cool that you decided to create a course on data
50:42
engineering That you said, you know what instead of
50:45
making this a paid course. I'm
50:47
just going to make it donation basis I want
50:49
people to learn. I want to help basically
50:52
Bring everyone into the field that wants to be in
50:55
the field and not have any type of cost be a
50:59
hindrance Yeah, so one of the big things that I
51:02
tried to do was not create a barrier to entry because
51:06
I just I don't see it being useful and I
51:08
understand that there's people out there who sell
51:10
their courses But for me, that wasn't my interest My
51:14
interest was getting people through And not to be like
51:16
purely altruistic about it because I wanted to get
51:19
people through and get them to a point where they are
51:23
Junior level data engineer capable, you know, I can
51:26
get things through an etl pipeline And then maybe
51:29
deploy a tableau dashboard or something on stream list
51:32
something basic, right? But employable was the most
51:35
important part there Uh, the basics of get the
51:38
basics of docker sequel and python just getting there
51:41
And then I think there's a second piece of that is
51:43
having the confidence to then take the next step to
51:45
take those intermediate level courses Um, and so
51:48
in my course, I say, hey, this is uh, this is the
51:51
way I would learn step by step Um, I supply a lot
51:54
of other people courses on u to me and then I also
51:56
even tell the students and I actually learned this
51:58
after the fact And I'd update my course that a
52:00
lot of local libraries Uh with your library card you
52:03
get one of the premium u to me subscriptions And so
52:05
you don't even have to pay For u to me if you have
52:08
a library card in many cities and towns and so,
52:11
you know, you don't pay the 30 bucks About that.
52:13
It is literally as free as I
52:14
can make it And so I say, you know, here's brett
52:17
fischer's docker course Um, he's huge in the docker
52:20
community here in virginia. Uh,
52:22
and so here's here's here's what you're gonna need to
52:23
know And it's a little bit of coding a little bit of
52:25
dev ops, you know So you can containerize things
52:27
because that happens a lot and it gets you functional
52:30
and and about six months end as it was 180 days my
52:34
goal is to put two people through An interview like a
52:37
tech interview, which is a little different And I want
52:40
them to walk me through a capstone project and tell
52:42
the you know, tell me why you made the decisions you
52:44
made Show me your application You know, show
52:48
me something that you're an expert in and I even make
52:50
it open ended like If you're in sports make it
52:52
sports related if you're in a pokemon make it pokemon
52:54
related Find yourself something that you you know
52:57
a lot about you're a subject matter and whether
52:59
it be pokemon cards or wine And build me a
53:02
dashboard build me a tool show me something that's
53:05
interesting that you've made using your skills And if
53:09
you can do that, well, it's my belief if you can
53:11
do that you can go work in a corporate environment
53:13
because you can learn their specific need whether it's
53:16
shoes or makeup or Ring lights, whatever it is
53:22
Yeah, and then you Also have some stuff you did
53:26
mention that you aren't doing this purely out of
53:29
altruism, right? You get to do fun stuff and
53:32
engage with people that go through that that have the
53:35
persistence Yeah, so if you get through, um, you know,
53:38
I know some recruiters out there That are looking for
53:40
folks and they'll pay me Uh a recruiting fee to
53:44
find them talent and I think um, you know, the
53:46
other thing is when most recruiters are not highly
53:48
technical people They're not going to be able to
53:50
evaluate somebody from a technical perspective And so
53:52
a lot of it is me saying now I'll go on my
53:54
reputation if I'm willing to co -sign you and say
53:56
hey You walk through a coding interview with me
53:58
and I don't think you're going to embarrass yourself
54:00
If you end up, you know in a technical interview
54:02
with somebody at a a capital one or a A Nissan
54:07
wherever it may be, right? I
54:08
think if you wanted to do a junior level position and
54:11
those kind of like You know 80 to 90 grand a
54:13
year kind of salary rolls That's where I want you to
54:16
get now. It's beyond that it's up to you
54:18
to learn more Right, you're still gonna have to refine
54:20
your skills, but everybody's got to get a start
54:23
somewhere and I know that right now As much as
54:26
everybody talks about, you know, AI and machine
54:28
learning and the need for these next generation high
54:30
performance computing models You're gonna have to have
54:33
Immaculate data to put in these models and that means
54:35
that you're gonna need a lot of quality data
54:36
engineers And I think it's one of the pieces that
54:39
rather unfortunately No, no shade to Nvidia. Please
54:42
Nvidia be nice to me. But uh, you know, the folks in
54:45
Nvidia, you know, they're selling high performance
54:47
computing Nobody wants to talk about the nuts and
54:49
bolts of clean data And so I'm I'm here saying I
54:52
don't I'm not going to be the dude who develops those
54:54
advanced mathematical models and algorithms That's not
54:57
me. I'm not interested, but I'm
54:59
going to help you big data pipelines. That's
55:01
what I'm good at Um, that's I'll let somebody
55:03
else who's much better at math do the algorithm side
55:07
it's so not sexy to think about the ways that you
55:12
can do the uh data pipelines versus The next
55:16
new thing in the large language models or that
55:19
high performance computing Yeah, I'll give you the
55:22
analogy, right? If there was between the
55:24
people who build the high performance car engines,
55:26
right? Oh, I just build a thousand
55:27
horsepower Five liter supercharged Mustang engine.
55:30
That's Nvidia. I'm
55:31
the guy at the jiffy loop. I'm
55:32
doing the oil change every 5 000 miles You got to
55:35
come and see me no matter what, right? I'm
55:37
not itch I won't build the high performance computing
55:39
for you But you know, if you need your wiper blades
55:41
your place, I can get the oil done for you My love
55:44
cell you on the air filter for 75 bucks because I'm
55:46
greedy, you know, like but that's me That's that's my
55:50
business. But the thing is you're going
55:51
to get a lot more oil changes than you do
55:53
Thousand horsepower, you know drag drag strip freddy
55:57
engines. And so I'm betting on oil
55:59
changes not high performance engines Oh, I see what
56:02
you're saying. So because of this high
56:04
performance car, you're going to need to change
56:06
the oil a lot more And that's the same as When
56:11
you've got these high performance chips or you're
56:13
doing this high performance computing You really need
56:16
to take care that your data is on point. Yeah,
56:18
I mean every single person who's got I mean I got
56:20
this this theory that we're all going to have, you
56:22
know GPUs on the edge at some point, you know a lot
56:25
of you know, I mean obviously, you know apples
56:26
got there What are they called the the the the the
56:30
what are they called the chips they integrated chips?
56:31
No, anyway point is Oh, yeah,
56:33
they have, you know Machine learning capability on your
56:36
phone, right? And so the bigger piece here
56:39
is that if we're all going to have this capability in
56:41
our pocket to some point the very near future Where
56:43
they won't always be you know requiring these, you
56:45
know $8 ,000 Nvidia GPUs, right? I'm
56:48
sure that they'll get less expensive When they do
56:51
we're all going to have this capability and we're
56:53
all going to require immaculate data for all of
56:55
our interactions All the devices that are going to
56:57
be distributed I think this this five to ten year
57:00
window we're looking at it's going to change many
57:02
things in our lives And when that happens, you're
57:05
still going to require a metric ton of data
57:07
engineers and I cannot wait to meet that market need
57:11
Yes That is well said so the So apple well, I
57:23
wanted to mention too I can't remember for the life
57:27
of me the name of the Computing language that you
57:30
met you were talking about repeat debt That's it's
57:34
gonna be the future right there before we're gonna
57:37
have all these different million sensors people are
57:39
gonna be Writing in what is it? Rumpied
57:42
repeat RAP? IDE Rap rapid with an e Oh,
57:48
there you go. RAPied. It's like french rapid.
57:50
Yeah It's cool because it's
57:52
french Yeah I know Yes, that is so good. So
57:58
before we go talk to me about buffer ranch, man I
58:01
got to get into this because I can't see one of
58:05
your videos of you Hawking buffer ranch and not
58:08
mention it on the podcast. So
58:10
here's the interesting thing I work with a couple
58:13
different agencies on tiktok and one of the agencies
58:16
sent me like a gift box of Heinz buffer ranch and
58:21
it was supposed to be I believe for some of their
58:24
food influencers because there's a lot of you know
58:26
people who do food talk out there And so I was
58:28
stoked because I thought I was going to get sponsored
58:30
by craft Heinz and so I got this three pack of
58:33
Heinz buffer ranch and um, I Sit in the email back
58:37
to my agent and I was like, hey, I can't wait to
58:40
do the thing How do you want me to do this? She
58:41
was like, ah, I think you may have gotten that in
58:43
error Uh, I don't really actually want you to be
58:45
sponsored by craft Heinz buffer ranch I'm like, no,
58:48
no, I'm gonna get this sponsorship no matter what
58:50
I tell you what I'm gonna give them one on the house
58:52
And so I just made a stupid like a side and I
58:55
made some fried chicken and said this Heinz buffer
58:58
ranch appels appears well with this wine or whatever
59:00
I said and Then it became like a recurring bit
59:03
because I sent them an email and they're like
59:05
actually we're fine We don't really want to
59:06
sponsor you and so I've probably sent their
59:09
marketing department like a dozen emails saying hey, I
59:12
made another video For Heinz buffer ranch and here
59:15
it is and it's always something wildly
59:16
inappropriate Like it's not the kind of video that
59:19
craft Heinz would ever want to be associated with
59:21
they're gonna send you cease and desist letters
59:23
It's probably get in with the cnd. Yeah,
59:25
and so now my audience and including like people make
59:28
inside jokes If you look at any one of the comments
59:30
on my tiktok post there's never really a few comments
59:32
people saying I thought this is going to be a
59:34
Heinz buffer ranch joke because somehow at the end
59:38
of a lot of my videos I bring up this bottle of
59:40
Heinz buffer ranch and Actually, by the way, it's
59:43
it's actually pretty good Like I'm gonna try to look
59:45
plug Heinz buffer ranch but like I do eat it now at
59:48
my house because I bought so many damn bottles of
59:50
this stuff You know because like there was a once skin
59:52
where I had like several bottles behind me on the
59:54
bookshelf And so I started having to eat it because I
59:57
had like six bottles and I started liking it because I
1:00:00
started eating it So it is kind of good We gotta see
1:00:05
we gotta reach out to him see if they'll sponsor this
1:00:07
episode. Let's see if they'll get ml ops
1:00:10
The ml ops podcast brought to you by Heinz buffer
1:00:13
ranch It's like wait a minute again It goes back
1:00:17
to this whole idea of pattern interrupts man That
1:00:20
is the best thing that I think is the most enjoyable
1:00:23
part about Talking to you and hanging out with you
1:00:25
is just it's a constant stream of pattern interrupts
1:00:29
that I do not Normally get on daily basis. Yeah,
1:00:32
I I'm really good at jeopardy for what it's
1:00:35
worth like a I believe that random knowledge Bad
1:00:40
tracks 100 % So Heinz buffer ranch, it reminds me
1:00:44
of back in the day how when we were first starting
1:00:46
the community We had one of our virtual meetups
1:00:49
during covid It was like two three weeks into covid
1:00:52
and it was talking about the differences between
1:00:54
kubeflow and ml flow and The presenter who came to
1:01:00
talk about it. He was saying like, okay, this
1:01:04
may be a little bit cheesy I'm gonna use a cheese
1:01:07
analogy like, you know kubeflow is your parmesan
1:01:10
and ml flow is your I think he was just like
1:01:14
cheddar and cheddar you can throw on anything and then
1:01:17
parmesan is a little bit more It refined and it
1:01:20
takes a little bit more work to get that going and
1:01:22
so at the beginning of the I knew that he was going
1:01:26
to talk about that in his abstract because he sent it
1:01:29
over beforehand. So I was like this episode is
1:01:32
sponsored by cheese and it was just like threw up a
1:01:37
Generic slab of cheese there and it was like
1:01:39
we're about to go into cheese very deep on this
1:01:41
episode Except you kind of like took the buffer ranch
1:01:45
and you ran with it man. I
1:01:47
stopped with the cheese after the first one Yeah,
1:01:49
I I I don't think I'm ever gonna stop it at this
1:01:52
point Like now I actually know probably have to I'm
1:01:56
supposed to be make I'm gonna be making a video
1:01:58
Uh, I'm probably in the next 20 or 30 minutes, but
1:02:00
I need to make another chick talk And so now that
1:02:02
I've talked about it, I do think I need to somehow
1:02:05
throw in the the behind buffer ranch. I'm
1:02:07
probably gonna have to do You can I see a world
1:02:11
where you are creating diagrams data flow diagrams
1:02:16
and You're using the buffer ranch as the data flow and
1:02:21
it's going through like okay now it goes into the
1:02:23
little container and Boom here we go and then it
1:02:28
goes from the container it gets transformed So we can
1:02:32
explain dbt is it so your data build tools the fusion
1:02:35
of creamy ranch and tangy buffalo All right, we could
1:02:41
go on all day. This has been great man. I
1:02:43
really appreciate this shame and especially because You
1:02:46
did this for a second time It is very kind of you to
1:02:50
come on here and chat with me again I encourage anyone
1:02:54
that is out there that is looking to get into The
1:02:57
data engineering space or if you have someone a
1:03:00
friend because maybe you're not actually looking to
1:03:04
take Uh, one -on -one type courses on data engineering,
1:03:07
but you have a friend that has been pestering you
1:03:10
about how to code Send them over to Shane's
1:03:13
course. I appreciate the meet your
1:03:14
ass always wonderful talk to you man You