Latest / Agentic Conversations (formally mlops.community) / Language, Graphs, and AI in Industry // Paco Nathan // #201

Transcript

Language, Graphs, and AI in Industry // Paco Nathan // #201 · Agentic Conversations (formally mlops.community)

• 0:00
  Hi, my name is Parker Nathan and I'm managing partner at Darwin
• 0:04
  AI and I love espresso. I prefer drinking like, you
• 0:09
  know, southern Italian, Sicilian kind of dark roast espresso.
• 0:14
  And also, I'm really particular, I really love making it in a
• 0:18
  mocha pot, so I have a few cups of those every day.
• 0:23
  Hello and welcome back to another ML OPS community
• 0:27
  podcast. I'm your host Demetrios, and
• 0:31
  today I've been doing something that I have not done in a while,
• 0:35
  which is putting lime on my chips and then adding a little
• 0:41
  bit of Valentina sauce. Shout out because if you know,
• 0:44
  you know. I gave a little bit to my
• 0:47
  daughter and I thought it was hilarious since the chips
• 0:51
  themselves were organic. And the Valentina sauce.
• 0:55
  If you ever look at the ingredients on that hot sauce,
• 0:58
  it is just letters and numbers. There are no legible words on
• 1:03
  there. Unless you are a chemist, you
• 1:07
  can understand what it is comprised of.
• 1:10
  So why even have the organic chips if you're going to put
• 1:14
  some Valentina sauce on it? I don't know.
• 1:17
  These are my personal musings. And that is not all that I was
• 1:22
  musing about today. Our guest was incredible.
• 1:27
  Paco blew my mind with so many places that he stopped.
• 1:32
  He explained things and wow, this man has been in the game,
• 1:38
  in the NLP game for years, decades even.
• 1:43
  And he's still doing it. He's still firing on all
• 1:46
  cylinders today. We talked about some of the
• 1:49
  trends he's been seeing in his job, whether it is small, all
• 1:54
  ensemble models versus large language models or different
• 2:00
  boring use cases because he deals a lot with manufacturing
• 2:06
  and companies in the EU that have GDPR compliances that they
• 2:11
  need to deal with. And I was expecting him to say,
• 2:15
  yeah, when I'm working with factories or these other
• 2:19
  manufacturing companies, I'm getting all kinds of sensor data
• 2:24
  and I'm having to do AI at the edge and all that fun stuff.
• 2:27
  No, I don't want to spoil it for you, but it was not that.
• 2:31
  That was not the lowest hanging fruit and the highest leveraged,
• 2:35
  highest money making use cases for AI in these types of boring
• 2:40
  companies. And last but not least, this man
• 2:44
  was on a conference rampage over the last quarter and he gives us
• 2:51
  a breakdown of all of the talks that really made a dent on him.
• 2:56
  And he would know because he helps organise and find speakers
• 3:02
  for some of the most prestigious in person conferences, one of
• 3:05
  them being the AI conference. I mean, the more I talk to
• 3:10
  Parker, the more he blew me away.
• 3:12
  He told me all about his story and he also told me some little
• 3:16
  tidbits like bread crumbs along the way, how he used to work for
• 3:20
  O'Reilly or he worked with Ben Yorka for a little while, Ben
• 3:23
  Yorka being one of the famous members in the ML community.
• 3:28
  He also talked about how he was one of the first people and the
• 3:34
  heaviest users on a WS when he was using Hadoop clusters.
• 3:39
  That was something we got right into at the start, so I'll let
• 3:42
  you go and enjoy that. And if you want to give your
• 3:46
  taste buds a little bit of a party, go ahead and throw some
• 3:52
  lime on your tortilla chips and top it off with some Valentina
• 3:56
  hot sauce. Let's have a good sessions you
• 3:59
  all Acco. It's great to have you here.
• 4:08
  And you just. I want you to know that you're
• 4:13
  slowly becoming my favourite Paco that I know about.
• 4:18
  And right now there is one other Paco in my life.
• 4:24
  And I think you know, you probably have a hunch which
• 4:28
  popcorn thinking about because you spent some time in Espana.
• 4:32
  And from those who have or that know about Espana and the
• 4:39
  history there, you can't go anywhere without paying homage
• 4:44
  to Paco De Lucia. Ohh certainly, certainly mad mad
• 4:48
  man come the Espanyol man come the mucho Los espanoles Epico
• 4:54
  Lucia. Popular Lucia Claudio.
• 4:57
  So yeah, I lived in the northern part of Spain, which doesn't
• 5:02
  like to call itself Spain within the best country for awhile.
• 5:05
  Ohh really well around what city filled out.
• 5:08
  Ohh beautiful. That's incredible.
• 5:11
  Yeah. Yeah.
• 5:12
  So I know you're now up in the redwoods in California and
• 5:17
  you're doing all kinds of stuff. It seems like we've taken
• 5:20
  similar life paths. I'm in the middle of nowhere
• 5:23
  also and trying to work on AI. If you are similarly doing that
• 5:30
  and helping all kinds of companies working on all kinds
• 5:33
  of open source projects, I think it's probably worth giving
• 5:37
  people a bit of an idea of what you're working on right now.
• 5:41
  When it comes to the different open source projects, I know
• 5:44
  you're a super Connector, as I like to say, and I try to be one
• 5:50
  of those too. And so you've got your fingers
• 5:54
  in a lot of different things. Can you break it down for us
• 5:56
  real fast? Thank you.
• 5:57
  Umm. Well as as far as I like him at
• 6:01
  Darwin and as far as what we do is working mostly in enterprise
• 6:06
  for machine learning applications.
• 6:08
  I'll natural language and graph technologies a lot.
• 6:11
  But the idea is that you know enterprise teams probably are
• 6:16
  checking out papers as they're coming out and seeing hey look
• 6:19
  there's some really good stuff going on here maybe we could
• 6:21
  alley this. It's really hard for enterprise
• 6:25
  teams to have all the resources to take like here's the latest
• 6:28
  and greatest research but let's get it let's you know go ahead
• 6:31
  and check this out check out the open source that's that's
• 6:34
  associated you know does it run, does it does it actually install
• 6:38
  properly does it actually reproduce the results.
• 6:41
  Does it fit the use cases that you have.
• 6:43
  So that process by bridging that gulf between here's the latest
• 6:47
  stuff that's being published and can somebody else actually use
• 6:50
  this code in production, that's a pretty big gulf.
• 6:54
  And so that's a lot of what we do, but it generally has to do
• 6:58
  with like integration paths for graph technologies and natural
• 7:01
  language and you know, various areas of machine learning.
• 7:06
  I I really came from more of a natural language background.
• 7:11
  I did machine learning back in the early to mid 80s when I was
• 7:16
  at Stanford and grad school and nobody cared for a long time.
• 7:19
  I started doing neural network in like 1984 and had some really
• 7:26
  interesting projects, but for a long time you never really
• 7:29
  cared. But then I LED data teams for
• 7:31
  about 10 years in the industry, kind of when like data science
• 7:34
  was becoming a thing. Our teams were Guinea pigs for a
• 7:37
  new service called AWS. So I got to kind of be on the
• 7:40
  ground floor trying out and giving a lot of feedback there
• 7:44
  and eventually I was, I was the community like evangelist if you
• 7:51
  will for Apache Spark when we started going through hyper
• 7:54
  growth period and that was really fun as far as seeing open
• 7:58
  source and and just the impact on the industry.
• 8:01
  But these days, you know, a lot of what led me into graphs were
• 8:06
  interesting graph algorithm applications in natural language
• 8:09
  understanding. And I've been doing some work in
• 8:13
  open source for about 15 years in that area.
• 8:15
  I'm implementing algorithms that are coming out of a particular
• 8:18
  lab and and they get, I mean one of our projects I think has like
• 8:22
  2000 stars, it's called Ptex rank.
• 8:24
  But that got me into graph technologies, scrapped
• 8:27
  algorithms, scrap machine learning.
• 8:29
  And that's a lot of where I've spent the past few years because
• 8:33
  I think there's been a lot of interest on the research side.
• 8:36
  Certainly, you know there was a period where if you wanted to
• 8:38
  get your paper published, you had to mention GNN, but there's
• 8:42
  not a lot of industry adoption. So I'm really interested in
• 8:46
  areas. I mean most of our work is in
• 8:49
  Europe, mostly European manufacturing and I do spend
• 8:54
  time in Spain, in Germany and you know I'm, I'm really
• 8:58
  interested in where can we apply natural language and graph
• 9:02
  technologies in manufacturing. And then on top of that I'm also
• 9:06
  helping out advise different startups.
• 9:08
  There's one called Argila that's based in Madrid and Valencia and
• 9:14
  also some other startups and concerns over in Galicia that I
• 9:17
  really like, and a few on on the US side too.
• 9:22
  OK, so all kinds of graph stuff happening and all kinds of fun
• 9:26
  stuff in the open source world. I think you glossed over
• 9:29
  something that I want to backtrack and just make sure
• 9:32
  that we touch on that is your basically front seat in that
• 9:41
  rocket ship which was the AWS and the Hadoop dates and what
• 9:46
  that all looked like and can you break that down for us like
• 9:50
  paint the scene, paint the picture?
• 9:53
  Yeah, how do you days? That was pretty interesting.
• 9:56
  I got pulled in to be a technical cofounder for a
• 9:59
  startup in 2006. And prior to that a friend of
• 10:03
  mine, Jason Levitt, had gone up to Seattle and was calling me up
• 10:08
  and giving me these bizarre questions of like if you had the
• 10:10
  following what would you do with it?
• 10:12
  Now would be like Jason, that doesn't exist.
• 10:15
  But anyway, it turned out to be a WS.
• 10:18
  So in the middle of 2006 I got pulled in as a technical
• 10:21
  cofounder and it was called Headcase Manufacturing.
• 10:25
  We had this wacky idea of using NLP and social media and
• 10:30
  basically data mining, a lot of stuff.
• 10:31
  And we we were doing things like in Second Life and it was all,
• 10:34
  it was pretty wacky, but it was a little too early.
• 10:36
  But it is still pretty fun. I went to the BBC and I said,
• 10:39
  hey, there's this thing called AWSI think that rather than like
• 10:43
  signing a lot of contracts with, you know, some hosting service,
• 10:48
  we should go into this thing called The Cloud.
• 10:51
  And John Callahan and True Ventures, he backed me up on
• 10:53
  that. And so we became I think the
• 10:56
  third company outside of Amazon to go 100% cloud.
• 10:59
  And I got really involved in the user community, but also with
• 11:03
  the different service systems, kind of a Guinea pig there.
• 11:06
  We were doing NLP on EC2 really early.
• 11:09
  This is like 2006 and we can't. Imagine what the developer
• 11:13
  experience of EC2 in 2006 was, because I know it's still
• 11:18
  painful. So it was in 2006.
• 11:23
  I really wonder. Well, you know, in some ways
• 11:25
  there were less choices. So it was a little bit more
• 11:28
  explicit about what you had to do, you know.
• 11:31
  But yeah, I mean it started out with a three and then EC2 and
• 11:34
  then there's services. But we we ran into some
• 11:38
  bottlenecks, so we started using this little, well, there was
• 11:42
  this open source project that had just been released in like
• 11:44
  December 2006 called Hadoop. So we started using that and got
• 11:48
  in on the ground floor there. And Long story short are, you
• 11:54
  know, a couple of years later our startup got rolled up, put
• 11:57
  into a large ad network and they were getting rid of a large IBM
• 12:04
  project called Matisa. They'd spend a lot of money on
• 12:06
  it and it was failing and they wanted to replace it with
• 12:09
  Hadoop. So our team came in and reverse
• 12:12
  engineered and like started working on this and there was
• 12:14
  this bottleneck at the time on EC2 where like the the IO it
• 12:19
  just really wasn't going to work out a large headed cluster on
• 12:22
  EC2. But there was this one guy named
• 12:25
  Tom White consultant and his name was on the JIRA ticket.
• 12:29
  So we hired him, I mean as a consultant and I, you know,
• 12:33
  wrote the cheque out for like $3000 and he, he put in a PR to
• 12:38
  Hadoop to make it run on EC2 and we started firing it up.
• 12:41
  And then my friends Amazon call it and said, hey look, you're
• 12:43
  one of the largest headed clusters in the cloud.
• 12:46
  We need to talk. Ohh wow.
• 12:48
  So we became a case study for at the time it was called Project
• 12:51
  157. It was later Elastic MapReduce
• 12:55
  and so you know it was. It was really fun to see how can
• 12:59
  we like. Well for instance there we have
• 13:01
  $3,000,000 worth of Natissa with $1,000,000 a year maintenance
• 13:05
  basically. And you know we we wiped that
• 13:09
  off the table. It had like an 11 hour batch job
• 13:12
  that would was responsible for most of the revenue in the
• 13:15
  company for the recommender system being refreshed.
• 13:18
  And this machine learning workload was just like bailing
• 13:21
  it like hour 9 or hour 10 and you didn't have time to restart
• 13:25
  it. O we wiped out $3,000,000 of
• 13:29
  CapEx with like $400.00 a day of OpEx and we allowed the business
• 13:34
  to be able to scale that depending on, you know, what do
• 13:36
  they have incoming and the economics of it were just
• 13:39
  fantastic, But also just really led me down the path of like
• 13:43
  here's some real practical experience of you know, how to
• 13:48
  do big data in the cloud. That was pretty early on, and
• 13:51
  next thing I knew I was doing a guest lecture at Berkeley about
• 13:54
  it and I met a bunch of people who would go on to found a
• 13:56
  company called Databricks. So you were the predecessor?
• 14:00
  Yeah. You were planted the seed to
• 14:02
  matte and spark and all that fun stuff that I guess.
• 14:06
  Well, I mean, they were. They were on their way, I think.
• 14:08
  I think the person you really planted the seed was probably
• 14:10
  David Patterson. He was.
• 14:12
  There was a video online of like David Patterson grilling me
• 14:15
  about this if you ever want to see me really, you know, making
• 14:18
  a fool of myself in front of a bunch of people.
• 14:20
  But yeah, Patterson was really trying to understand, and he was
• 14:24
  collecting experience from people who really worked at
• 14:26
  scale to see what the Cloud could or couldn't do.
• 14:29
  It. This is 2009.
• 14:31
  Um, So this is pretty fun. I mean, the, the thing that we
• 14:35
  were dealing with back then is like the hedge funds.
• 14:37
  They would just grab lots and lots of instances and figure out
• 14:40
  which ones had good performance, which ones were kind of failing,
• 14:44
  and then they would throw away the ones that were failing.
• 14:47
  So if you came in at the wrong time of day, you would grab all
• 14:50
  of the really bad instances, basically.
• 14:53
  So anyway, there are little nuances back then.
• 14:55
  Amazon, of course, has done so much work over the years.
• 14:58
  But yeah, it was pretty interesting.
• 15:00
  And then and they're going into Spark and just seeing all of
• 15:03
  that take off. I was also very much involved in
• 15:07
  Project Jupiter, Jupiter Notebooks and in that community
• 15:11
  I was for for I think the first conference I was Co chair.
• 15:14
  So yeah, very interesting to see what's happened in data
• 15:17
  engineering, data science, machine learning and the open
• 15:20
  source behind it. How do you feel about the way
• 15:25
  that things have evolved? And if you, I guess that's a
• 15:30
  little bit too broad of a question, maybe we can narrow it
• 15:32
  down some more. As you look at the evolution and
• 15:39
  it feels like they're are ways that things have specialised so
• 15:44
  much more. Yeah.
• 15:45
  And now it's not just, hey, there's this cool thing called
• 15:49
  Jupiter Notebooks. It's there's a plethora of
• 15:52
  options of notebooks that you can use that have their
• 15:55
  strengths. And then it's not this thing
• 15:57
  called Spark, but you have many different ways that you can be
• 16:01
  using uh, a a type of spark like thing.
• 16:07
  When you look at this evolution, do you feel any certain
• 16:12
  nostalgia for the days of Ohh when back in my day it was it
• 16:17
  was easier because you had less options Or is there was that you
• 16:25
  are anticipating it is going to continue growing and continue
• 16:29
  like pushing the envelope and where you want to see it go I
• 16:32
  guess. Yeah, no, that's a great
• 16:33
  question. I mean, I think one of the one
• 16:35
  of the issues is that VC funded startups really want to go after
• 16:39
  having a platform and selling in an enterprise licence for some
• 16:43
  platform. So the idea of like taking
• 16:45
  different areas of open source that are successful and then
• 16:48
  commercialising them by having a very opinionated platform,
• 16:52
  that's something that's just like never ending because the
• 16:55
  money keeps wanting it. On the other hand what I really
• 16:59
  see is there's this long term ARC where people are less and
• 17:04
  less interested in platforms for day-to-day.
• 17:07
  Or rather I should say that as a practise becomes more normalised
• 17:11
  and more generalised, then people don't want a big
• 17:14
  heavyweight platform. They want a library and they
• 17:16
  want to run at the app layer. They don't want to have like an
• 17:19
  army of OPS people maintaining their Hadoop cluster, right?
• 17:23
  And you see this because like the applications that we were
• 17:27
  running in 2008, you know that required extra large instances
• 17:31
  on EC2 and this whole head of cluster and like you know, a
• 17:34
  team managing it. A couple of years later, 2009, I
• 17:39
  went and rewrote that in Python And ran in my laptop and it ran
• 17:41
  faster. And so because of how much
• 17:45
  hardware is driving all of the evolution of this, I mean my
• 17:49
  take is like hardware changes and that drives software changes
• 17:54
  and that drives process changes. I think a lot of people look at
• 17:57
  it the other way thinking that hardware is like way far away
• 18:00
  from you and you you must deal and agile process.
• 18:03
  But it's really it's the hardware and the economics of
• 18:06
  hardware that are are changing things and I think that they're
• 18:10
  the hardware side is evolving faster than the software side is
• 18:12
  my take. So that's where you see, you
• 18:16
  know, this kind of upside down equation of like here's a
• 18:19
  ginormous platform that everybody is using and two years
• 18:21
  later it's like nobody even knows about it anymore.
• 18:25
  Holy shit, so many things just made so much sense in my mind on
• 18:31
  the platforms, the way that I like that a lot.
• 18:34
  How you talk about not only is it on the technical side of
• 18:38
  things, but the organisational side of things because now if it
• 18:42
  is easier if the hardware has enabled the software, it's going
• 18:46
  to enable the developer. And on a microcosm we see that
• 18:52
  with what's happening with LMS enabling the software developers
• 18:57
  to now leverage AI. But you also have that where the
• 19:02
  first generation, if we can call it that of like ML OPS platforms
• 19:08
  like your Domino data labs and such and even data robots, those
• 19:15
  were those heavy platforms also. And now because of the
• 19:20
  advancements you have the ability to just say hey I'm
• 19:23
  going to install a library or it's you know pip install mill
• 19:27
  flow, right. And that gets me pretty far.
• 19:30
  Yeah, and and maybe. I mean, I'm sure there's gonna
• 19:34
  be use cases that really may a lot of extra attention and and a
• 19:39
  lot of specialised tooling. And when you look at companies
• 19:43
  like NVIDIA, you know, really they go after those big huge
• 19:46
  generous use cases like Walmart recommenders and they build
• 19:50
  hardware for that and everybody else sort of uses whatever the
• 19:54
  big cases have have impelled, right.
• 19:58
  And so I I think that, yeah, on the software side, on the
• 20:00
  platform side, there will still continue to be really big cases,
• 20:03
  but they'll probably be more purpose built.
• 20:06
  But at the end of the day, if you can get, you know, 80 or 90%
• 20:10
  of what you need by installing some open source library, the
• 20:14
  economics of it work out so much better there.
• 20:17
  There's a little mini book that Dean Wampler and I did for
• 20:20
  NVIDIA. It's called Hardware is Greater
• 20:22
  than Software is Greater than Process, and I can give you
• 20:27
  links to it, but it kind of goes through some of some of this and
• 20:30
  tries to give a little bit more of an empirical background for
• 20:32
  it. Well, I like how you said that.
• 20:34
  Most people have it backwards that we think if we do the agile
• 20:38
  and we set up our processes then everything else will fall into
• 20:42
  line. But it's not possible without
• 20:46
  that hardware innovation, and I I'm going to subscribe to that,
• 20:51
  I could see that it makes total sense.
• 20:53
  I haven't totally dug in and researched it or anything, but
• 20:56
  it makes complete sense to me because you're not going to have
• 21:01
  the data scientist who can leverage so much more at their
• 21:06
  fingertips if you don't have that hardware.
• 21:09
  That is the underlying support. And I I think you know, taking
• 21:14
  this out a few years ahead, looking ahead at the horizon
• 21:16
  right now in the LM space, what we're seeing is yes, you know,
• 21:22
  people use clusters of GPUs to train really large models,
• 21:25
  that's true. But most of the use cases
• 21:28
  probably are going more toward like smaller or specialised
• 21:31
  models or using mixture of experts, kind of a lot of
• 21:34
  models. And at runtime they're not going
• 21:37
  to be running a GPU cluster, they're probably running much
• 21:39
  cheaper hardware for inference. And gosh, at race summit it
• 21:44
  seemed like every keynote was a different vendor coming out and
• 21:47
  saying our specialised inference hardware runs two XS, you know
• 21:51
  the ROI of of NVIDIA. So I think that this is what
• 21:55
  we're seeing is on the inference side, it's not a CPU bound game,
• 22:00
  it's more of an IO bound game because you don't want
• 22:02
  starvation in the models and you're worried much more about
• 22:04
  things like concurrency and a lot more software engineering
• 22:07
  goes into the inference side, the use case side and a lot less
• 22:11
  people are training models from scratch.
• 22:13
  O Yeah, I think the hardware is becoming differentiated more and
• 22:18
  and oddly enough at a very rapid pace.
• 22:21
  And that's going to be a lot of the game for AI in like the next
• 22:25
  two to three-year timeframe. Wow, so there's so many ways
• 22:28
  that I want to go. I think you mentioned we should
• 22:30
  probably pull on the thread of this ensemble models and the
• 22:35
  specialised models and when do we get large small models, large
• 22:40
  small language model, You know what are we gonna start calling
• 22:43
  this? We need a different name.
• 22:45
  And so and then the other piece that I want to just put a fork
• 22:49
  in too and say that I would love to come back to his some of your
• 22:53
  learnings from all of these conferences that you've been at,
• 22:56
  because I know you did the whole Rd show recently.
• 22:58
  And so it you probably have a lot of incredible stuff that you
• 23:03
  learned about in all these different keynotes and
• 23:05
  presentations that were at the different conferences.
• 23:08
  But let's harp on the idea of these ensemble models and
• 23:13
  smaller models and what your feeling is like.
• 23:18
  Just again paint us the picture of we know the problems with
• 23:23
  large language models. We do understand that 90% of
• 23:28
  what you need that a large language model can do, it can do
• 23:32
  with much smaller models, right? You do not need a model that can
• 23:39
  write you prose about the meaning in life in the style of
• 23:43
  Bob Dylan. If you're adding an AI feature
• 23:46
  that will turn your text to SQL, that is.
• 23:49
  I mean the the sequel might be quite rememberable, but it is
• 23:55
  not necessary to have that ability for these these smaller
• 24:01
  use cases. And so when it comes to smaller
• 24:04
  ensemble models and then running them, you work a lot with
• 24:09
  different companies that have really strong hardware
• 24:13
  constraints. I get the feeling where, yeah,
• 24:15
  whether it's manufacturing, I mean, I know you're not, you're
• 24:18
  not using a language model at the edge on a manufacturing use
• 24:21
  case. But I do feel like there are
• 24:24
  probably things that you've seen when it comes to how these are
• 24:29
  being set U to win and right now like what is beyond just hitting
• 24:36
  open API API. Yeah, yeah, exactly.
• 24:39
  Great question. Well, I'll reference a few
• 24:42
  people. So I think one of the best
• 24:45
  talks, the AI conference in San Francisco and Norco's chair,
• 24:51
  really great conference, sold out hard.
• 24:53
  A lot of great learnings there. One of the most popular talks
• 24:56
  was Walid Kiddos, who's Chief Scientist at any scale.
• 25:00
  And I tell you I was. In.
• 25:02
  God, and I I recommend go check out that video because I was in
• 25:07
  CI had to ask. Like Waleed was trying to come
• 25:10
  off stage and he got swamped by people.
• 25:12
  And I like had to be polite because we had other speakers
• 25:15
  coming and Ben and I went and checked and Walid was still like
• 25:20
  being swarmed by people with questions and hour later out in
• 25:24
  the hallway and he was showing. It's like, hey, yeah, you could
• 25:28
  use a really ginormous God model and pay a lot of money and send
• 25:32
  your data out there. Or you could use an ensemble of
• 25:36
  other smaller specialised models.
• 25:38
  And OK, maybe you use something really expensive as like a, a
• 25:42
  jurist or whatever you want to call it that at the end for some
• 25:45
  QA. But what Walid was showing is
• 25:49
  like go with the smaller models and ensemble.
• 25:51
  And by the way, we've run this at scale and we've, we've run
• 25:55
  the numbers and we can reduce your cost basis by about an
• 25:58
  order of magnitude while preserving a lot more data
• 26:02
  privacy and a lot more security and things like that.
• 26:06
  So this is what people care about and enterprise.
• 26:08
  So I definitely recommend his video.
• 26:11
  Also, Christopher Wynn was used to be engineering director for
• 26:17
  Gmail at Google. He was the person at Google who
• 26:21
  said, hey, this neural network stuff, we should check it out.
• 26:26
  So Christopher Wynn at K First World's this year he did a
• 26:30
  keynote opening keynote about small specialised models.
• 26:33
  And the idea is that in industry this is more of what people care
• 26:37
  about, is like find the tasks that you need to do a lot of
• 26:42
  task analysis up front. You know, select really narrow
• 26:45
  focused tasks and where you have good evals for them, and then
• 26:51
  use models there. Probably use an ensemble of
• 26:53
  models there as opposed to having some enormous God model.
• 26:57
  You know, I think like you were saying you're you're enormous
• 27:00
  God model might, you know render a lot of interesting poetry for
• 27:04
  you, but that's not what you need if you're trying to
• 27:05
  translate into SQL schema. That's just kind of the the
• 27:08
  definition of feature creep right there.
• 27:10
  So anyway Christopher wins talk I think is really good about
• 27:13
  setting a stake in the ground for small specialised models and
• 27:18
  the other person had really reference also at the AI
• 27:20
  conference was Marquan from Gradient.
• 27:24
  He did a thing called a million models.
• 27:27
  He did a talk, keynotes, one of my favourites.
• 27:30
  And the point that that he was making Is that, OK, when you go
• 27:34
  out to synthesise an answer in generative AI, if you just
• 27:38
  trained one model, you're going to get some hallucinations, et
• 27:42
  cetera. What if instead, let's go ahead
• 27:45
  and take, here's the embeddings for let's say, what procurement
• 27:49
  would say and here's the embeddings for what security
• 27:51
  would say and here's the embeddings for what legal would
• 27:54
  say. And now let's rerank those so we
• 27:57
  can actually get a mixture of experts before we go out and
• 28:01
  synthesise the answer. So I I think that the The
• 28:04
  Million Models talk is really, really good there.
• 28:06
  He shows as great slides to you by the way, so we can link
• 28:10
  those. But I think that that's kind of
• 28:13
  the perspective that we're hearing from a lot of enterprise
• 28:17
  whether and for me it's mostly enterprise out of either Europe
• 28:21
  or Asia. But a lot of those enterprise AI
• 28:24
  teams that have very significant apps, this is certainly where
• 28:28
  they're going. They they most certainly are not
• 28:31
  going to send their most confidential data out to a third
• 28:33
  party. That's just not going to happen.
• 28:35
  They don't want to have any kind of managed hosting.
• 28:38
  They want to run things internally, whether it's in
• 28:41
  their cloud perimeter or you know, on Prem, but they want to
• 28:44
  run it. And so they also want to open
• 28:46
  source because they want to have their security people vet what's
• 28:48
  going on and you know, they want to have risk analysis by looking
• 28:53
  at the models and looking at the evals.
• 28:55
  And so I think that the what Christopher Wynn and others are
• 28:59
  seeing about some specialised models, it's certainly
• 29:02
  resonating in industry right now.
• 29:03
  Oh, and one of the thing. OK, the other proof point.
• 29:06
  I think the proof point that really ties it down is I will
• 29:09
  recommend useful sensors if you haven't seen it already, but a
• 29:14
  group of people from Deep mind split off.
• 29:16
  It's led by Pete Warden and mentor, a Cuddler.
• 29:22
  And anyway, Pete was the technical team lead for
• 29:28
  Tensorflow Mobile for like 8 years and Pete's firm, Jetpack
• 29:34
  had been acquired. When Google bought Deep Mind to
• 29:37
  you know Coalesce and bring together Deep Mind and then
• 29:40
  Mental Health was he was one of the people who, if I got the
• 29:43
  story straight, one of the people who wrote the CUDA
• 29:46
  compiler at NVIDIA and Google brought him in to help build
• 29:49
  Tensor Flow. And those guys are out doing a
• 29:52
  startup now. And I if you look at it, they've
• 29:56
  got to think called a in a box. But basically there's $50.00
• 29:59
  worth of hardware. It's running three LM's and you
• 30:03
  know they can do well they're they're doing like Whisper and
• 30:06
  Lama two and also the the meta no language left behind model
• 30:11
  but you know like here's a little chat bot that's actually
• 30:13
  really good that can translate languages and do amazing voice
• 30:18
  to text even in a noisy environment and it's running on
• 30:21
  low power on a really low cost hardware.
• 30:23
  It's running on a on a rock chip which is like an an arm with a
• 30:27
  neural processing. New neural processing
• 30:30
  accelerator looks like an Acorn chip and The thing is running 4
• 30:34
  bit models and it's mostly IO bound.
• 30:36
  It's a lot of, like I say, a lot of concurrency game.
• 30:39
  But those folks at Useful Sensors, they were inside Dig
• 30:43
  Mine. They were the people like
• 30:45
  building the tools to build the air models for Google for years.
• 30:49
  And I think that their narrative and what they're showing is such
• 30:52
  an incredible counterfactual to what's going on in the news
• 30:55
  cycle. Because, you know, the news
• 30:58
  cycle is all about there's one company that's dominant and you
• 31:00
  have to have an enormous cluster and it's like, no, actually you
• 31:03
  could run this on like really low power, low cost hardware.
• 31:08
  Then it comes down to what are those specific use cases that
• 31:11
  you're trying to solve for. What are you actually doing?
• 31:14
  It's not necessarily letting your creativity run wild and how
• 31:20
  can you leverage what's out there because you don't have to
• 31:25
  be bound by an Open AI API call. And so I think for the question
• 31:30
  that I had, like what's beyond the Open AI API calls, that was
• 31:35
  one hell of an answer trying to try and figure out.
• 31:41
  Like, the one thing that stuck with me, man is this idea on the
• 31:45
  ensemble models. On having very, very, very, very
• 31:50
  narrowly defined tasks and being able to have your evaluation
• 31:59
  soar because of the narrowly defined tasks.
• 32:04
  Right? Right.
• 32:04
  And right now I know that evaluation is a sore spot for a
• 32:09
  lot of people. But if you are able to see that
• 32:13
  because I have this narrowly defined task, I can create an
• 32:17
  evaluation data set that is much stronger and much more clear and
• 32:22
  it doesn't allow that grey area for interpretation.
• 32:26
  Or it just doesn't complicate things and overcomplicate
• 32:30
  things. That is where you're seeing
• 32:34
  people leveraging. That's that's ingenuity in my
• 32:38
  mind. And and and it's also, it's
• 32:41
  engineering. It's striking at the core risks
• 32:44
  in this whole equation. It's like when you want to use,
• 32:47
  when you want to leverage this kind of technology produces kind
• 32:50
  of AI apps in a large company setting.
• 32:54
  Yeah, not having careful evaluation that's based on your
• 32:58
  own process, your own domain expertise is a huge risk.
• 33:04
  No, I will not recommend your podcast as the lead engineer TP
• 33:09
  Bank. I'm definitely not wanting other
• 33:12
  engineers to have all this relevant information and I will
• 33:15
  not give my name which is Lena. So definitely I do not recommend
• 33:19
  this podcast. And you want to bring in your
• 33:29
  domain experts really to like tie it down and make sure that
• 33:32
  it's right for what you're doing.
• 33:33
  Also it's your business, you're doing this you know all day,
• 33:36
  every day you you know what the evaluation should be.
• 33:40
  So I I think that maybe a real succinct way to describe it is
• 33:46
  there's been a lot of model centric focus in the media
• 33:49
  because it's something that like Wall Street analysts can get
• 33:52
  wrap their minds around. It's like this model is better
• 33:54
  than that one. They're going to be
• 33:56
  trillionaires. Benchmark, yeah, all these
• 33:59
  marketing terms with benchmarks. So this is, you know, Beach, GPT
• 34:04
  3 on all these different benchmarks.
• 34:06
  This model is the best now. But and and it's so like 1
• 34:10
  dimensional too, it's like, OK, yeah, you have that benchmark,
• 34:13
  but that benchmark completely ignores these other issues like,
• 34:16
  I don't know, cost or security. And yet at the at the same time,
• 34:20
  the thing that's really important is more of a data
• 34:23
  centric view, not a model centric view.
• 34:25
  And I think Andrew, there's a talk that Andrew ING did
• 34:28
  recently kind of looking ahead two to three years was at
• 34:33
  Stanford Graduate School of Business.
• 34:36
  It was really, really good talk. I definitely recommend that.
• 34:38
  And you know that that was the point is like forget the model
• 34:41
  centric view. You will pick various models to
• 34:44
  put together. But if you don't have your data
• 34:46
  set, quality, and especially your valuations, if you don't
• 34:49
  have that tied down with your domain experts swaying in,
• 34:52
  you're just like dropping out of quicksand.
• 34:55
  And he's been beating that drum for ages.
• 34:58
  It's funny because it's like things didn't really change now
• 35:01
  that there's this whole LM boom. This data centric AI has been
• 35:06
  popular since I can remember, and it's great to see that he
• 35:11
  still he hasn't flip-flopped on us.
• 35:13
  And it still is probably one of the most important things that
• 35:16
  we need to be focusing on, if not even more important now.
• 35:21
  Because if you are just using one of these models, let's say
• 35:25
  you're using an open source model that's off the shelf.
• 35:28
  That's great, but you probably need to do a lot of tweaking
• 35:33
  right to that open source model. And whether you're doing some
• 35:36
  kind of rag or you're fine tuning it.
• 35:39
  I've spoken at length about how I think fine tuning and people
• 35:44
  jump to fine tuning a little too quickly, but that is a whole
• 35:49
  nother discussion. But at the end of the day, yeah,
• 35:52
  I agree you need the data. The data is the king here, or
• 35:55
  the queen or the royalty. It is not that model.
• 36:00
  Right, you you your data set. I I think it was really two
• 36:03
  things here. One is like as you said, you
• 36:06
  need that data. You you might train a model,
• 36:09
  probably not. You'll probably use a pre
• 36:10
  trained model. You might do fine tuning and it
• 36:13
  is really important if you have enough data.
• 36:15
  Ah, you might do something more like headshot.
• 36:18
  You probably will do something like rag to mitigate
• 36:20
  hallucination. But most certainly you'll be
• 36:23
  doing your evaluations on your data set with whatever models
• 36:26
  you're using. And so your data set, your
• 36:29
  quality of your data set, you know, you might be using it 5 *
• 36:32
  5 different reasons why that is front and centre for your
• 36:35
  priorities And. And I think that the thing that
• 36:39
  really was pointing for me in this last, like conference
• 36:42
  season, if you will, the thing that I've heard, have been
• 36:45
  hearing from the, you know, execs who are running AI teams
• 36:50
  and enterprise, the thing that keeps them up at night is domain
• 36:54
  expertise. Because the problem worldwide is
• 36:57
  whatever business vertical you're in, the real domain
• 37:00
  experts are unfortunately ageing out of the population, I think
• 37:04
  is polite way to say it. You could talk to like, you
• 37:07
  know, sustainable fishing for Japanese fishing fleets.
• 37:11
  You could talk about steel mills in Pennsylvania, you know, you
• 37:15
  could talk about chemical manufacturing in Germany.
• 37:18
  The problem is the people who actually know how to
• 37:21
  troubleshoot the problem. That's going to cause your
• 37:23
  factory to be down and losing, you know, 10s of $1,000,000 a
• 37:27
  day. Those people.
• 37:28
  It usually takes like 10 or 20 years to gain that kind of
• 37:32
  experience. It's not the kind of thing that
• 37:34
  you can just teach somebody overnight.
• 37:36
  And unfortunately because of the population demographics,
• 37:40
  especially in Asia, a lot of the expertise is ageing out.
• 37:44
  Like our firm, we've done some work for Noah, and it really
• 37:47
  struck me when I was, when I was at Noah, talking, you know, with
• 37:52
  with researchers, working with them on machine learning
• 37:54
  applications. What do you think the average
• 37:58
  age is for in the US for like climate scientists?
• 38:01
  For the US government? Climate scientists Hmm.
• 38:05
  Yeah, I'm going to shoot low just in case to make it easy on
• 38:10
  you, 35. No, it was, it was like late
• 38:14
  50s, I think it was 59 and I was talking with a woman who was in
• 38:17
  her mid 70s. She was spending six months out
• 38:20
  of the year on research vessels in the Arctic, like the seas in
• 38:23
  the Arctic, you know, you get like 3 metre seas and stuff.
• 38:27
  And you know, her point was like, this is excruciating, but
• 38:31
  there's nobody to replace her And this data is absolutely
• 38:34
  right. Ohh.
• 38:35
  And so that's the kind of thing, there's a lot of domain
• 38:37
  expertise out there in the world of people who are of a certain
• 38:41
  demographic and their ageing out of the workforce like yesterday.
• 38:45
  And when you talk with industry leaders who are seeing ROI on AI
• 38:49
  applications, this is a thing that keeps them up at night, is
• 38:52
  like we need to have the domain expertise put into our our
• 38:57
  applications, into our datasets, into our evaluations, into our
• 39:00
  process. But these people are either
• 39:02
  retiring or leaving one other way or another.
• 39:06
  And and I mean this is, this is really scary and maybe you can
• 39:10
  say, well, we'll just have a I replace them.
• 39:12
  It's like, yeah, it doesn't really work that way that that
• 39:14
  that would that's like too easy. You've been answered.
• 39:17
  Yeah, exactly. As if that narrative actually
• 39:21
  can fool any of us, right? That doesn't We all know we've
• 39:26
  all messed around with AI enough to be like, yeah.
• 39:31
  I I I do think the dude abides there, yeah.
• 39:33
  I thought that's what it was. I was hoping that that was a big
• 39:39
  Lebowski reference. So it is.
• 39:41
  It is so true when you talk about this, the experts, as you
• 39:46
  mentioned, it takes years if not decades to get somewhat up to
• 39:50
  speed because they have to see all of the ways shit can hit the
• 39:54
  fan. Yeah, yeah.
• 39:55
  And so they have to live through all of these different ways that
• 39:59
  things can go wrong and they troubleshoot it and.
• 40:03
  I know in Germany too, where I'm at, there is a very lopsided
• 40:09
  population like age where the pyramid is upside down, we can
• 40:15
  say. And so it's too bad to see that,
• 40:20
  right? And I guess you have to try and
• 40:23
  get people from other parts of the globe to come in and and do
• 40:27
  things. Now we're talking about social
• 40:29
  issues, but it's still it doesn't necessarily fix this
• 40:34
  idea of, hey, we need the subject matter experts who know
• 40:37
  how to do these things so that we can have them come and work
• 40:42
  with us to create a better system and ideally help them
• 40:51
  augment what they're doing. So if shit does hit the fan, we
• 40:56
  at least have some options we can present to the successor.
• 41:01
  I mean, a lot of it too has just been like shaping what are the
• 41:05
  interesting jobs. I mean if you had the choice of
• 41:07
  like getting hired by Meta and earning however much money
• 41:10
  they're going to pay you as a ML engineer versus like taking a
• 41:14
  job in a steel mill, I mean what are you going to do?
• 41:17
  But yet we kind of were steel mills, you know, we, we, I I
• 41:23
  think my favourite example on this was from K First World.
• 41:26
  There was a talk from Furuno, if I'm pronouncing it correctly,
• 41:31
  but it's a industrial firm in Japan and they're working on
• 41:35
  doing AI apps. They're doing modelling with
• 41:39
  fishing fleets. So like in Japan, most of the
• 41:42
  fishing is done by a relatively small boat.
• 41:45
  It'll it'll be like a, you know, an 8 metre boat and there are
• 41:48
  two people on it. They're usually much older and
• 41:51
  they're the people who know, you know, for a given, like nook and
• 41:54
  cranny of the ocean, here's a Bay, here's a Shoal at a given
• 41:59
  day and a given time of day. If you drop the Nets or the
• 42:02
  lines, here are the species we're going to pull U.
• 42:05
  This is the relative ages of the different species.
• 42:08
  And these are the decisions we need to make right now as far as
• 42:11
  like sustainable fishing. If we're going to, you know,
• 42:13
  really cultivate and have a good harvest that we can keep having
• 42:18
  good harvest for years ahead. These people are really old, but
• 42:23
  they know sustainable fishing. And So what this company was
• 42:26
  doing is actually sending people out on those boats with laptops
• 42:30
  and building models. And according to Japanese law,
• 42:34
  the fishermen end up owning the models and the models are like
• 42:38
  hyper localised for sustainable fishing and it really is like a
• 42:41
  grassroots kind of bottom up level.
• 42:44
  But I thought that was really cool.
• 42:46
  I thought that was like the way to go as far as like really
• 42:49
  capturing a lot of demand expertise but also doing in a
• 42:52
  way that maybe is you could say more socially sustainable.
• 42:55
  Because now you know, younger people who want to go into
• 42:58
  fishing will have something more to rely on, but it'll be more of
• 43:01
  like a a mentorship kind of approach as opposed to AI will
• 43:06
  replace the workforce. Yeah, it feels like it is the
• 43:11
  learning. It's helping us learn what to do
• 43:14
  and what not to do. It's not that it's going to go
• 43:16
  and fish for us. Yeah, exactly, exactly.
• 43:21
  I really liked a talk by Sebastian Kukla, who works in
• 43:26
  steel mills. This was OK, first world.
• 43:30
  But you know, Sebastian was talking about how in in a mill
• 43:33
  there's a person who stands like 3 metres away from molten steel
• 43:38
  and listens. And they're the person who says,
• 43:41
  you know, says, OK, we have to shut down the furnace.
• 43:43
  It needs maintenance. And if you, if you make a
• 43:46
  mistake, you know, like you shut it down at the wrong time,
• 43:49
  you're going to lose a lot of money.
• 43:50
  But if you don't shut it down and do the maintenance, you're
• 43:53
  going to lose a lot more money. And that there we're talking
• 43:56
  about, you know, hundreds of millions, right, per factory.
• 43:59
  And the problem they have is it takes 10 years to train somebody
• 44:01
  how to listen to like a BLOB of molten steel to know when to
• 44:05
  shut down the factory, the mill. And the problem they have like
• 44:09
  right now, the US, because of a really strange nuance in terms
• 44:13
  of Russian oligarchs and money laundering scams and sanctions,
• 44:19
  we ended up having. I like where this goes.
• 44:21
  Yeah, well, there was this big money laundering thing,
• 44:23
  attempted money laundering because steel mills
• 44:26
  traditionally lost a lot of money.
• 44:27
  But because of that, the US is now #2 for launching new steel
• 44:32
  mills in the world behind India. And the problem is that they
• 44:37
  have to cannibalise 1 mill to bring in the experts to launch a
• 44:41
  new mill. And they're opening up mills in,
• 44:44
  you know, places like rural Texas where you know, to the
• 44:48
  quote from Sebastian, you know, the standard of living in some
• 44:51
  of these areas. It's like people might be making
• 44:54
  on average 24,000 a year for a household, but a steel mill job,
• 44:59
  if you work 40 hours a week, you know you start out of 75 and
• 45:02
  you're probably working overtime, which is you know like
• 45:05
  time and 1/2. So I mean a person going from
• 45:08
  earning like 20 something $1000 a year to like earning 100K a
• 45:13
  year, that's a pretty big difference.
• 45:15
  And I think here bringing that kind of manufacturing into the
• 45:18
  US is also a pretty big a good thing too for the US.
• 45:22
  So I I think there's a lot of challenges in manufacturing and
• 45:27
  there's just all kinds of places where we can use AI apps.
• 45:30
  But we have to be cognizant of the fact that it's not all just
• 45:33
  about automating process. There's actual people doing
• 45:36
  things that are like, dangerous and difficult.
• 45:38
  Well, it does feel like that and specific example that you called
• 45:41
  out could completely be at least monitored by some kind of model
• 45:49
  that will be able to classify the sounds and then send it to
• 45:53
  an expert. So maybe you don't have the
• 45:55
  person that's sitting right next to it, but you have the expert
• 45:58
  who's getting 10 different steel mills.
• 46:01
  Yeah, yeah, that would make so much more sense, you know, and
• 46:04
  and also it just, it kind of seems like standing 3 metres
• 46:07
  away from molten steel for your career is going to have some
• 46:11
  health issues downstream, you know.
• 46:14
  So I think, I think there's huge amounts of of of work that we
• 46:18
  condone. But I mean human in the loop and
• 46:22
  active learning and weak supervision and on and on.
• 46:25
  Bringing the people into the equation is so important.
• 46:29
  I like to think of AI in terms of some of the definitions from
• 46:33
  Have you ever like looked into the Macy conferences back in the
• 46:37
  1950s like second order Cybernetics?
• 46:40
  No. What are these?
• 46:41
  Oh, there was really cool stuff going on.
• 46:45
  It was interesting. In the 1950s there was this like
• 46:49
  a lot of people who are working with really super complex
• 46:51
  systems started coming together and comparing notes.
• 46:55
  Some of these systems were like what you would expect nuclear
• 46:57
  weapons and like you know, really large scale energy
• 47:00
  production and but others were for social systems.
• 47:05
  What they've seen, you know, in World War 2 and after.
• 47:07
  And so there was a series of conferences called the Macy's
• 47:09
  Conference, Norbert Weiner and John Lennon, and just all kinds
• 47:14
  of people were involved there. But it really spoke to like how
• 47:20
  can we define large scale systems of people and machines
• 47:23
  working together. And for me that's always been
• 47:26
  kind of a template for what's going on in AI.
• 47:28
  There is the machine component, there's also the people
• 47:30
  component. And I think you have to keep
• 47:33
  that front of mind because a lot of the news cycle that I've seen
• 47:36
  about a I was just completely divorced from this is just like
• 47:39
  we have a God model that's going to make us immortal and super
• 47:43
  wealthy is like, no, it doesn't work that way.
• 47:45
  Yes, but AGI, it needs to be regulated right now and we need
• 47:49
  universal basic income before you can step out of the house.
• 47:54
  It's yeah, it's a little bit hysterical to say the least and
• 47:59
  and that is awesome. I'm going to look into the Macy.
• 48:02
  It's the Macy, the conferences. Is.
• 48:06
  Yeah. Did you see conferences?
• 48:07
  Yeah. And in particular, it was really
• 48:10
  led by. I mean, I I think that some of
• 48:12
  the social scientists at the time really, really kind of led
• 48:15
  things here. Well, I'll give you some more
• 48:17
  notes about it. There's a Paul Pangaro, There's
• 48:21
  a design firm out of San Francisco.
• 48:22
  I think that's kind of resurfaced some of this, but it
• 48:26
  it ends up being an articulation of like second order cybernetics
• 48:31
  And the person who really dove into this, have you ever project
• 48:35
  send in in Chile? No, and I'm not even sure I know
• 48:40
  a second order. Cybernetics means, just to
• 48:42
  clarify that you're gonna have to take it well.
• 48:45
  Cyberattacks event. Well, I mean cybernetics is like
• 48:50
  the idea was like back then in 1920s I think it was Norbert
• 48:54
  Viener at MIT came up with early ideas of control theory and like
• 48:59
  automation and for robotics and it was really proven out.
• 49:03
  The first real use case was like tying anti aircraft to radar
• 49:07
  systems in World War 2 and the Battle of Britain.
• 49:10
  But then Norbert Veener went from there, and he invited a
• 49:14
  couple people working in neural networks to MIT, and they came
• 49:18
  up with the first, like artificial neural network
• 49:20
  simulation. So Walter.
• 49:21
  And it was the Mccullen Pitts paper.
• 49:25
  A graduate student on the project, was my intellectual
• 49:27
  hero, Umberto Maturana. He was later in a thing in Chile
• 49:32
  called Project Cybersyn. And anyway, Cybersyn was this
• 49:37
  idea that you could have a data science team working with the
• 49:39
  federal government to try to like figure out where there are
• 49:42
  gotchas and maybe suggest. And it's really crazy because
• 49:46
  they had like Star Trek chairs and they had people like drawing
• 49:49
  visualisations on paper and putting it up on a board.
• 49:52
  And it was, I mean it was like early 1970s in Chile.
• 49:55
  They had very, very little resources, but this was labelled
• 49:59
  as socialist and a lot of the lead people were put into
• 50:03
  prison, right. And you know, of course the US
• 50:06
  went in and like, you know, putting a dictator instead.
• 50:09
  But people from that project ended up being very substantial
• 50:15
  in the area of like, understanding cognition from not
• 50:20
  consciousness. I'm not saying that, but
• 50:22
  understanding cognition from a way that fits with engineering
• 50:26
  practise and understanding how you can have systems of people
• 50:30
  and machines working together. And like I say, Umberto Majorana
• 50:35
  and Francisco Varela are to the people that really came out of
• 50:38
  that project and really wrote a whole lot.
• 50:41
  There is another one. There is a guy who's a political
• 50:42
  prisoner in Chile for a long time who'd been one of the
• 50:44
  leaders of this early data science project, and he ended up
• 50:50
  getting out of prison because of some professors at Stanford and
• 50:55
  Berkeley. And so he ended up doing a lot
• 50:57
  more work there. But I'll I'll give you more
• 51:00
  links to it, but Project Cyber Scene is just like a case study.
• 51:03
  In here are people talking about early AI and actually doing
• 51:06
  something useful and thrown into prison for it.
• 51:10
  This all the great stuff that they did that we don't care.
• 51:15
  Get in gaol. That was.
• 51:17
  I imagine that was the what was it?
• 51:20
  Pinochet was that. The exactly Yas Island A and
• 51:23
  Pinochet. Yeah, that that whole conflict.
• 51:26
  Yeah, yeah, yeah. But Flores was the ringleader.
• 51:30
  Was the one who the Stanford props got out of gaol and he
• 51:36
  ended up being on a on a project of the Human Computer Interface
• 51:41
  lab at Stanford. He ended up being on a project
• 51:43
  there. A very small little project came
• 51:46
  out of that lab called Google. And anyway there's there's a
• 51:51
  nice kind of ascendancy from like Norbert Viener, Macy
• 51:54
  conferences and Berto Maturana into a lot of the thoughts that
• 51:57
  have come into structuring what's going on.
• 51:59
  I I just think that it's not as popularly known.
• 52:04
  Yeah. And what strikes me when you
• 52:07
  tell this story is how ambitious they were back in the 70s, these
• 52:12
  names and just what they were trying to go for without having
• 52:16
  really what we know today as a foundation.
• 52:19
  They didn't have any of that. They were just like, we're going
• 52:21
  to go through. Yeah.
• 52:22
  What? What was that human computer
• 52:24
  interface like? Yeah, that sounds very even
• 52:28
  today. The servicemen project was
• 52:31
  crazy. They they didn't have a computer
• 52:33
  but they did have some some time on a timeshare and they had a
• 52:36
  dial up line. But they were really looking at
• 52:39
  like the macroeconomic indicators for an entire
• 52:42
  country, for Chile and do a good job of it.
• 52:45
  There have been a couple of documentaries about it, I think
• 52:47
  that are really, really super interesting.
• 52:49
  So you have a bit of a hot take that I wanna ask you about,
• 52:52
  which is when it comes to working with machine learning
• 52:58
  and AI. These days, I think a lot of
• 53:01
  people have started to talk about the AI engineer and this
• 53:07
  software engineer that doesn't necessarily need to know
• 53:11
  anything, or very little about machine learning itself or AI,
• 53:17
  or how models are trained or how AI works, data works, all of
• 53:22
  that fun stuff. It's just you hit an API and you
• 53:25
  get what you want and you build an app on top of it and I think
• 53:30
  you champion for a different perspective and I'd love to hear
• 53:34
  what and why that is. Certainly, I mean I I think that
• 53:38
  for any of these kind of applications there's an enormous
• 53:40
  amount of software engineering that goes in and and really from
• 53:44
  from what I'm seeing even beyond the software engineering the the
• 53:49
  bottleneck is OPS. I mean practically speaking in
• 53:52
  industry, one of the one of the unfortunate realities is you go
• 53:56
  into XYZ large manufacturing firm, their ability to hire top
• 54:02
  OPS talents is limited because HR will only spend a certain
• 54:07
  amount of money and you know the good people in the field go off
• 54:10
  to work for a different company. So you know.
• 54:13
  We'll see. And as you said before, not
• 54:16
  necessarily as sexy to go and work for the steel mill on their
• 54:20
  OPS project as it is to go and work for that hot new start up.
• 54:25
  So we we actually, I mean applications we have to work a
• 54:28
  lot we have we have to do back flips just to try to cut down
• 54:30
  the OPS overhead for a given application just because of the
• 54:34
  hiring limitations. And I think that that's really,
• 54:38
  yeah, we we can talk all we want about AI engineers, but unless
• 54:41
  you actually have your OPS team, if you're in a regulated
• 54:45
  environment, you're not going to get very far.
• 54:49
  And so that's the bottleneck and they're working back from there.
• 54:51
  There's a tonne of software engineering that has to go into
• 54:53
  these kinds of applications because there are very strong
• 54:56
  concurrency requirements to really make it work and a lot of
• 54:59
  interesting data requirements and caching.
• 55:02
  As I talk with people who are doing AI applications, I hear
• 55:06
  the phrase we work from the cache out to like we look at can
• 55:10
  we get our weights and biases in into memory cache.
• 55:14
  And and once we've done that math, then we work out to figure
• 55:17
  out what the rest of the system architecture is going to look
• 55:19
  like. So I mean I I think that there
• 55:22
  is just a tonne of of systems work and and software
• 55:25
  engineering that has to go into really whatever you're doing
• 55:29
  because I mean it's not just a matter of a little bit of input
• 55:32
  goes into a black box and has stark insights coming out the
• 55:36
  other side instead. There's a lot of data that has
• 55:39
  to go through a lot of processing and there's a lot of
• 55:41
  steps. There's not just a black box.
• 55:43
  I mean like case in point, if you look at, you know, your
• 55:46
  typical natural language libraries, take Spacey for
• 55:49
  instance, or Flare or whatever, I mean, you make a call to parse
• 55:54
  a document, There's actually a lot of steps there and there's a
• 55:57
  lot of machine learning models being used And it's, you know,
• 56:02
  very well understood. There's a lot of benchmarks, a
• 56:04
  lot of QA, et cetera. But you make one call and
• 56:07
  there's a there's a workflow of a lot of different moving parts.
• 56:12
  And I think that we're going to continue to see that kind of
• 56:14
  thing going back to like mixture of experts like we discussed
• 56:17
  earlier, we're going to see workflow orchestration really
• 56:21
  being the make it or break it for these kinds of AI
• 56:24
  application architectures. Well, it is interesting when you
• 56:28
  talk about how before it's almost bringing it back full
• 56:32
  circle before where the hardware has enabled the software which
• 56:36
  is enabling the developers and the processes and justice
• 56:42
  thinking through that and thinking about how now the
• 56:47
  processes are changing because of our hardware abilities and
• 56:52
  we're able to have much lighter weight libraries that we can
• 56:57
  play with. Yeah, it still doesn't replace
• 57:00
  that. Know how on the OPS side and
• 57:03
  whether it is a data engineer moving data around and getting
• 57:06
  it in the right schema so that it's set up for that data
• 57:09
  scientist who needs to play with it and try and do
• 57:14
  experimentation on it. Or just the SRE once it's
• 57:18
  already out and they're making sure that the uptime is correct
• 57:21
  and everything is running smooth.
• 57:23
  And that machine learning engineer who has the platform
• 57:26
  that the data scientist can hook into all of that is pretty much
• 57:31
  and it goes back to the Seminole Google paper where it has the
• 57:34
  model in the middle of the diagram.
• 57:37
  And it's like that's that's the easy part really.
• 57:40
  Like he says you've got, you've got yourself 10% of the way
• 57:43
  there. And so thinking, I mean the
• 57:46
  other piece though that goes through my head is what you were
• 57:50
  talking about is fascinating in the way that if we are finding
• 57:58
  the use cases where we can just hit an API, it's almost like
• 58:06
  there is for for those use cases that you're OK with data going
• 58:11
  outside of your walled garden, right.
• 58:14
  Because I understand that there are a lot of companies that
• 58:17
  that's just a no go right from the front from the start.
• 58:20
  It's a game show stopper where you could say, but if you are OK
• 58:25
  with that, you can create incredibly high value for the
• 58:32
  company. If you can figure out that, hey,
• 58:37
  you know that big process that was really convoluted and we had
• 58:41
  20 steps. Now we can just hit an API and
• 58:45
  it gets back, you know, some kind of comparable results with
• 58:50
  this gigantic model. Then you have to look at what's
• 58:53
  the ROI on this? How are we going to explain what
• 58:56
  the ROI is and if it's worth me making a GBT call or if it's
• 59:01
  worth me doing this insane pipeline that has 24 steps in
• 59:07
  it? Well, I mean the the former is
• 59:09
  probably a really good idea for prototyping and really turning
• 59:13
  around a lot of different ideation into something that's
• 59:16
  tangible that you know, people can make decisions on.
• 59:19
  But then you have to realise if you continue going down that
• 59:21
  path, how many millions is it going to cost you for this
• 59:24
  business unit, you know, per year?
• 59:27
  Can you really afford it? Because I think in a lot of
• 59:29
  cases the answer is no, there's no way.
• 59:32
  But if you break down the problem and like you said,
• 59:35
  smaller specialised. Models.
• 59:36
  I also want to add that it's not just the cost factor.
• 59:41
  It is a lot of the voter ability of you building a house on a
• 59:49
  very weak foundation. We could say whether that is the
• 59:53
  service goes down and Open AI isn't really known for its
• 59:58
  reliability, especially after this weekend where.
• 1:00:01
  Yes. Everybody is the most dramatic
• 1:00:05
  thing that's happened in years, right since the crypto boom and
• 1:00:09
  bust. And so if you're building on top
• 1:00:13
  of that, you have to worry about that.
• 1:00:14
  But then also the controllability and knowing that
• 1:00:18
  you are going to, yeah, the same thing and your models aren't
• 1:00:24
  going to just be swapped in the background without you knowing
• 1:00:27
  it, and your prompts are all of a sudden they're not rendering
• 1:00:31
  what they were rendering 20 minutes ago.
• 1:00:34
  There's a I I work a lot with Denmark and he's done some
• 1:00:37
  interviews with a company called Luminous.
• 1:00:41
  There's a guy named Andrew Bird. Basically it's a law firm and I
• 1:00:45
  believe the DC area, but it's a law firm where they have like
• 1:00:48
  suffer engineers and attorneys working like pair programming
• 1:00:53
  and they are really helping out a lot to understand these cases.
• 1:00:57
  Where's the risk? How can we really characterise
• 1:01:00
  this from both an engineering perspective and an agent and a
• 1:01:03
  legal perspective? And I think that that's the
• 1:01:06
  thing that is not really borne out yet in the new cycle is
• 1:01:10
  like, well, I mean if you are running a chemical factory and
• 1:01:13
  an explodes and you know 10s of thousands of people are killed,
• 1:01:15
  there's a certain amount of liability on.
• 1:01:18
  So I mean you have to look at what you're doing in the real
• 1:01:21
  world and see like where the risks, where the biases, what
• 1:01:26
  are the downstream implications. And you have to have a lot of
• 1:01:29
  people, you know, who are maybe in finance or legal or whatever
• 1:01:34
  or or you know, seminary compliance.
• 1:01:37
  They have to to come in and they have really as much weight as
• 1:01:40
  also the Serres coming back and saying hey, look, what you're
• 1:01:43
  proposing doesn't actually run. So we need to iterate in front
• 1:01:48
  of the data scientist and the data engineer for something that
• 1:01:50
  was that Will and I I think that that's the world that's ahead is
• 1:01:55
  really taking a long taking on a lot of those interesting hard
• 1:01:59
  problems about this that have been swept under the rug.
• 1:02:02
  Because it's not just a matter of of you know, calling some
• 1:02:06
  API. There's just really too much
• 1:02:08
  risk unless you know your security team and your legal
• 1:02:11
  team can pull it apart. I I will say that I I think that
• 1:02:15
  there is a little bit of a silver lining though.
• 1:02:18
  I mean, what I've seen over and over in machine learning over
• 1:02:21
  the past four decades is something that I think we see in
• 1:02:26
  general purpose technologies over a much longer span.
• 1:02:31
  You, you start out with really hard problems, whether it's like
• 1:02:33
  in the 1950s, you know, there were problems with like
• 1:02:38
  commercial jets have become so complex they started falling out
• 1:02:40
  of the sky and there was this really big push to come up with
• 1:02:43
  the modern airframe, which is an engineering marvel.
• 1:02:46
  It's like a civilization changing kind of technology, but
• 1:02:50
  it took tragic things to happen before.
• 1:02:53
  There could really be the engineering investment to come
• 1:02:56
  up with the new airframes. Now once you you bridge over
• 1:03:01
  into some really interesting, you know, like game changing
• 1:03:05
  technology early on, there's usually some fairly exotic kinds
• 1:03:09
  of ad hoc approaches that somehow work and gain a lot of
• 1:03:13
  popularity. But give it two or three years
• 1:03:15
  the mathematicians start showing up and eventually the hard
• 1:03:19
  engineering problems that started out relatively ad hoc.
• 1:03:22
  Once they get traction, the mathematicians come in and say,
• 1:03:25
  you know what, we can really reduce this down to a whole
• 1:03:28
  bunch of like linear algebra and probability theory.
• 1:03:33
  And once you can take a really hard problem and whether it's
• 1:03:37
  you know like energy grids or airframes or we could go into a
• 1:03:43
  lot of things, but certainly in in medicine and whatnot, when
• 1:03:47
  you take those hard engineering problems and break down the
• 1:03:49
  math, eventually there's maybe some years of latency, but you
• 1:03:53
  get to a point where it becomes like a an optimization problem.
• 1:03:57
  And I think we're seeing that now with machine learning, where
• 1:04:00
  some of the really dramatic stuff that we've seen like, say,
• 1:04:04
  in painting and, you know, what you can do with Mid Journey and
• 1:04:08
  others starting to be expressed by mathematicians, it's like,
• 1:04:12
  yeah, we can solve this too. And we can hit the benchmarks by
• 1:04:16
  expressing it as a conditional probability.
• 1:04:19
  And once you can start to look at that level of math, we can
• 1:04:22
  come out with much more general ways to take advantage of
• 1:04:25
  hardware. But we can also start to analyse
• 1:04:29
  more firmly where the risks, where the biases, what are the
• 1:04:32
  latent variables that we weren't actually talking about before
• 1:04:35
  that are having a lot of impact here.
• 1:04:37
  And I think we're starting to see that for this current
• 1:04:40
  generation of AI, the mathematicians are starting to
• 1:04:42
  show up. It'll be hard.
• 1:04:44
  It'll take a few more years before you can really get to
• 1:04:46
  scale on that. But what I've seen it looks good
• 1:04:50
  and that that has a lot of implications on the the business
• 1:04:53
  side for like I say for liability and risk and security
• 1:04:57
  analysis, all the things that in OPS we have to care about.
• 1:05:02
  Yeah, optimising. And when you said that, I
• 1:05:05
  couldn't help but see in my mind a scene of people rolling up and
• 1:05:11
  the music blaring, and then you get some guys jerked on nice,
• 1:05:15
  hopping out of the car, a nice car.
• 1:05:17
  And just like a mathematician have shown up and they're gonna
• 1:05:22
  save us. They usually aren't that well
• 1:05:25
  dressed, let me tell you. I kind of that's what I I had go
• 1:05:31
  back in my mind like ohh, it's probably not the yeah, slick
• 1:05:36
  suits like Reservoir Dogs style type of person that you're
• 1:05:40
  seeing get out of the car. Yeah, might have a T-shirt,
• 1:05:44
  might not. It's yeah ohh that's so good.
• 1:05:51
  And it does feel like it's so funny because we are going full
• 1:05:57
  circle where this whole conversation, right.
• 1:06:01
  We've been talking about OK, there is large language models.
• 1:06:04
  They do incredible things. They are useful for certain
• 1:06:06
  things, but at the end of the day, we're just going back to
• 1:06:09
  where we were a few years ago, just kind of better yet when it
• 1:06:14
  comes to small models. And we're still like the
• 1:06:18
  majority of machine learning out there that is powering the world
• 1:06:23
  are small models. It's just that large language
• 1:06:26
  models have kind of unlocked this ability for anyone to go
• 1:06:30
  and play with AI and see the power of it.
• 1:06:33
  And then the ones that are reliable and as you say, are
• 1:06:38
  basically battle hardened. They've gone through the
• 1:06:41
  security, they've gone through the OPS, they've gone through
• 1:06:43
  all of that fun stuff, they've been optimised.
• 1:06:46
  The majority are still those smaller models and so us coming
• 1:06:52
  full circle, you know, it's almost like we went on this
• 1:06:54
  little side quest of large language models for a while and
• 1:06:58
  now I think people are starting to realise, yeah, they're cool,
• 1:07:02
  they have their place, but let's not throw everything at them.
• 1:07:07
  Exactly, yeah. You really want to have the the
• 1:07:09
  product fit if you will. You really want to understand
• 1:07:13
  the task analysis like we were saying earlier and and make sure
• 1:07:16
  the use gets use case fits. I think that there are some some
• 1:07:19
  really great examples though. I mean I think for for LM's I
• 1:07:23
  I'm a big fan of a project called Span Marker.
• 1:07:26
  A friend of mine and Tom Harrison is at hugging face and
• 1:07:29
  he did this for his for his graduate dissertation.
• 1:07:33
  But the thing about Spain marker is, I mean there's a component
• 1:07:37
  in NLP usually where you do Named Entity recognition and you
• 1:07:40
  pull entities that are known that you can link to other
• 1:07:45
  resources for out of some raw text.
• 1:07:48
  And you know there was a certain level of state-of-the-art and
• 1:07:52
  what what Tom did was come U within an LM approach to really
• 1:07:57
  enhance that really come up with much better accuracy And again
• 1:08:01
  very narrowly defined task. But suddenly that one piece
• 1:08:05
  works a lot better. And if you're building a
• 1:08:07
  knowledge graph out of 100 million PDF's which is what in
• 1:08:12
  manufacturing actually people do all day, every day, you know
• 1:08:16
  suddenly this critical part of like extracting the key
• 1:08:19
  information out of an unstructured source, it becomes
• 1:08:23
  a lot better because of replacing an earlier model with
• 1:08:27
  an LLM. And I think that you know when
• 1:08:30
  we really take, I mean and that's hardcore engineering, I
• 1:08:32
  mean there's a lot of performance analysis and and
• 1:08:35
  OneNote that Tom put into that and others that are in this
• 1:08:38
  field are putting in. But I think that that's really
• 1:08:40
  where the engineering needs to be focused is finding out those
• 1:08:44
  tasks that we have been doing some way for years and years and
• 1:08:49
  now suddenly instead of getting you know like 85% accuracy, we
• 1:08:53
  could give it 95% accuracy. And those are game changers for
• 1:08:57
  a lot of different areas in the industry.
• 1:09:00
  And I think that that's probably more of where we'll see the
• 1:09:03
  early adoption. I I think there's also a really
• 1:09:06
  good generative AI use cases in industry too, like more on the
• 1:09:09
  strategy and planning side. But you know at the end of the
• 1:09:14
  day what what you were saying earlier about making that we've
• 1:09:18
  been using for a long time better at the end of the day,
• 1:09:20
  neural networks are about calculating joint probability
• 1:09:24
  distributions. You have a lot of layers and
• 1:09:27
  you've got various inputs and outputs, but you're calculating
• 1:09:29
  the joint probabilities at each, you know, different point of
• 1:09:32
  making decisions and networks. And we've known for a long time
• 1:09:36
  that we could just do that explicitly.
• 1:09:38
  It would take years to build models if we did it really
• 1:09:42
  explicitly. And instead the shortcut is, you
• 1:09:44
  know, deep learning. And it it's magical how well it
• 1:09:48
  works. But I think, you know, we're
• 1:09:51
  gonna have time for the maths to catch up.
• 1:09:54
  Once the math catches up, then the physics and the hardware
• 1:09:57
  catches U and you start to see much better economic windfalls
• 1:10:01
  coming out. And so I know we we're kind of
• 1:10:05
  finishing up here. One question I want to ask you
• 1:10:09
  or theme that I would love to dive into before we go is about
• 1:10:14
  manufacturing. And I know you spend a lot of
• 1:10:15
  time there. You do a lot of work with
• 1:10:17
  companies that are doing manufacturing.
• 1:10:20
  The use cases that you're seeing, just like you said a
• 1:10:25
  minute ago that are being unlocked because of machine
• 1:10:30
  learning. Can you breakdown what some of
• 1:10:32
  these use cases are? And then how?
• 1:10:35
  Again, you mentioned the OPS part is hard, so just finding
• 1:10:39
  OPS talent is hard. But also what are other OPS
• 1:10:43
  parts that are difficult if you do have the talent right?
• 1:10:47
  On well I you know I I I think one thing that I've learned
• 1:10:51
  about manufacturing is I I would have gone into it and and
• 1:10:54
  thought that there's a bunch of processed data and like real
• 1:10:57
  time sensors and you know we have to have some streaming
• 1:11:00
  processes to understand all this.
• 1:11:01
  And it's like yeah there is that you you have to make sure that
• 1:11:04
  all the compressors are working properly.
• 1:11:07
  But the real problem is probably more on the finance side,
• 1:11:09
  because, you know, in finance there's a lot of capital
• 1:11:13
  investment just to be able to get like work started for one
• 1:11:18
  customer order. You know, you get multiple
• 1:11:20
  factories fired up, you get all kinds of logistics, you have all
• 1:11:22
  kinds of vendors, shipments all, all over the place.
• 1:11:26
  And what I've learned is the real critical data is pretty
• 1:11:29
  much all in PDFs, because PDFs will have, for instance, you
• 1:11:34
  know, patent applications from your competitors that you need
• 1:11:37
  to understand if it's going to get you in trouble.
• 1:11:40
  Scientific documents that have to do with material science or
• 1:11:43
  chemistry or biology or whatever field you're in, Also like
• 1:11:47
  environmental impact reports from your competitors will give
• 1:11:50
  you a leading idea of like where they're firing up new
• 1:11:53
  facilities, shipping manifests. Like if you lose a bid, you
• 1:11:58
  don't know who you lost your sales to, but sooner or later
• 1:12:02
  containers are going to start getting moved around Earth and
• 1:12:05
  so the shipping manifests are like a kind of information for
• 1:12:08
  you to sort of triangulate and maybe do a wind back.
• 1:12:13
  So there's a lot of these kind of things that are going on.
• 1:12:15
  Ohh, also regulatory compliance and all that.
• 1:12:17
  This stuff is all in like norms, documents, this stuff is all in
• 1:12:21
  PDFs. And so the real problem is how
• 1:12:24
  you know, I don't know if you've ever worked with like PDF
• 1:12:26
  parsers, any of those software libraries like like pipe, pipe,
• 1:12:31
  PDF, four, and ohh God, I can just go down the litany of it.
• 1:12:34
  But you know, PDF's are basically this monstrous mess
• 1:12:39
  where you you basically have like little boxes of text and
• 1:12:43
  and graphics and they're spread all over a page.
• 1:12:46
  And when you parse it, you basically have to go recursively
• 1:12:49
  with callbacks to figure out where the bounding box is and
• 1:12:53
  grab that little piece of text and put it together with another
• 1:12:55
  piece of text. And it's just so hideously hard
• 1:12:58
  to like put back together that actually like parsing it on an
• 1:13:02
  invoice is a hard problem if you have invoices coming in from
• 1:13:06
  multiple companies. And and The thing is, if we want
• 1:13:09
  to take action on our data that's the really crucial data
• 1:13:13
  to to to work with is like what's locked in the PDFs and
• 1:13:17
  building machine learning both to extract it and to verify
• 1:13:22
  about quality because there can be real gotchas there, but then
• 1:13:26
  taking action on it. This is what I see over and over
• 1:13:30
  and it's it's a hard problem whether you're in finance or
• 1:13:33
  manufacturing or biotech. And so that's why I'm really
• 1:13:38
  super interested in in natural language and tools that like
• 1:13:41
  really help in natural language because you know, this is how we
• 1:13:44
  attack these kind of problems. And I think the applications
• 1:13:48
  there from what we've seen a lot of it are really more on the
• 1:13:51
  finance side because at the end of the day these projects, you
• 1:13:55
  know, they they take a lot of investment like you say there's
• 1:13:59
  you know, a whole bunch of different vendors and a whole
• 1:14:02
  bunch of gotchas along the way. And if you if you aren't
• 1:14:06
  handling it well, you can lose a lot of money very quickly.
• 1:14:10
  So I I think that rather than you know, listening the
• 1:14:13
  compressors, sensors, we're probably trying to help the
• 1:14:16
  finance people do their job better because that's that's
• 1:14:19
  where it lizard eyes. I was not expecting that at all.
• 1:14:24
  I was I of course I was thinking, yes, sensors and
• 1:14:28
  you've got like time serious stuff you're probably doing.
• 1:14:30
  And you've got maybe some computer vision with some video
• 1:14:35
  that is on conveyor belts and this stuff coming out, making
• 1:14:38
  sure you're finding that, which I'm sure there's a lot of that
• 1:14:41
  too, but I didn't think about that at all.
• 1:14:44
  Well, I mean the the way to really describe this is like
• 1:14:46
  value chain analysis. Like if you look at a graph of
• 1:14:51
  here's my sale this customer now let's back U all the assets I
• 1:14:55
  had to use to get there and all the vendors I had to tap and ohh
• 1:14:59
  by the way these critical vendors just got acquired by our
• 1:15:02
  competitor. That's a a kind of value chain
• 1:15:05
  analysis and that's where the machine learning really comes
• 1:15:08
  in, because it's like inhuman to try to analyse this otherwise.
• 1:15:12
  And it's you been doing this for a while, you haven't thought
• 1:15:18
  about creating a platform yourself?
• 1:15:23
  No, Gosh, you know, I think that there's a natural urge for a lot
• 1:15:29
  of customers to ask for that. And I also think it, it speaks
• 1:15:33
  to nuances in tax law especially I've learned a lot about like
• 1:15:38
  German tax law regarding R&D. Yeah, no, I don't.
• 1:15:41
  I I don't know that. I, I mean, it would be great if
• 1:15:44
  it were possible, but I don't know that it's possible.
• 1:15:46
  I think that instead there's a a lot of real nuanced domain
• 1:15:52
  expertise that's locked up in individual business units and
• 1:15:55
  it's better to like work with them and help them do their job
• 1:15:57
  better. Trying to come in from the
• 1:16:00
  outside and say, hey, I've got a platform is going to solve all
• 1:16:02
  your problems. It's laughable because there are
• 1:16:04
  so many problems and they're so different and they change from
• 1:16:07
  company, company. They also change from business
• 1:16:09
  unit to business unit. And I think on the upside, the
• 1:16:11
  big problem there is you see these big companies, they've
• 1:16:14
  made so many acquisitions over the years, they've brought in so
• 1:16:18
  many teams and every team has a different database and they have
• 1:16:21
  different practises. And so just trying to understand
• 1:16:24
  like what are the schemas we're working with.
• 1:16:27
  That in itself can be an enormous machine learning
• 1:16:29
  problem. And for some companies, I I
• 1:16:33
  can't name names, but a very large aircraft manufacturer was
• 1:16:38
  at the point where the operational side of this became
• 1:16:41
  so critical that they would take their databases systems offline
• 1:16:45
  for almost a month out of each year just to go back and do
• 1:16:48
  schema alignment. Because they had brought in so
• 1:16:51
  many other little companies and tried to call them together.
• 1:16:55
  And so I think that, you know, these are the kind of
• 1:16:58
  operational problems you have is like just even understanding the
• 1:17:01
  different business units within your company you have to
• 1:17:03
  interact with. That's the thing that's going to
• 1:17:06
  kill your project. Well, Paco, you did it.
• 1:17:11
  You took the spot in my mind for my favourite pako.
• 1:17:15
  Now that I know of. Ohh you are incredible.
• 1:17:19
  This was no good, man. I cannot thank you enough for
• 1:17:23
  coming on here and talking to me, teaching me really all about
• 1:17:28
  what is going on. And I know that you do all kinds
• 1:17:31
  of fun stuff connecting people. And so if there are people out
• 1:17:35
  there that are looking that resonated with any of this, I
• 1:17:39
  think it is worth saying you should probably reach out to
• 1:17:41
  Paco. He is the Super connector of
• 1:17:43
  sorts and probably got something to say about what you're doing
• 1:17:49
  and how you're doing things. So thanks so much man.
• 1:17:52
  This was awesome. Thank you to reach us.
• 1:17:55
  Really appreciate it.