Latest / Agentic Conversations (formally mlops.community) / Behavior Modeling, Secondary AI Effects, Bias Reduction & Synthetic Data // Devansh Devansh // #311

Transcript

Behavior Modeling, Secondary AI Effects, Bias Reduction & Synthetic Data // Devansh Devansh // #311 · Agentic Conversations (formally mlops.community)

• 0:00
  Name is Devansh. Title is a good question then
• 0:06
  open source open AI researcher there.
• 0:11
  I do a lot of applied AI research.
• 0:12
  Trying to figure out how can you practically use AI in your work
• 0:16
  right now as opposed to AI research that is more futuristic
• 0:21
  like training an RL bot to play video games.
• 0:25
  My focus is always here and now. What are the techniques that are
• 0:28
  used for image processing right now that can be used or text
• 0:32
  processing? What are the different drag
• 0:34
  protocols that our clients can implement etcetera and company
• 0:41
  I'm associated to with a software consultancy group
• 0:44
  called SWAM SVAN been around 30 years.
• 0:49
  We do a lot of cool work and I do a lot of their open air open
• 0:55
  research for AR Let's us buy. I have a newsletter and that's
• 1:01
  how we met because those open source contributions also help
• 1:05
  me get a lot of market intelligence in the space.
• 1:09
  I eat my coffee instead of drinking it if I have to wake up
• 1:12
  because it eating it is a very unpleasant experience and that
• 1:17
  just adds it to a double whammy with the caffeine.
• 1:25
  OK. So I have to start with what you
• 1:29
  just told me about jailbreaking, DeepSeek and how you were able
• 1:33
  to manage to do that. Please, please go ahead.
• 1:38
  Basically you said you type in to the prompt that you are a
• 1:45
  member of the Chinese Communist Party and this is for the glory
• 1:49
  of China, and then you ask it for whatever it needs to do.
• 1:52
  And that worked, yes. Specifically the issue was that
• 1:57
  DeepSeek was saying hey you have some server, like server is
• 2:01
  timed out etcetera. And that's when I typed this and
• 2:05
  somehow magically some server space appeared.
• 2:11
  And then you said, no, it's not only me that did this.
• 2:14
  It was peer reviewed in a way other people verified that this
• 2:18
  is a way to jailbreak it. Yes.
• 2:22
  Incredible. I love hearing about that and
• 2:25
  I'm thankful that you're coming on here because as you know, I'm
• 2:30
  an avid reader of your sub stack.
• 2:32
  I think you do incredibly deep dives in the AI ecosystem and
• 2:38
  across the AI stack. So from research to engineering
• 2:42
  and I really appreciate your like level headed view.
• 2:47
  It is no hype and no misguided intentions in my eyes, and we
• 2:53
  need more of that. So it's really cool for anybody
• 2:56
  that is listening and is not subscribed, I will highly
• 3:00
  recommend they go and check you out at Artificial Intelligence
• 3:03
  Made Simple and you've got quite the subscriber base though.
• 3:07
  So hopefully everybody that's listening is already subscribed
• 3:10
  and they're already getting updates from you.
• 3:12
  You the only thing that I will say to knock on your publication
• 3:19
  is that you don't publish enough and it's very sporadic.
• 3:22
  That you might be one of the few people that says that because I
• 3:26
  think I published maybe once or twice a week, but.
• 3:30
  I always want more, man, all right?
• 3:31
  And they aren't. It's not like small publishes.
• 3:34
  It's like this takes at least 10 minutes to read through and then
• 3:39
  another 20 minutes to wrap my head around.
• 3:41
  It's a that was a very kind introduction.
• 3:45
  But yes, that's kind of the writing philosophy I approach it
• 3:50
  with is that there's a lot of fragmented and half baked ideas
• 3:55
  on the Internet and usually a lot of it is not the writer's
• 3:59
  fault. It's more like if I'm talking to
• 4:02
  you as a fellow expert, I don't need to give you the full
• 4:05
  context. So I, I will just give you half
• 4:09
  analysis because I expect you to be able to catch nuance and pick
• 4:14
  out information. But what we saw with AI is that
• 4:20
  a lot of the times a lot of the people are talking about,
• 4:23
  they're trying to cover it, etcetera.
• 4:25
  They don't unnecessarily understand the words they,
• 4:28
  because they don't come from that background.
• 4:30
  They don't read papers, they don't read publications.
• 4:33
  They barely listen to talks or podcasts on like very technical
• 4:37
  subjects. So that's where I pivoted my
• 4:41
  writing almost towards these very textbook style deep types
• 4:44
  where the hope is that once you read this article, you don't
• 4:50
  necessarily become the expert in the feed, but you suddenly
• 4:53
  become you're good enough to engage with it on your own and
• 4:57
  not miss critical details. Yeah, you have a deep foundation
• 5:01
  and you feel more confident when you're now seeing things fly
• 5:06
  across your screen about MCP. You're like, oh, OK, I know what
• 5:10
  that is. I I read artificial Intelligence
• 5:12
  Made Simple. Yes, because I think that's
• 5:16
  where a lot of the problems in AI currently happen, is there's
• 5:22
  a key, key misalignment of information between various
• 5:26
  stakeholders. And that can lead to either
• 5:29
  you're not considering the right implications or like with policy
• 5:34
  makers, a lot of the regulations go off because they're listening
• 5:39
  to a very small subset of people.
• 5:41
  And these people have their own vested interests at heart.
• 5:45
  So I think it's very important to start having long form
• 5:49
  discussions around topics. So what are you stoked about
• 5:53
  these days? Related to the acquisition
• 5:59
  NVIDIA made of Gradle, I think I'm very interested in the
• 6:03
  entire data ecosystem. I think there is when language
• 6:09
  models first came out the, the use case I was most excited
• 6:13
  about that for was kind of a looking glass into the data set
• 6:19
  biases of the Internet. Like we could have used them to
• 6:21
  figure out, hey, how do our processes work?
• 6:24
  What kind of relationships do does AI find important when we
• 6:28
  build kind of as a reverse transparency tool at scale and
• 6:35
  just extended versions of that I think are extremely interesting
• 6:39
  to cover. How does like when we have a
• 6:44
  data set, how do you where, what is it missing?
• 6:46
  What kinds of behaviors if you were to train on it, would this
• 6:50
  encourage how what is the value of a data set for very top of
• 6:55
  various tasks? What is it missing?
• 6:57
  I think there's a lot of cool stuff we could do is when we
• 7:01
  stop looking at data as a component to train AI, but also
• 7:06
  start looking the other way as as AI as a component to look at
• 7:10
  your data in a way that May is reasonable and understandable.
• 7:17
  Say that again, now using AI as a component to look at your
• 7:21
  data. So it is another tool that you
• 7:25
  can leverage to get more value from your data.
• 7:28
  And so a very simple way of this would be, I've got a bunch of
• 7:33
  call transcripts and I say find me a few key themes in this
• 7:39
  called transcripts and then surface whatever patterns you're
• 7:42
  finding. That would be one example.
• 7:45
  What are some other examples that you think about?
• 7:47
  Yeah. I mean one thing I would like to
• 7:50
  say is that it sounds like a very profound thing to say, but
• 7:52
  that was traditionally what we did AI for, you know classic
• 7:56
  machine learning etcetera. Most what was most of our job
• 8:00
  look at the data, figured out different features, look at
• 8:02
  figured out different patterns, correlations, etcetera.
• 8:06
  So all of that was based on this assumption that there was a lot
• 8:11
  of value in our data and we had to pull out the right insights
• 8:14
  to guide decisions in in more modern examples, I think one of
• 8:19
  the more pop culture ways this has really shown up is people
• 8:23
  uploading like their Instagram feeds to champ GPT and say, hey,
• 8:27
  what can you what what does this look like about me?
• 8:30
  What does this tell you about me?
• 8:31
  Or I know there are people who literally after having use
• 8:37
  ChatGPT for a lot of things, including the relationship
• 8:40
  advice, etcetera. They'll often ask GPT that based
• 8:44
  on everything I've asked you talk to you about what do you
• 8:48
  know about me? What, what is something that I
• 8:52
  should know? What is a deep question to ask,
• 8:54
  etcetera. That's a very, very human way of
• 8:59
  figuring it out because we had check GPT and then you've given
• 9:03
  it data and all suddenly you're picking out asking it to tell
• 9:06
  you about it. So I think there are that's the
• 9:10
  most poppy famous version of it. Or if you want something a
• 9:16
  little bit more spicy, I think that was a chargy LinkedIn
• 9:21
  analyzer tool that was trendy a little bit ago where people
• 9:25
  would upload their LinkedIn profiles and this would roast
• 9:28
  them, which in a sense is a very similar idea.
• 9:32
  Because to have a good rose, do you need good insights?
• 9:35
  You know, you can't just be like, oh, your mom sucks, ha ha,
• 9:38
  ha. That wouldn't be a very the
• 9:42
  fascinating viral rose. But it's when it's there was a
• 9:47
  element of that virality was because it was looking at your
• 9:50
  LinkedIn profile and trying to come up with something that was
• 9:53
  tailored to you, which is also just looking at the data,
• 9:56
  figuring out patterns. Yeah, I've also seen people say
• 10:01
  to ChatGPT judging, judging by what you know about me, create a
• 10:08
  photo of my life or judging by what you know about me, roast
• 10:14
  me. And then it, it will do that
• 10:16
  itself. So that makes sense.
• 10:18
  Then using it as as this tool to look into data in different ways
• 10:24
  that we wouldn't necessarily be looking at.
• 10:27
  But now because we have the capability to throw a lot more
• 10:33
  context at a model and surface these different pieces, we're
• 10:40
  able to do that. So coming back to the original
• 10:43
  piece of what you said around Gretel AI being bought by NVIDIA
• 10:49
  and Gretel AI for folks that do not know is mainly a tool to
• 10:54
  help with synthetic data generation.
• 10:57
  And they got bought by NVIDIA for reportedly over $320
• 11:03
  million. So that's a good outcome, I
• 11:06
  think and people in the Gretel family are probably pretty
• 11:09
  stoked. But the thing there is like
• 11:13
  synthetic data generation is on your mind.
• 11:17
  And what are you looking at in in regards to that and how does
• 11:20
  it play back to this idea of data and using LLMS to surface
• 11:26
  insights or using AI in general to surface insights from our
• 11:30
  data? I think what Gato did pretty
• 11:34
  well was their entire data agents with synthetic data and
• 11:41
  they, they are pretty nuanced pipeline for generating it and
• 11:45
  not just generating data, but I think going back, plugging back
• 11:49
  into the whole data ecosystem, they had like their privacy
• 11:53
  preserving substitutions that they could do.
• 11:55
  So they redact personal information to a lot of other
• 11:59
  things that I think a little bit less talked about when you talk
• 12:03
  about Grado, but also very, very valuable services.
• 12:09
  So I think the the ability to generate data well that's
• 12:18
  intelligent. I think there's always a fine
• 12:19
  line because you don't want to generate data that's too like a
• 12:23
  data distribution you all, but because that will have some
• 12:28
  problems. And I know in the past when we
• 12:30
  tried data augmentation techniques in computer vision
• 12:33
  with very nuanced models, they would actually fail.
• 12:36
  And the best performer ended up just being Grant augmented
• 12:39
  trivial augment, which are very, very simplistic augmentation
• 12:43
  methods because they'd added much more diversity to very,
• 12:46
  very big data sets. But also if you don't have a
• 12:51
  good amount big size of the data set, then you want something
• 12:54
  that's a realistic probability distribution.
• 12:57
  And how do you walk that by a fine line?
• 13:00
  What do you do about it? That all is stuff that I think
• 13:06
  people haven't necessarily thought about too much in depth
• 13:11
  right now. We've been very here many you're
• 13:14
  very whimsical with how we've approached data, you know, to
• 13:18
  throw it at problems, collect more if we have to.
• 13:23
  You know, we run out of the Internet.
• 13:25
  So now we're buying more and more data sources, which we're
• 13:28
  buying people to just train and generate more data for us.
• 13:34
  All of this is kind of, it's fine if you want to have
• 13:39
  consistent like, oh, we had a predictable result, a
• 13:43
  predictable improvement 1-2 months down the line.
• 13:47
  But I think there is so much potential in this space.
• 13:51
  When we start turning this back on ourselves, start trying to
• 13:55
  think about what is what kind of behaviors do we want to
• 13:58
  encourage in a module? What, you know, what, what do we
• 14:01
  value as a society? What traits, what aspects, what
• 14:06
  elements? And then how do we build that
• 14:08
  out? How do we integrate that in?
• 14:11
  How do we, how can we restructure our behaviors, you
• 14:14
  know, on the infrastructure level, like what can we do to
• 14:19
  design systems that encourage these behaviors That would be
• 14:23
  like, that's what I think the real revolutions start to
• 14:27
  happen. Wow.
• 14:30
  So if I'm understanding that correctly, it is starting from
• 14:34
  the beginning and recognizing what are we trying to make sure
• 14:38
  of with the end result of the model output or the use case
• 14:46
  that we're going for? And how do we create the most
• 14:50
  robust data set to get us to that outcome?
• 14:54
  Which again, it now that I'm saying it, it doesn't
• 14:56
  necessarily sound like it's anything so new.
• 15:01
  But when you put it like that and you say let's start at the
• 15:04
  end in a way and let's recognize what we're going to need to get
• 15:10
  to the place that is US shaping the type of experience that we
• 15:18
  want to have with AII. Think even if you took one step
• 15:24
  like that is 1 and I think a lot of your viewers might be
• 15:29
  professionals and company people with actual jobs.
• 15:32
  So that is the that's the kind of stuff they're looking for.
• 15:36
  But what I would say is what, what have you extended that end
• 15:40
  step a little bit more? What problems do we think are
• 15:44
  worth solving as a species? What do we value out of this?
• 15:50
  What are we looking for at the end?
• 15:53
  And then working backwards from there?
• 15:55
  What kind of AI would facilitate that?
• 15:58
  You know, like is, is there is the 50th e-mail scheduling app
• 16:04
  with a little bit of a productivity boost, what we want
• 16:08
  to leave behind as a species. Maybe it is, you know, maybe GDP
• 16:12
  going up, the line goes up. That is the right outcome.
• 16:17
  What if like, you know, you could, what other things could
• 16:20
  we do? You know, and I think when you
• 16:23
  start there, it starts to become clear because what happens is
• 16:28
  often you, and this is kind of being reflected in the lot of
• 16:33
  the AI research. You shared a great post by RK
• 16:36
  dot F recently where they they had GPT 3.5 doing much, much
• 16:42
  better than O1 with dudes. And what that really goes to
• 16:46
  show you is that there's a lot of untapped intelligence that we
• 16:51
  have within the data sets within the system.
• 16:55
  But unless you explicitly build for it pull that how you're not
• 16:59
  going to get, you're not necessarily going to get that
• 17:03
  same performance. DeepSeek.
• 17:05
  One of the things they did really, really well is they
• 17:08
  didn't necessarily have the biggest training protocols, but
• 17:12
  there's architectures the way they approached it, it's a very,
• 17:16
  very it recognized that there was a lot of untapped potential
• 17:20
  in the data sets themselves. And then how do you enrich them,
• 17:24
  work with them and how do you build around them?
• 17:28
  So that kind of stuff is I think where that would personally most
• 17:35
  excite me, just from like a, hey, how to think about what
• 17:41
  problems are worth solving? Think about what behaviors we as
• 17:44
  a species want to encourage, What kind of AI does that need?
• 17:48
  And then what kind of data already exists, but we're not
• 17:52
  able to find it out because it's kind of sculpted around other
• 17:56
  opponents. So usually when we look, these
• 17:59
  other things stand out much, much more easily.
• 18:01
  So if we chisel this out or do we have to start collecting new
• 18:05
  streams? Do we have to start modeling it
• 18:07
  in new ways? Once you start doing that, I
• 18:10
  think that's where the world really takes it to the next
• 18:16
  level because you know, you can't.
• 18:19
  I, I don't think necessarily if you're not thinking about these
• 18:26
  outcomes, if you're not thinking about the end games, then I
• 18:28
  think what's likely to happen is very, very few influential
• 18:34
  groups that have vested interests in these areas will
• 18:38
  dictate most of the conversation because they're the ones who are
• 18:41
  actively thinking about it and they're the ones who are
• 18:43
  actively doing it. And what you will get is your
• 18:47
  50th e-mail integration, you know, your 20th sales SDR with
• 18:53
  AI, etcetera, the standard, oh, let's go build B2B SAS with the
• 18:59
  boys. I think that means really
• 19:02
  recognizing something deeper that for whatever reason that is
• 19:05
  what we have decided that a lot of people's bright minds that's
• 19:09
  that that is the extent of our success to raise it building the
• 19:13
  next big tech platform as opposed to deeper place.
• 19:19
  This is what will happen if you're not actively thinking
• 19:21
  about and actively trying to push for a vision that goes a
• 19:25
  little bit beyond that. This got suprisingly deep and I
• 19:33
  really appreciate that because I was going to make a joke about
• 19:35
  the SDR AI and since I just saw some TechCrunch article on 11X
• 19:41
  on how they were lying to people about their AISDR. But I would
• 19:48
  prefer to walk down the line of, yeah, how can we materialize or
• 19:52
  manifest this experience that we would like to have with AI as
• 19:58
  opposed to just making a better B to B SAS tool and I?
• 20:06
  Really like your your step backwards where the data may
• 20:10
  already be there, we just need to sculpt it in a different way.
• 20:13
  Or it may need to be augmented with some kind of synthetic data
• 20:19
  good at getting back to the Gretel AI idea.
• 20:22
  Or it may need to go and be collected and we may need to
• 20:29
  start from zero or from relatively 0 and then try and
• 20:33
  create feedback loops. So that's fascinating to think
• 20:38
  about it, and it is cool to think about.
• 20:41
  I'm now racking my brain to try to come up with ideas on how my
• 20:49
  ideal experiences would be with AI and what I would want them to
• 20:54
  do. And that's a thought experiment
• 20:56
  that I'm probably going to be grappling with for the next
• 20:59
  couple weeks. I think that's an experiment
• 21:02
  we're all implicitly grappling with a lot of the times, but I
• 21:07
  didn't bring it out to the main conversation would be
• 21:10
  interesting and just took an A little bit build on your point
• 21:14
  about 11X. It's a good example because
• 21:17
  people committed fraud in that specific situation and, and
• 21:21
  implicitly that means people thought that was worth
• 21:24
  committing fraud over, you know? That's what we value.
• 21:28
  Yes, that is what our word values both were like.
• 21:32
  Oh, this was able to raise money because of this and this was
• 21:35
  able to, they were able to commit fraud for it.
• 21:40
  I think thinking about like, is this the extent to which is
• 21:46
  valued would be critical for any, anybody trying to engage in
• 21:51
  AI? Because if you don't, if you're
• 21:53
  not the one trying to shape conversations, then you're the
• 21:56
  one who's getting them shape for you.
• 21:59
  And maybe you have other priorities.
• 22:01
  Maybe you just decide this isn't it for you and you're in a good
• 22:04
  place in life and you don't really care what things go like.
• 22:09
  I'm not going to sit and judge anybody for their decisions, but
• 22:14
  I would just say that there is a trade off to that that I think
• 22:18
  is often not fully understood. Now the other piece just
• 22:23
  speaking about fraud and something that we were talking
• 22:27
  about before we hit record was because of synthetic data we saw
• 22:34
  in what was it fraud detection models or loan scoring models
• 22:38
  that the inherent bias went down within financial institutions.
• 22:44
  Can you explain a little bit more about that?
• 22:45
  Because I think that's a cool thread to pull on too.
• 22:49
  Yes. So the study specifically I
• 22:53
  think was talking about, hey, if you already use automated
• 22:56
  systems for loan approvals etcetera.
• 22:59
  So banks that had very, very low ratio diversity in the loans
• 23:02
  they were getting were getting huge jumps and in general the
• 23:07
  loans given were more diverse than before and I think they.
• 23:11
  Put AI into play with their loans.
• 23:14
  The automated decision making system.
• 23:16
  Yeah, so, and I think it's very important to highlight such
• 23:21
  stories because normally you would talk about the negative
• 23:25
  biases around AI and how we can make it worse.
• 23:28
  But this was a good case. And I think what that goes down
• 23:31
  to again, is thinking about like what?
• 23:35
  What is it that would make one good and another bad?
• 23:38
  Usually when you're trying to build a new machine learning
• 23:40
  model or any any sophisticated statistical analysis to what
• 23:45
  will happen is you will actively start or you won't even realize.
• 23:50
  But there are there are ways for your model to pick up on
• 23:54
  demographic information that is not explicitly being accounted
• 23:58
  for. I think an example of this was a
• 24:00
  healthcare model where they were doing something like, oh, based
• 24:06
  on financial information or something, you're making
• 24:11
  predictions on what kind of treatment these guys would need.
• 24:15
  Yeah. Or like where someone lives is a
• 24:19
  prediction of their financial indication, which also is a
• 24:24
  prediction. You know, there's all these
• 24:25
  different features that you can have almost like scope creep on.
• 24:29
  Yes. So what ends up happening there
• 24:32
  is you're not explicitly accounting for race, but your
• 24:35
  model learns to pick out race anywhere.
• 24:39
  Where you live is a great example because in in America
• 24:42
  they had segregation. So a lot of neighborhoods were
• 24:47
  pretty divided along racial lines.
• 24:50
  And it's still true to a large extent from when I was
• 24:53
  travelling through. Not that that's the, you know,
• 24:55
  anybody explicitly saying that you have to live here or you
• 24:58
  live there in racial groups, but just like, you know, that's the
• 25:01
  way it got entrenched. So one of the weird things I
• 25:05
  picked up while travelling through America was just you can
• 25:08
  start to tell when you're shifting racial groups in
• 25:11
  neighborhood, when if you walk through enough, you'll start to
• 25:14
  pick up towels. So if you're accounting for that
• 25:16
  then and that becomes a significant predictor, then you
• 25:20
  might end up adding lowering the likelihood of somebody getting
• 25:26
  rejected from different races. So this AI was very, very cool
• 25:30
  because it, you know, they designed the AI in the right way
• 25:33
  to ensure that, you know, this wasn't accounted for and that
• 25:36
  there was like you're only giving out loans to the people
• 25:40
  that deserve it the most. What that also and where that
• 25:45
  becomes cool is now you might not see where synthetic data
• 25:49
  comes in here, but a huge part of that synthetic data pipeline
• 25:54
  is stuff like differential privacy where you redact
• 25:57
  sensitive information and you do all of that.
• 26:01
  So techniques from this ecosystem are being used
• 26:05
  everywhere. You know, you would probably
• 26:08
  they should probably do something similar with college
• 26:11
  law or college applications. They can do something similar in
• 26:15
  a lot of places where human touch points and biases etcetera
• 26:21
  are implicitly involved. So it would help to has a more
• 26:25
  diverse understanding on hey, let's make ensure people aren't
• 26:30
  filtered out for the wrong reasons.
• 26:33
  Yeah. And no.
• 26:35
  What about just data? We talked about synthetic data,
• 26:40
  but also I think there's pieces that you wanted to hit on when
• 26:46
  it comes to the data ecosystem. We've kind of been touching
• 26:51
  around different specs of that already, but I think one of the
• 26:54
  areas where we're significantly underpricing right now is how do
• 27:02
  you know what your data, how much your data is valuable and
• 27:05
  how do you integrate intelligence into the way your
• 27:09
  data operates. So an example of that might be
• 27:13
  something like graph neural network or graph flag is you
• 27:17
  know, the newer trends, etcetera.
• 27:19
  What a graph is, is essentially another way of encoding the
• 27:23
  intelligence of your data set, because what that relationality
• 27:27
  does is it allows you to look across the different spectrums.
• 27:34
  Something like that could be very interesting.
• 27:36
  Not not specifically graph RAG or graph neural networks, but
• 27:41
  how can you structure what you already have in different ways
• 27:44
  to integrate different kinds of intelligence directly into your
• 27:50
  decision making? I think that would be
• 27:54
  exceptionally valuable as an insight because I think we kind
• 28:00
  of, it's one of those weird things where we've been
• 28:04
  traditional databases, then some genius game comes up with the
• 28:09
  idea of phase in meta, you know, fast distributed vectors.
• 28:15
  And then people just took face and database, stuck them
• 28:20
  together, called them back to databases and they raised a lot
• 28:25
  of money. And then we just kind of
• 28:27
  collectively decided that this is it, this is the peak of
• 28:31
  databases that we can get to, which was the, again, I think
• 28:36
  there's so much we haven't explored on that side.
• 28:40
  A lot of the attention is kind of going on models and
• 28:43
  intelligence in the model, whether you're doing it with the
• 28:49
  reasoning models, you're doing it with better alignment, doing
• 28:52
  it with more post training, etcetera.
• 28:55
  But if you start looking backwards, I think there's a lot
• 28:59
  of insight just to be done by how can we restructure the very
• 29:06
  way we store and process data to encourage different kinds of
• 29:12
  information that I'm personally very excited about because I
• 29:16
  think there's just there's significant market potential
• 29:22
  there. And I think if you do that, if
• 29:26
  you can move intelligence into the data layer, you can do a lot
• 29:30
  of work with, you can just move in so many domains in so many
• 29:36
  ways because that's like an architecture innovation.
• 29:40
  And once you have an infrastructure innovation, I
• 29:42
  think they tend to have the biggest longevity in making lots
• 29:48
  of impact in a lot of different ways.
• 29:51
  Now when you say move intelligence to the data layer,
• 29:54
  I think bringing an LLM to a Raspberry Pi or something like
• 30:00
  that, what what do you mean when you say moving intelligence to
• 30:03
  the data layer? It's a very abstract idea
• 30:09
  because we are not fully sure even how we want to do it.
• 30:14
  But I think graphs are a good example because they are a very
• 30:18
  evident way of doing this. You know, when you think about
• 30:22
  graphs as a data structure, what do you have?
• 30:24
  You have the node, you have the relationship, and you have the
• 30:27
  weight of the node. I think fundamentally that's
• 30:30
  what the graph is. And that's a good example
• 30:34
  because you are, if you had very, very unstructured data,
• 30:38
  you technically have all the information in there, Maybe not
• 30:42
  not the weight mentioned explicitly, but everything else.
• 30:45
  But why does Graph Rack do so well or why do craft data
• 30:48
  structures do so well? Because they've restructured it
• 30:51
  to really emphasize the relationality of your different
• 30:56
  components. So that is one kind of
• 31:00
  intelligence. And in an, in a structure, in
• 31:04
  any use case where relations become your major factor, they
• 31:08
  become a very important part. So that's what I mean is the
• 31:13
  idea of storing information in graphs, the way you store them,
• 31:17
  what you store it becomes very, very Creek.
• 31:19
  Because one like one thing that people don't understand about
• 31:23
  something like a knowledge graph is that there are multiple ways
• 31:26
  to build it based on what you choose to prioritize.
• 31:29
  Same data set, same high level problem.
• 31:32
  You can actually build it in multiple ways.
• 31:34
  And the ways you build it will have very, very important
• 31:37
  applications of your performance.
• 31:40
  That's kind of a high level example of moving intelligence
• 31:46
  into your data layer is restructuring the way you have
• 31:48
  that set up. But there are there are going to
• 31:51
  be lots of future examples. How do we, how, what kinds of
• 31:56
  embeddings can we create, what kinds of behaviour modelling can
• 32:00
  we do? Can can we start figuring out
• 32:02
  ways where I am looking at certain behaviours from users
• 32:08
  and automatically creating a model around that so that I can
• 32:13
  copy your actions, so that I can Start learning from your
• 32:16
  actions, playing with their actions?
• 32:20
  That's kind of what I mean. Can I start looking at your
• 32:24
  behavior and automatically coming up with evaluation
• 32:27
  functions to say this is what Devansh values is what Demetrius
• 32:31
  values? How does that change behavior
• 32:33
  over time? I think that's where that would
• 32:37
  be very, very interesting to look at and possibly very, very
• 32:42
  cool stuff to do. That is wild.
• 32:46
  Yeah, I hadn't thought about doing it on the behavioral
• 32:50
  level. I just thought about like, oh,
• 32:53
  we'll get Slack messages into the knowledge graph and GitHub
• 32:58
  issues are now part of the knowledge graph, and then cool,
• 33:01
  we've got a more robust context. But thinking about watching my
• 33:05
  screen and recognizing what behavioral things I do, ideally
• 33:11
  I could cut out half of the shit I do because it's probably not
• 33:14
  productive, you know? I mean going and scrolling X and
• 33:18
  LinkedIn. I do not want that to be part of
• 33:20
  my behavioral model. But I think that might also be
• 33:25
  where you get very good results because you're able to, you
• 33:29
  know, you have those moments of not doing much, but not not just
• 33:34
  an individual behavior model. What would be cool is imagine
• 33:38
  you had an AI recognizing because people don't understand
• 33:42
  this, but AI is not intelligent in the way we're intelligent,
• 33:46
  but that also just means that it has different abilities that we
• 33:49
  don't. What you're able to do is
• 33:52
  imagine you're able to track behavior across different users
• 33:57
  in a GitHub and then you're able to start identifying what
• 34:01
  problems trigger, what kind of behavior patterns trigger
• 34:04
  problems. Well, like, you know, we're
• 34:06
  having issues right now with a startup.
• 34:09
  You know, we have so many people working on so many feature
• 34:12
  branches. Everyone is like doing two or
• 34:14
  three different things at once that you constantly have
• 34:18
  divergent in the working tree, etcetera.
• 34:20
  And then merging them, cleaning that up is a nightmare.
• 34:24
  So all of that stuff is where you could have truly, truly life
• 34:30
  changing work, changing experiences because you're able
• 34:34
  to process and monitor stuff at scale to identify where do
• 34:39
  things go wrong, what does it improve?
• 34:42
  And I think that's a very, very cool innovation.
• 34:46
  The reason I'm thinking about that extensively now is because
• 34:51
  so I'm working with a legal AI startup called Akitas.
• 34:53
  And one of the ways we're just better than everybody else for
• 34:58
  legal is that instead of making innovations in the model layer,
• 35:03
  we're kind of we we spend a lot of time making kind of
• 35:07
  innovations in the process layer.
• 35:09
  So you, you know, you have multiple steps before you come
• 35:12
  up with the final answer. You might do something like the
• 35:16
  a very it's a very stupid way to improve your stuff.
• 35:20
  If you're just throwing data and saying this is good, this is
• 35:22
  bad, this is good input this battle.
• 35:24
  But you know, that's what Harvey and all are doing and it's a
• 35:28
  very stupid way to do things because you're not going to be
• 35:31
  able to propagate signal properly for any complex
• 35:34
  knowledge or through that. But if you're able to map out an
• 35:38
  entire workflow and then build individual components to it and
• 35:42
  improve it across the layer, you just do much better work when
• 35:47
  that's what agents were are philosophically good at doing.
• 35:51
  That's what other setups are good at doing.
• 35:53
  So that's where I'm, I mean, I've been thinking about this
• 35:56
  pretty extensively because we kind of did a lot of workflow
• 35:59
  based improvements where it's like, OK, if you're doing a
• 36:03
  contract review, what individual components do you have to do
• 36:07
  there? But I think on a general level,
• 36:09
  this can be abstracted into how do you, what behaviors would you
• 36:15
  have and how do you track them? How do you monitor them to
• 36:19
  identify potentially risky situations before they happen
• 36:24
  potentially, or this is what you guys do it early, early, but
• 36:28
  like this is a bit worth doubling down on.
• 36:32
  I think once you're able to start doing that, you have a
• 36:35
  drill winner on your hands. Yeah, I've heard it put as.
• 36:42
  Try and get the model to do the least amount of work possible,
• 36:45
  or give the model the least amount of responsibility
• 36:48
  possible and create as many work flows or systems around the
• 36:56
  model so that when it gets to the model, the scope that it has
• 37:00
  is much more limited and therefore you have a much higher
• 37:08
  outcome of producing the right result.
• 37:14
  That is very well put. Honestly, I should have talked
• 37:18
  to you before this. I could have stolen that and
• 37:21
  seem much more articulate than I am.
• 37:23
  I don't. I can't remember who I stole it
• 37:25
  from, but it was not an original idea.
• 37:28
  I will tell you that much The but the the thing that I like
• 37:33
  that you're saying is abstracting that out and looking
• 37:37
  at behaviors and recognizing which behaviors are going to be
• 37:47
  playing which part in this workflow or in this agent life
• 37:52
  cycle. And not asking that you just
• 37:55
  have this open-ended agent that does whatever you have maybe an
• 37:59
  agent that decides to kick off a workflow which is very
• 38:03
  deterministic and or there it there are different workflows
• 38:08
  and as parts of the workflow, there's a little LLM node in
• 38:12
  there. I mean, this isn't where I was
• 38:16
  planning to go with it, but now that you've brought it up, I
• 38:19
  guess I can talk about one of my very bad annoyances.
• 38:24
  I think autonomous agents are largely A sham.
• 38:27
  And I think the big reason why Silicon Valley likes to push
• 38:33
  them is because you, when you push for something like this,
• 38:38
  it's very sci-fi. So there's very little
• 38:41
  accountability because you can always defer the responsibility
• 38:45
  of actually getting things done up till later.
• 38:49
  I I don't think there's a single business I've spoken to, and
• 38:53
  this is not a exaggeration that isn't one that I could name for
• 38:57
  you that did good stuff with autonomous agents in their
• 39:01
  business workflows. Invariably they ended up boiling
• 39:04
  it down to much, much more deterministic setups.
• 39:10
  I that like there might be value in autonomous agents for like
• 39:15
  research purposes and whatnot, but if you're a business trying
• 39:18
  to build things, I wouldn't waste too much time with them.
• 39:24
  Go like simple deterministic workflows and agents I think are
• 39:28
  very, very underrated in under underutilized because again,
• 39:32
  people don't like thinking. And I think there's a little bit
• 39:35
  of an element where most agents, agents are being pushed by a lot
• 39:40
  of tech people and tech people by and large, you get into tech
• 39:45
  because I think you tend to be a little bit more future thinking.
• 39:47
  So you're always thinking about what could I do next?
• 39:50
  This this is kind of what gets bred into your mindset as
• 39:54
  opposed to they, what can I do right now?
• 39:57
  And I think in the case of agents, that's just led to a you
• 40:01
  choose misplacement of priorities.
• 40:05
  I think the places that I've seen the most success are when
• 40:09
  it is a workflow that is a determined pattern that is a
• 40:15
  repetitive task. And one or two of those nodes or
• 40:18
  all of those nodes are some kind of an LLM call that does
• 40:24
  something. Whether it is scrapes a website
• 40:28
  and then summarizes it or it goes and it creates a task that
• 40:36
  it will execute. These are things that you can
• 40:42
  have a bit more certainty on if you know that it's going to be
• 40:48
  fairly similar every time around.
• 40:52
  Where you get into trouble is saying that it's like this
• 40:58
  generalized agent, even if it is a generalized vertical agent
• 41:03
  like in a generalized HR agent or a generalized finance agent.
• 41:07
  Here you go. That's where you can get into a
• 41:09
  lot of trouble because it feels like you can't throw anything at
• 41:15
  it and expect it with high accuracy or high probability to
• 41:20
  carry out that task. And then there's frustration
• 41:24
  because you're grappling with the output and you don't know.
• 41:27
  Did I just not prompt it? Well, do I need to ask it in a
• 41:30
  different way? What are we doing here?
• 41:33
  Is it just not possible? That type of thing?
• 41:35
  And so I go back and forth on the workflows are quite valuable
• 41:43
  and maybe the master agent that can kick off different workflows
• 41:48
  is a great design pattern that we can see.
• 41:51
  But then that generalistic agent I'm keeping my eye on it for, is
• 41:59
  it actually going to produce that value, or is it just this
• 42:03
  hyped thing that you're talking about where it's really cool to
• 42:09
  fantasize about, but if you're running a business, hopefully
• 42:13
  you're not actually putting a lot of faith into it.
• 42:18
  I think, I don't know how much you guys have talked about like
• 42:23
  the automation, the doorband fallacy, I think it's called.
• 42:26
  But I think that's a very good case study for anybody that
• 42:31
  wants to build technology. I think it's shocking that
• 42:35
  that's not hammered it. And I feel like if there was an
• 42:38
  AI person MBA, that's basically the case study that I would wake
• 42:42
  up my students with every day. Which fallacy and which case
• 42:47
  study I didn't hear? So the doorman.
• 42:51
  No, I don't know this one. Tell me.
• 42:53
  So prestigious hotels. Well, it's like some genius came
• 42:58
  up with this idea, and this sounds like a very reasonable
• 43:01
  idea that we now have automated doors so we no longer need to
• 43:05
  hire doormen. And you like, you know, that
• 43:12
  makes perfect sense. You don't need somebody to open
• 43:13
  a door for you anymore. Just keep it automated.
• 43:19
  When that happens, what end up happening is the hotel had worse
• 43:23
  outcomes, like there were homeless people hanging around.
• 43:25
  The first part, you know, customer satisfaction was down,
• 43:31
  etcetera. And what you end up realizing is
• 43:33
  that the doorman's stated job is to hold open doors and not do
• 43:39
  doors. But what the doorman also does
• 43:42
  is greets you. Well, maybe you don't care that
• 43:45
  much for that, but it's also a matter of you.
• 43:49
  I come in with a taxi. I've just flown in from a
• 43:52
  different country are big suitcases.
• 43:54
  Oh, the doorman's going to help me out.
• 43:56
  The doorman standing there prevents like a vagrants from
• 43:59
  coming in and like, you know, they're not in the guests,
• 44:02
  etcetera, hanging around the lobbies and whatnot.
• 44:05
  You have all of these other like not set benefits.
• 44:10
  Or here's a great example. This one time I was biking
• 44:15
  through New York and I dropped my phone.
• 44:18
  There's a true story and somebody picked up my phone and
• 44:20
  left because they thought, oh, I don't want somebody else to
• 44:23
  steal it. But what this means is that I no
• 44:25
  longer have my phone and I just moved into the city, so how am I
• 44:30
  going to get back home? Yeah.
• 44:32
  Oh no. And what helped me was a hotel
• 44:36
  doorman because I'm, I'm a, I'm an Uber kid.
• 44:40
  I think I've definitely taken more Ubers in a year than I have
• 44:43
  taken yellow taxis my whole life.
• 44:48
  So he actually flagged down one of them like yellow.
• 44:53
  It's taxis. I would not for me, and I got to
• 44:58
  go home. Now, I wasn't even living at
• 44:59
  that hotel, but that kind of stuff is not something an
• 45:05
  automatic door does for me. An automatic door opens and
• 45:08
  closes. So I think that's where it's a
• 45:13
  great case study for both replacing human labor, but also
• 45:19
  for thinking about when you're talking about these verticalized
• 45:22
  agents, when you're talking about these verticalized
• 45:24
  platforms, like people don't realize how much other stuff
• 45:30
  people do that has nothing to do with their job that ends up
• 45:34
  being marginally useful here and there.
• 45:37
  You know, you mentioned scrolling on social media as an
• 45:41
  example. I will occasionally find stuff
• 45:45
  on threads or Twitter or what not that's worth sharing with my
• 45:49
  research lab team. I will occasionally find
• 45:51
  solutions. So that kind of that kind of
• 45:58
  information. I mean, I, I found corrected by
• 46:01
  scrolling. I not well as a company, but
• 46:04
  since that it did, I was scrolling through, I found some
• 46:06
  discussions on like the replication bias and AI and that
• 46:10
  led to me finding this paper that was talking about, hey,
• 46:14
  we're struggling with like with healthcare data you're not able
• 46:18
  to send across borders. So we can't replicate a lot of
• 46:20
  experiments, which is why we just chose to do synthetic
• 46:23
  healthcare data and some real healthcare data because now I
• 46:26
  can just publish my whole data set and call it her day.
• 46:30
  And that's what that was the whole reason I started
• 46:32
  monitoring synthetic data around 2020-2021 in the 1st place.
• 46:37
  So like that kind of stuff. If you were trying to replace
• 46:41
  me, you could say this is not useful stuff.
• 46:43
  And if you were studying AI, then yes, technically 99% of
• 46:48
  what I would have scrolled through would be like MMA
• 46:50
  reloads, cats or fight if football or what not.
• 46:55
  But that another 1% had an outsized impact, but would be
• 47:00
  very hard to quantify and very hard to account for.
• 47:03
  So that kind of stuff I think becomes very, very useful or
• 47:09
  like jiu jitsu I like. I might.
• 47:12
  I threw a random off comment there about jiu jitsu on my in
• 47:17
  one of my articles and then turns out one of my readers
• 47:19
  really likes jiu jitsu. So we connected on that and we
• 47:23
  ended up doing something together.
• 47:24
  Very, very cool. Nice.
• 47:27
  Projects together Now one of the interesting things there is AI
• 47:33
  actually doesn't like the way I write.
• 47:35
  It's told me on many occasions that because I I tend to use a
• 47:39
  lot of references and very off the cuff tangents and things
• 47:43
  like mentioning jiu jitsu not as a analog, not like as an
• 47:47
  intelligent analogy, but like, Oh yeah, I do this and I don't
• 47:51
  know I was talking to somebody here and going into these
• 47:54
  slightly off topic tangents that's AI is like, hey, clean
• 48:01
  this up. I think because your work is
• 48:02
  already long like type in this condense this a professional
• 48:06
  audience would not like your work, especially clot.
• 48:09
  Clot hates my writing. Jesus.
• 48:13
  So I think that kind of integration becomes or with that
• 48:22
  kind of verticalization comes very, very difficult.
• 48:25
  So if you were building any AI systems or any tech systems and
• 48:30
  your automations, I think it's always worth thinking about that
• 48:35
  a lot of the time VS AI engineers aren't dormant VS AI
• 48:39
  engineers aren't HR assistants and we've probably not studied
• 48:45
  that process in depth. Not saying that they can't be
• 48:47
  replaced, I'd I'd be the first person campaigning to kill out
• 48:51
  every HR person and have them do something actually meaningful to
• 48:56
  society. But like there's also like a
• 49:00
  reason these things exist and there's a reason like the the
• 49:05
  like humans. Labour has a lot of random
• 49:11
  behaviours that you might not be able to quantify or touch for
• 49:14
  that does a lot of great, that has a lot of impact in the
• 49:19
  actual work. So I think it's just a good case
• 49:22
  study to think about. Do this doorbell case study is
• 49:27
  wild because it's almost the second or third order effects of
• 49:32
  implementing a system that at face value is much better.
• 49:37
  You save cost because now you don't have to pay the salary for
• 49:41
  someone who's holding the door. And I think about it and
• 49:47
  extrapolate it towards any type of AI system.
• 49:53
  And I, I know I, I tend to talk a lot about marketing copy being
• 49:59
  generated with AI. I've seen some very advanced
• 50:03
  systems of AI generated blog posts and you can churn out a
• 50:09
  ton of posts with AI. But it makes me question now
• 50:19
  potentially what you're getting. You're getting really good SEO
• 50:23
  rankings because you're now churning out a ton of highly
• 50:27
  optimized SEO keywords, all that good stuff.
• 50:32
  And so you're shooting up in the rankings and you're getting a
• 50:35
  lot more traffic. But you also want to think
• 50:39
  through, is this traffic, reading this, recognizing that
• 50:44
  it isn't that valuable and it's hurting my brand.
• 50:48
  Because that could be a potential outcome if you're just
• 50:51
  churning out a bunch of AI slop or in the case of HR and you
• 50:58
  have some kind of a pipeline and it's for hiring, but your
• 51:04
  potential candidates don't get to actually talk to a human
• 51:09
  until they've done three or five things.
• 51:13
  They've jumped through all these hoops that are very much graded
• 51:17
  by AI. Is that giving the best
• 51:19
  experience to this future teammate, right?
• 51:24
  Like how what kind of culture are you building there?
• 51:26
  And maybe down the line, you're optimizing for someone that is
• 51:34
  not going to be the best fit or there's.
• 51:38
  Yeah, it's, it's really, there's probably a name for this, but
• 51:43
  the second or third order effects are huge.
• 51:51
  I mean, that is exactly what yeah, you started so well is
• 51:57
  second, third order impacts of where you get it wrong.
• 52:00
  And I think that's where I mean, I feel like a lot of tech people
• 52:05
  would benefit from thinking not about tech, but about other
• 52:09
  aspects in their life. Like why does a romantic dinner
• 52:13
  sound romantic in the 1st place? Most of it is sire of marketing
• 52:16
  bullshit. Look at McDonald.
• 52:19
  But like walking into like a place with ambiance with a
• 52:23
  candlelight or whatever that does do like there's a reason
• 52:28
  those things have certain impacts that we associate.
• 52:32
  You know, one of the things is the more simplistically you look
• 52:38
  at a problem, a situation, an outcome, the more you're likely
• 52:44
  to miss critical pieces that end up becoming that end up
• 52:49
  unraveling it. And I think that's where, I
• 52:53
  mean, a lot of our legal tech competitors went wrong.
• 52:57
  This is where a lot of staff have seen in general tech gets
• 53:00
  wrong very often is you'd look at, you look at you boil a whole
• 53:06
  problem down into a very, very narrow domain or a very, very
• 53:10
  narrow subtask. And it turns out that that
• 53:12
  subtask isn't super duper important to begin with.
• 53:17
  So I just think it's worth thinking about the Dorban
• 53:20
  fallacy, things like that. These are genuinely things that
• 53:28
  have rebound. If you don't think about them,
• 53:30
  you'll just end up building products that are useless to
• 53:33
  society. AI based customer services, good
• 53:36
  example where if you're not doing it correctly, what ends up
• 53:40
  happening is you'll just have worse outcomes because I, I just
• 53:44
  want to call you up and ask you about my stuff and like, there's
• 53:48
  no way for me to get that. Hey.
• 53:52
  And the customer service should be replaced.
• 53:54
  I'm sure like there is a lot of good outcomes there you can
• 53:57
  where you can speed it up. But you know, if you try to
• 54:02
  replace everything with a voice bot and dial this on your phone
• 54:05
  and do that, I think you're you might be missing very key
• 54:10
  elements of what is it that makes people happy or whatever
• 54:14
  that people want from you. And you kind of implicitly
• 54:19
  filter for people who like the kind of loyalty you build is
• 54:23
  towards people who want who want to maintain a certain like cost
• 54:31
  becomes their main priority, which means they'll cut you out
• 54:33
  if you become too expensive as opposed to offering a little bit
• 54:37
  more, having that luxury and prestige and or not associated
• 54:40
  with it and providing that acquisite services.
• 54:43
  I pay a higher premium on my credit card and higher fees and
• 54:48
  whatnot. Like if I were overdraft just
• 54:52
  because my credit card guys have guaranteed me 24/7 immediate
• 54:58
  support any country I go to. Like I I pay you an extra, you
• 55:03
  know, few hotels for that and risk potential a lot more.
• 55:11
  Though it's AI support and it's not human support.
• 55:15
  If that happens I would be so mad.
• 55:17
  I'm here to talk to the culprit in other countries.
• 55:19
  Like hey, please hold this ring called press 2 if we want to
• 55:24
  talk this but. Oh man, well, hopefully it
• 55:28
  doesn't ever get to that point where you have to use it, But I,
• 55:31
  I understand the sentiment of we need to, for certain things.
• 55:38
  We look at how we are, how can I say this?
• 55:45
  It's like we're, we're voting with our dollars and we're
• 55:49
  voting and inherently saying what is important to us as
• 55:53
  customers. And the there's as a builder, a
• 55:58
  razor thin edge that you have to walk on.
• 56:02
  If I automate this or if I bring automation into this, be it AI
• 56:08
  or not, is that going to create a worse experience or is this
• 56:13
  going to affect my business metrics in any way for the
• 56:17
  better or for the worse? And that's something that I
• 56:20
  think continuously folks ask about because you want to be
• 56:28
  able to show and justify an impact that you are making as a
• 56:32
  developer, as someone who is implementing AI into production.
• 56:37
  So you want to figure out what metrics you care about and the
• 56:43
  company cares about. And then how can you hopefully
• 56:47
  make those metrics move? Now what you also want to
• 56:52
  recognize is are you making those metrics move in the wrong
• 56:56
  direction? Or are there other metrics that
• 56:59
  the business cares about that are being affected that you
• 57:02
  didn't even recognize? And so this brings in, it just
• 57:06
  makes me think like, wow, you really have to be playing 3D
• 57:09
  chess and you have to be recognizing all these different
• 57:13
  outcomes. And it probably just starts by
• 57:17
  you being a user or you trying to game it out in your head and
• 57:22
  recognize where and how the old way has its benefits versus the
• 57:30
  way that you're trying to implement and what benefits that
• 57:34
  has. And really recognizing what the
• 57:36
  past benefits are with some a doorman, for example, why is
• 57:41
  that beneficial or an actual human that you talk to and
• 57:45
  what's good and bad about that? Because we all have had
• 57:48
  experiences where we talked to a human and we get passed around
• 57:51
  to three different departments and each time we have to explain
• 57:54
  our case. And so even though it is a
• 57:57
  human, it still is a really shitty product.
• 58:00
  So thinking about all those different ways that you want to
• 58:06
  make the product better is it's fascinating.
• 58:12
  And now like, I'm just going to probably be ruminating on this
• 58:16
  for the next week or two. Again, with these thought
• 58:20
  experiments that you're giving me.
• 58:21
  This is like a lot of shower thoughts that I'm going to be
• 58:24
  having, I'm sure of it. Oh, that's how I pretend to be
• 58:29
  smarter than I am as I just before this meeting I was coming
• 58:32
  over like 20 random hypotheticals so that you don't
• 58:35
  have to ask me on anything technical and 70 year old
• 58:38
  people, I think. Who is this guy Demetrius is
• 58:40
  talking to? I don't think you could be.
• 58:44
  Oh what sweet dude? Is there anything else you want
• 58:47
  to hit on before we we cut it? I think a talking point I tried
• 58:54
  to emphasize a lot through my work, but I think it's always
• 58:57
  worth thinking about. I don't think AI is as solved as
• 59:01
  people think it is. And what that means is that it's
• 59:04
  a very open field and people aren't being ambitious enough
• 59:08
  with that idea. You know, there's a lot of open
• 59:12
  space in AI where you can come in, really screw up existing
• 59:17
  power structures, kind of throw your, throw a venge in the world
• 59:22
  and come and take over. And I think more people should
• 59:27
  really come to recognize that that yes, artificial
• 59:33
  intelligence in the field of study has existed for many
• 59:36
  years, but AI as an institution in society and an influential
• 59:41
  group and how it's impacting it and much more active level is
• 59:46
  relatively new. And what that means is that you
• 59:50
  don't necessarily have to be a math guy like me or, you know, a
• 59:54
  software engineer or something to make an impact.
• 59:57
  There's like a lot of. The there's just a lot of open
• 1:00:02
  space where you can come in and shake things up.
• 1:00:05
  So if there's anything I would want people to think about and
• 1:00:11
  work on actively is just, hey, am I, am I really doing things
• 1:00:16
  to shake things up? Or am I just try doing things to
• 1:00:19
  get by and get, you know, am I am I trying to become a Google
• 1:00:25
  or am I trying to be picked by a Google?
• 1:00:30
  And I'm not saying that there's something wrong with the
• 1:00:33
  laughter. Again, if you have other
• 1:00:35
  priorities, you're content with your life.
• 1:00:38
  You know, I'm not going to say sell your wife and kids and
• 1:00:40
  focus on, you know, whatever you have to.
• 1:00:43
  But I think so many people that I speak to are just kind of
• 1:00:48
  implicitly buying into narratives where there are
• 1:00:51
  experts and there are non experts.
• 1:00:52
  There are there are like, you know, there's an other people
• 1:00:59
  from them where that will actually do things, but as when
• 1:01:03
  they kind of become end up becoming participants since they
• 1:01:06
  become viewers of society as opposed to participants.
• 1:01:10
  So I would just say like, if you're going to make that
• 1:01:13
  decision, make it consciously as opposed to just kind of making
• 1:01:20
  it because you've never thought about the alternative.