Latest / The DEX Show: A Show for IT Change Makers / Rude AI Agents?!? An AMAZING LOOK Under the Hood @ Nexthink's AI Labs

Transcript

Rude AI Agents?!? An AMAZING LOOK Under the Hood @ Nexthink's AI Labs · The DEX Show: A Show for IT Change Makers

• 0:02
  Speaker 1: Hey, it change makers and welcome to the Deck Show
• 0:05
  Speaker 1: with tom Ac gra Let's get into it.
• 0:09
  Speaker 2: What happens when AI stops being a tool you use
• 0:14
  Speaker 2: and starts becoming an active participant, an active colleague in
• 0:18
  Speaker 2: whatever it is that you do for work. Well, who
• 0:22
  Speaker 2: better for these two gentlemen to help us navigate such
• 0:26
  Speaker 2: brain bending territory. We're welcoming back to the show. First
• 0:32
  Speaker 2: of all for his third appearance, I believe, Mohammed Cavsi
• 0:36
  Speaker 2: and a mama for anyone who missed your prior appearances,
• 0:39
  Speaker 2: maybe give a fuller introduction of yourself.
• 0:42
  Speaker 3: Of yes, Dom Muhammad, I'm the head of AI Innovation.
• 0:47
  Speaker 3: Next think I'm leading the decks front of your lab,
• 0:50
  Speaker 3: the Dix front. Your lab is our replied the research team,
• 0:55
  Speaker 3: and it's and we are focusing on building the next
• 0:57
  Speaker 3: generation of decks and IT products. Right, So basically, in
• 1:02
  Speaker 3: simple terms, we we prototype ideas before they become products.
• 1:08
  Speaker 4: And is it?
• 1:09
  Speaker 2: Is it? Your third? Is your third appearance on the
• 1:11
  Speaker 2: on the Tech Show?
• 1:13
  Speaker 3: I have a doubt it's either the third or the fourth.
• 1:16
  Speaker 2: Wow, we are we are racking up a few and
• 1:18
  Speaker 2: we had to combine it. We had to, We had
• 1:20
  Speaker 2: to have a first time around, just just to make
• 1:23
  Speaker 2: it sure it didn't get too chummy, too familiar here,
• 1:26
  Speaker 2: So welcome to the first time. You've seen ben Yahia
• 1:30
  Speaker 2: before we before we get into it, you've seen same same,
• 1:34
  Speaker 2: same invitation. Please introduce yourself, give a full picture of
• 1:37
  Speaker 2: your work here and next thing for the audience.
• 1:39
  Speaker 4: Thank you, Yeah, thank you. Tom, so me a scene.
• 1:42
  Speaker 4: I joined Next Thing two years ago. I started on
• 1:44
  Speaker 4: Assist actually now called Workspace, where I contributed to key
• 1:48
  Speaker 4: features including textre and Creator, which I believe Mamma did
• 1:51
  Speaker 4: did the podcast on previously one of his two or podcasts.
• 1:56
  Speaker 4: I also had to bring ass to g A and
• 2:00
  Speaker 4: now moving to Spark, specifically the Spark Agent team, which
• 2:04
  Speaker 4: I now lead. So super excited about what we're building
• 2:08
  Speaker 4: in Spark, like reimagining it and ultimately even improving the
• 2:13
  Speaker 4: everyday work life of employees. So super excited to be
• 2:16
  Speaker 4: here today.
• 2:18
  Speaker 2: Fantastic. So maybe bring us up today, Muhammed, since you
• 2:23
  Speaker 2: asked him of a show just over a year ago.
• 2:25
  Speaker 2: In fact, what have your teams been, what has your
• 2:28
  Speaker 2: team been working on? Primarily on what's that journey been like?
• 2:32
  Speaker 3: So, as I said, Tom, we're exploring the future of
• 2:37
  Speaker 3: dex and IT technology and our lets. A main theme
• 2:41
  Speaker 3: in the last years has been AI agents for the enterprise.
• 2:46
  Speaker 3: And you know, we've seen a big shift recently in
• 2:51
  Speaker 3: AI agents, right. They shifted from their ability like to
• 2:55
  Speaker 3: answer questions and today that are able to observe the
• 2:58
  Speaker 3: reason to take actions right and iterate. And those capabelities
• 3:04
  Speaker 3: were really excited about because we think they have a
• 3:06
  Speaker 3: huge potential for decks for it for our customers. The
• 3:10
  Speaker 3: key areas we're exploring, right, it's the ability of AI
• 3:13
  Speaker 3: agents to understand what's happening on the screen of a device, right,
• 3:19
  Speaker 3: their ability to code, to use tools and execute actions. Also,
• 3:25
  Speaker 3: something we might talk about during this episode is their
• 3:28
  Speaker 3: ability to learn right from their action in the past,
• 3:33
  Speaker 3: their mistakes, and also the interaction they have with humans.
• 3:38
  Speaker 3: And also it's an opportunity and it's for us to
• 3:41
  Speaker 3: rethink how AI agents are automating the task together and
• 3:46
  Speaker 3: working along with humans. Right. So all of that we're
• 3:49
  Speaker 3: exploring those different questions and as Sen said before, as
• 3:53
  Speaker 3: yes Scene said before, the goal is always to deliver
• 3:58
  Speaker 3: the best let's say digital employees for at scale, right,
• 4:03
  Speaker 3: and those I mean I'd be happy throughout this episode
• 4:06
  Speaker 3: to go like a little bit deeper on those aspects
• 4:08
  Speaker 3: I mentioned about the agents.
• 4:11
  Speaker 2: And your software engineers are now using claud code daily, right,
• 4:18
  Speaker 2: what's that been like for you? I guess is like
• 4:20
  Speaker 2: is it as a leader first of all, Muhammed, and
• 4:22
  Speaker 2: then we'll talk to you see maybe who's more involved with
• 4:24
  Speaker 2: the utilization of the day to day.
• 4:28
  Speaker 3: No, I mean clod code. If you take an example
• 4:30
  Speaker 3: of clod code that we use actually daily. Next thing
• 4:36
  Speaker 3: in any company that's writing code and building products. First
• 4:41
  Speaker 3: on the personal level, I use cloud code like daily
• 4:44
  Speaker 3: for her job, but also during the weekends. Right. For
• 4:50
  Speaker 3: last weekend, I was using it with the with my
• 4:53
  Speaker 3: daughter to develop a video game and help her learn math. Right,
• 4:58
  Speaker 3: so it can be also fun only for the professional
• 5:02
  Speaker 3: part of our lives, I use it also for home automation.
• 5:05
  Speaker 3: I use it for many things outside work, but for
• 5:08
  Speaker 3: work it definitely has the biggest impact today. This is
• 5:11
  Speaker 3: what we see in the last year, what we've seen
• 5:16
  Speaker 3: that software engineering has completely transformed in the sense that
• 5:21
  Speaker 3: transformed from using AI as a as let's say, a
• 5:26
  Speaker 3: campaign a companion, right, and more being in the mode
• 5:30
  Speaker 3: of where AI is helping you complete your code and
• 5:32
  Speaker 3: or too complete mode to agentic coding where you have,
• 5:38
  Speaker 3: as you said Tom at the beginning, a colleague, a
• 5:41
  Speaker 3: junior colleague, right, that can write code from scratch if
• 5:46
  Speaker 3: you explain it to well enough, that can make mistakes,
• 5:48
  Speaker 3: but with enough iterations it will provide a very good
• 5:54
  Speaker 3: output and can be completely autonomous and generating code. So
• 5:57
  Speaker 3: this is completely transformed the way we build soft and
• 6:01
  Speaker 3: our job as software engineers, with the impact for us
• 6:04
  Speaker 3: as a as a software company, but also with the
• 6:09
  Speaker 3: impact and it t right, and this ability to write
• 6:13
  Speaker 3: code is not only transform the job of such engineer
• 6:15
  Speaker 3: but also transform it as we know today.
• 6:19
  Speaker 2: Mm hmmm, mm hmmm. So have you had to restructure
• 6:24
  Speaker 2: your team, restructure your your your how you supervise your team,
• 6:29
  Speaker 2: how you supervise your direct reports, how you supervise the
• 6:32
  Speaker 2: code that's being produced and developed? How has this huge
• 6:35
  Speaker 2: step forward forced you or allowed you to reconfigure how
• 6:42
  Speaker 2: how you lead and how you run AI lads The next.
• 6:44
  Speaker 3: Thing, so the team, the team didn't change, what changes
• 6:48
  Speaker 3: our way of working? Right? Embracing AI in order to
• 6:57
  Speaker 3: to research and prototype allow us to be much faster,
• 7:00
  Speaker 3: much more efficient, allow us to prototype faster, to explore
• 7:07
  Speaker 3: different avenues, right, and now our job and research is
• 7:12
  Speaker 3: really to iterate fast, to do an exploration and to
• 7:17
  Speaker 3: learn as fast as possible.
• 7:18
  Speaker 2: Right.
• 7:19
  Speaker 3: So it didn't change the team, but changed the way
• 7:22
  Speaker 3: how we work and also the change somehow the dynamics
• 7:26
  Speaker 3: in the work, because if you think about it, the
• 7:29
  Speaker 3: bottleneck for us is shifted from the ability to write
• 7:32
  Speaker 3: code and to build products to more engineering judgment means
• 7:36
  Speaker 3: our ability as a team to work together but also
• 7:39
  Speaker 3: to review what you build and to progress to learn
• 7:45
  Speaker 3: and iter it. Right. So the team is still the same,
• 7:47
  Speaker 3: but the way we work and the criticality of engineering
• 7:51
  Speaker 3: judgment is much more important than before.
• 7:55
  Speaker 2: Intriguing you see, talk to us about your day to
• 7:59
  Speaker 2: day work coding with AI and your experience of that
• 8:03
  Speaker 2: tech from a perspective as an engineer.
• 8:08
  Speaker 4: Yeah, I mean I use growth code daily as as
• 8:11
  Speaker 4: moments and an Indeed, like in the last year, there
• 8:14
  Speaker 4: has been there has been a shift like it's it's
• 8:17
  Speaker 4: kind of revolution for for software engineer, and I think
• 8:20
  Speaker 4: the biggest change here is that coding feels less like
• 8:22
  Speaker 4: starting from scratch like you can and feels more like
• 8:25
  Speaker 4: directing a team a team of agents. Actually it helps
• 8:29
  Speaker 4: a lot with many things like boilerplate and and other things,
• 8:33
  Speaker 4: and especially I found it super useful when you want
• 8:36
  Speaker 4: to dig into a code base which you're not familiar with.
• 8:40
  Speaker 4: But I have to say that since recently, maybe a
• 8:43
  Speaker 4: few a few weeks in the team in Spark team
• 8:46
  Speaker 4: like we are not only using the tools as is,
• 8:50
  Speaker 4: but we are thinking about also the future. So we're
• 8:52
  Speaker 4: trying to push automations to the next level. And basically
• 8:56
  Speaker 4: we're building a software engineer, an agentasically that can do
• 9:01
  Speaker 4: what we do, that can react to issues, that can
• 9:04
  Speaker 4: investigate alert gather contexts, and create prs like completely automatically,
• 9:09
  Speaker 4: and we just review the work basically. And while doing this,
• 9:13
  Speaker 4: I mean the experience doing this, we realized that the
• 9:18
  Speaker 4: that the agents they need more than access to the
• 9:21
  Speaker 4: code and to the repo. They need a way to basically,
• 9:25
  Speaker 4: they need our way to software. So that's why we
• 9:30
  Speaker 4: created like a coding harness where we basically embed all
• 9:35
  Speaker 4: our architecture patterns like testing, strategy, the team knowledge. So
• 9:40
  Speaker 4: basically there was a shift where kind of we're not
• 9:44
  Speaker 4: just using those AI tools, we're building the infrastructure that
• 9:47
  Speaker 4: lets AI agent work reliably inside our engineering system. So
• 9:53
  Speaker 4: the day to day resembles more trying trying to go
• 9:57
  Speaker 4: a bit meta right and empower those agents to perform
• 10:03
  Speaker 4: well and find techniques that that would basically push them
• 10:09
  Speaker 4: to behave the way we want them to behave. So
• 10:11
  Speaker 4: internally we we have this this running joke which is
• 10:15
  Speaker 4: we are automating ourselves, right and and honestly it's it's
• 10:19
  Speaker 4: a little bit true, but hopefully we are automating like
• 10:24
  Speaker 4: the tired, repetitive tasks, low leverage ones, and not the
• 10:28
  Speaker 4: judgment the taste, which I think stay super important. Like
• 10:34
  Speaker 4: when when you're running those those agents we perform that
• 10:38
  Speaker 4: you realize very quickly that they can add a huge
• 10:40
  Speaker 4: amount of complexity super fast. So the ability to differentiate
• 10:45
  Speaker 4: what good from bad, right, and to produce code that's
• 10:47
  Speaker 4: going to be maintainable for the future. I think it's
• 10:50
  Speaker 4: something that's uh, that's that's yeah, still part of the
• 10:54
  Speaker 4: day to day and still going to be part of
• 10:56
  Speaker 4: the day to day in the future.
• 11:00
  Speaker 2: You see you picked up on some really interesting things there.
• 11:06
  Speaker 2: I'm curious, right, because what's it been like as as
• 11:09
  Speaker 2: a coder? You know, no doubt, an exceptional coder to
• 11:14
  Speaker 2: suddenly see something as powerful as this technology. You know,
• 11:18
  Speaker 2: you made the joke about automating yourselves. We're all experiencing
• 11:22
  Speaker 2: that almost whatever our profession, right, we're seeing technology that
• 11:26
  Speaker 2: can match us in some ways out do us, et cetera.
• 11:31
  Speaker 2: What's that been like for you? I guess for what
• 11:34
  Speaker 2: I'm looking for is emotionally.
• 11:38
  Speaker 4: I mean, it's it's very exciting, I have to say,
• 11:42
  Speaker 4: like to be empowered to do more, to explore more,
• 11:46
  Speaker 4: especially if if you have a lot of ideas on
• 11:49
  Speaker 4: what to build in the product, like what to improve.
• 11:52
  Speaker 4: Now the cost of doing so it's it's very low,
• 11:56
  Speaker 4: Like you could just spend a few hours and try
• 11:58
  Speaker 4: to to automate or to build wherever you want to build.
• 12:00
  Speaker 4: So it's like I don't know, having a fast car
• 12:03
  Speaker 4: Ferrari that you're driving.
• 12:05
  Speaker 2: It's just super exciting, madam before Okay, So a lot
• 12:11
  Speaker 2: of successful experimentation has taken place with open claw in
• 12:16
  Speaker 2: the consumer market. Talk about that, Talk about what was
• 12:20
  Speaker 2: useful about it from your perspective and what this could
• 12:23
  Speaker 2: be an imply now for the enterprise.
• 12:27
  Speaker 3: So open cloud is this personal AI agent for the
• 12:32
  Speaker 3: consumer market, and it took us by storm at least
• 12:35
  Speaker 3: it took the consumer market by storm because you had
• 12:39
  Speaker 3: the ability to build an AI agent easily even if
• 12:44
  Speaker 3: you're a non technical user, and allows us to automate
• 12:48
  Speaker 3: tasks for you like very daily tasks. Also interact with
• 12:53
  Speaker 3: it with WhatsApp. And it was a let's say, defining
• 12:57
  Speaker 3: moment this year for both the consumer and also the
• 13:00
  Speaker 3: enterprise because the enterprise market and the enterprise leaders were wondering, Okay,
• 13:06
  Speaker 3: what's the open cloud strategy? What does it mean for us?
• 13:09
  Speaker 4: Right?
• 13:12
  Speaker 3: And if we look at the technical site, open Cloud
• 13:18
  Speaker 3: show that AA agents can be very useful when they
• 13:22
  Speaker 3: have the ability to use the computer and to control
• 13:24
  Speaker 3: the computer right. They have the ability to write code,
• 13:27
  Speaker 3: to use CLIs or the comment line interface, make errors,
• 13:32
  Speaker 3: but also have the ability to retry. And they show
• 13:34
  Speaker 3: that this is very powerful for the consumers that can
• 13:36
  Speaker 3: ANOTO make a lot of tasks. At the same time,
• 13:41
  Speaker 3: the consumer market, as we know, is very different from
• 13:43
  Speaker 3: the enterprise market. In consumer market, if you have autonomy
• 13:49
  Speaker 3: with this agent, you get very powerful toys you can
• 13:51
  Speaker 3: play with and have fun. And the enterprise, if you
• 13:54
  Speaker 3: have this autonomy without having the governance the security, it
• 13:58
  Speaker 3: becomes a risk at scale right, and now the enterprise,
• 14:06
  Speaker 3: if you imagine you have those open cloud means you
• 14:08
  Speaker 3: have thousands of agents and your enterprise environment that are
• 14:10
  Speaker 3: automating tasks for you. You need to think about the policey,
• 14:14
  Speaker 3: you need to think about the authentication, but permission. You
• 14:17
  Speaker 3: also need to think about sand boxing for example, because
• 14:20
  Speaker 3: the agents have the ability that are very powerful because
• 14:22
  Speaker 3: they have the ability to write code right autonomously. But
• 14:26
  Speaker 3: in the enterprise you need sand boxing. It means you
• 14:27
  Speaker 3: need to provide them the environment that's shielded enough from
• 14:32
  Speaker 3: the production environment for them to try and to iterate. Right.
• 14:38
  Speaker 3: So our view has been really local autonomy of this agent.
• 14:41
  Speaker 3: It's very powerful, but for us, for the enterprise, it's
• 14:45
  Speaker 3: not enough. For as we said, okay, it's not enough
• 14:49
  Speaker 3: the local autonomy, so we need to do much more.
• 14:51
  Speaker 3: And we worked, i mean with an exploration and this
• 14:55
  Speaker 3: is there are two elements or two layers that we
• 14:57
  Speaker 3: need to add in the enterprise. The first one is
• 15:01
  Speaker 3: runtime safety layer that allows us to write those to
• 15:06
  Speaker 3: run this agent in the enterprise environment safely means governance, control, control, policy,
• 15:10
  Speaker 3: and security. And also there are there is a very
• 15:13
  Speaker 3: important layer around the enterprise learning layer, meaning providing the
• 15:20
  Speaker 3: layer for the agents to learn from what worked, what failed,
• 15:24
  Speaker 3: and what humans did in order to keep an improving.
• 15:28
  Speaker 2: You seeing what will it take in your view for
• 15:30
  Speaker 2: coding agents to really evolve into reliable enterprise grade it troubleshooters.
• 15:40
  Speaker 4: I believe that, I mean coding agent become enterprise greade
• 15:45
  Speaker 4: it from shooters when when they stop being like this
• 15:48
  Speaker 4: kind of chat about with tools and become part of
• 15:51
  Speaker 4: the real diagnostic system. And we talked about coding right,
• 15:58
  Speaker 4: and one can make the parallel here. I think the
• 16:00
  Speaker 4: trouble shooting it is much lessier than coding. So encoding
• 16:05
  Speaker 4: usually I have repos some tests CI in enterprise it
• 16:09
  Speaker 4: t it's it's much more complex, like there is the OS,
• 16:13
  Speaker 4: the driver, the network security which moment was talking about,
• 16:18
  Speaker 4: and a toy ton of noisy telemetry. So I mean
• 16:23
  Speaker 4: the hard part is not to make I think the
• 16:26
  Speaker 4: model smarter. It's more helping helping it, like collect the
• 16:30
  Speaker 4: right context, filter out noise, reason about cause and effect,
• 16:35
  Speaker 4: and take actions and actions that are are safe. So
• 16:39
  Speaker 4: I think many things matter, and one of them, for example,
• 16:43
  Speaker 4: is evaluations. So we need to be able to test
• 16:47
  Speaker 4: agent on on realistic trouble shooting journey, not just like
• 16:53
  Speaker 4: some of benchmark and to be able to say, okay,
• 16:56
  Speaker 4: did did the agent ask the right question? Did you
• 16:58
  Speaker 4: choose the right tools, did it avoid risky actions? Did
• 17:02
  Speaker 4: it know when to escalate or what to escalate? And
• 17:06
  Speaker 4: so this relates also to a bit to the context management.
• 17:09
  Speaker 4: So you cannot just dump every log and devise details
• 17:14
  Speaker 4: into the model. The system has to. Yeah, it has
• 17:18
  Speaker 4: to to summarize to active context management, so to know
• 17:22
  Speaker 4: what to ignore and to know what matters. And I
• 17:26
  Speaker 4: mentioned a little bit about causal causal thinking before. It's
• 17:31
  Speaker 4: because and we've experienced it like with Spark, it's very
• 17:35
  Speaker 4: easy for those models to find correlations. Uh. So I
• 17:38
  Speaker 4: don't know a driver change yesterday, but that does not
• 17:42
  Speaker 4: mean it caused the issue. Right, So a good agent
• 17:45
  Speaker 4: need to you know, form a hypothesis, look for evidence,
• 17:49
  Speaker 4: rule things out, uh and and and fire find proper
• 17:53
  Speaker 4: proper causations. So there is a lot of work also
• 17:55
  Speaker 4: to to both in development but also in the evaluation
• 17:59
  Speaker 4: to drive this behavior and obviously safe actions. So the
• 18:05
  Speaker 4: ability to allow agent to act on the device, to
• 18:08
  Speaker 4: be able to travel through, to be able to remediate
• 18:10
  Speaker 4: in a safe way. That's very important to approved permission actions.
• 18:19
  Speaker 4: And the last piece, which I think is one of
• 18:22
  Speaker 4: the most important one is learning. So and if I
• 18:27
  Speaker 4: take the example of Spark, every conversation in Spark should
• 18:31
  Speaker 4: should make the system better and actually is making the
• 18:34
  Speaker 4: system better to capture what worked, what failed? What maybe?
• 18:39
  Speaker 4: How do I turn something into into a playbook, a
• 18:42
  Speaker 4: test case? And over time the systems become more reliable
• 18:48
  Speaker 4: because it's not just answering on general knowledge, it's really
• 18:51
  Speaker 4: acculating this operational experience from the enterprise itself. So from
• 18:56
  Speaker 4: the ground it starts to reconnect patterns that repeat themselves
• 19:02
  Speaker 4: or to avoid past mistakes and to improve an the way.
• 19:09
  Speaker 4: So I think that feedback loop is is what turns
• 19:13
  Speaker 4: like the agent from one one shot troubleshooter, let's call
• 19:17
  Speaker 4: it like this into a system that really compound knowledge
• 19:22
  Speaker 4: basically and learn learn, learn from from from experience and
• 19:25
  Speaker 4: from history.
• 19:28
  Speaker 2: In a word, you're seeing in a world rather where
• 19:32
  Speaker 2: AI agents are automating much of our work under human supervision.
• 19:38
  Speaker 2: What happens to u X or indeed O S.
• 19:43
  Speaker 4: I mean I recently went to a conference and saw
• 19:49
  Speaker 4: a talk from I do I do Solomon. He is
• 19:52
  Speaker 4: the creator of of agent Craft and m c p
• 19:55
  Speaker 4: u I. He's also co creator of of m c
• 19:58
  Speaker 4: p apps. So, and his point was, I mean he
• 20:00
  Speaker 4: made the agent craft, which is a solution to a
• 20:05
  Speaker 4: problem which now we run more agents in parallel and
• 20:08
  Speaker 4: the human becomes the bottleneck. Right, And agent Craft is
• 20:12
  Speaker 4: actually a game. So it's a mix between warcraft and
• 20:15
  Speaker 4: agent So that's why he called it agent craft. So
• 20:18
  Speaker 4: it's a game like interface that basically allows the human
• 20:25
  Speaker 4: to control agents in the game. Right, So when I
• 20:30
  Speaker 4: don't know when when the agent like needs your intervention,
• 20:34
  Speaker 4: you would see a pop up in a game like
• 20:36
  Speaker 4: interface or when a human needs to intervene for specific
• 20:40
  Speaker 4: action for example. So I don't think that enterprise software
• 20:43
  Speaker 4: will literally look like a game, but the metaphor is right,
• 20:47
  Speaker 4: like the humans become become the orchestrator, and the agent
• 20:51
  Speaker 4: will generate options and human will decide like directions, quality, accountability,
• 20:57
  Speaker 4: et cetera. So I don't know what the yeah, what
• 21:01
  Speaker 4: the UX would look like, but I think it would
• 21:03
  Speaker 4: look like something like this where the human just manages
• 21:06
  Speaker 4: the fleet of agents in an interest that might look like.
• 21:09
  Speaker 5: Like a gig engage pop ups about it just continued
• 21:15
  Speaker 5: all day long. Guys, tell us, tell us some stories
• 21:18
  Speaker 5: about what it's been like with these next generation colleagues
• 21:22
  Speaker 5: you've been you've been nurturing over and they could be frightening, funny, inspiring, illustrative.
• 21:31
  Speaker 4: I mean, I have I have a story that that
• 21:35
  Speaker 4: that happened to us recently in in a QA environment
• 21:39
  Speaker 4: while we were preparing a model upgrade, and I mean
• 21:44
  Speaker 4: functionally spark and the agent that powers that looked good,
• 21:48
  Speaker 4: right it understood the task, like retrieved the right context
• 21:52
  Speaker 4: and guided the user through the troubleshooting flow. But then
• 21:55
  Speaker 4: we we noticed the behavior regression. So when do users
• 22:01
  Speaker 4: were confused or repeating themselves, for example, the agent started
• 22:05
  Speaker 4: saying things like I already told you this multiple times, Okay,
• 22:09
  Speaker 4: I'm repeating myself. You need to do x so, and
• 22:12
  Speaker 4: they even started to be rude, like even sometimes in
• 22:15
  Speaker 4: sogemb this is this is true, like this is a
• 22:19
  Speaker 4: true story, and I mean technically the answer might have
• 22:25
  Speaker 4: might have been correct, but the behavior was not acceptable,
• 22:28
  Speaker 4: right so, and an IT agent often like is often
• 22:33
  Speaker 4: like helping someone who who is already frustrated because their
• 22:36
  Speaker 4: VPN or laptop or something doesn't work right, so he
• 22:40
  Speaker 4: has to be patient, he has to be professional. So
• 22:42
  Speaker 4: that was I think that story made like a strong
• 22:46
  Speaker 4: reminder that production agent are like stochastic software. We don't
• 22:49
  Speaker 4: control them, and a model like upgrade can change like
• 22:52
  Speaker 4: not only a curacy, but it can change. Don't judgment
• 22:55
  Speaker 4: to use edge cases. So we built sure this and
• 23:00
  Speaker 4: we double down on our guardrail evaluations and adversarial example,
• 23:07
  Speaker 4: repetitive user, confused users, So we had a bunch of
• 23:10
  Speaker 4: simulation of different types of users to to guard rail
• 23:14
  Speaker 4: this against this happening again. Actually, so I mean the
• 23:19
  Speaker 4: listen was that evaluating agents is not just about like
• 23:22
  Speaker 4: whether you solve the task or not, it's also about
• 23:24
  Speaker 4: whether they remain safe, they remain respectful, trustworthy, And that's
• 23:30
  Speaker 4: that's that's that was actually lesson learned.
• 23:33
  Speaker 2: Let's unleash the rude version of the world, guys.
• 23:37
  Speaker 3: I think that was my idea as well. You know, yes,
• 23:39
  Speaker 3: seen the rude version of of your agents could be
• 23:44
  Speaker 3: even even like a feature because today if you if
• 23:48
  Speaker 3: you use GPT, there's so psycophantic, they're so trying to
• 23:52
  Speaker 3: please you that becomes very annoying. Yeah right, you're right,
• 23:57
  Speaker 3: and sometimes I rather have a button with the rude
• 23:59
  Speaker 3: version this discussion, and to be brutally honest, I think
• 24:04
  Speaker 3: it would be our buy this agent.
• 24:06
  Speaker 4: The funny thing is that we hired someone recently in
• 24:09
  Speaker 4: the team who was working at another company and they
• 24:12
  Speaker 4: had like a very similar problem in production for their clients,
• 24:16
  Speaker 4: like the agent after an upgrade started the insulting clients
• 24:20
  Speaker 4: like was Gemini and cause a big incidence and many problems.
• 24:25
  Speaker 3: So and and you know, just to go it's a
• 24:28
  Speaker 3: bit deeper because I think it's a very interesting point,
• 24:30
  Speaker 3: the fact that agents are trying to please you and
• 24:36
  Speaker 3: avoid to contradict you, except if you prompt them for that.
• 24:43
  Speaker 3: It's also something that's hindering learning right because when you
• 24:47
  Speaker 3: when you want to learn, you want to you want
• 24:50
  Speaker 3: to be challenged, you know, and you want some time
• 24:54
  Speaker 3: to be confronted with the truth or another perspective. And
• 24:58
  Speaker 3: I think the fact that they're trying to please us
• 25:00
  Speaker 3: and they are psycophantic, it's a it's a it's an
• 25:05
  Speaker 3: obstacle to learning right. And what's what's funny, Tom and
• 25:09
  Speaker 3: I will I will close after the giggy parentheses. It's
• 25:15
  Speaker 3: a this is the case because as agents, they have
• 25:19
  Speaker 3: been trained to be like this, right because in their
• 25:22
  Speaker 3: post training where where we basically align them with human
• 25:25
  Speaker 3: preference humans when they rate the answer of CHGPT or clothe,
• 25:31
  Speaker 3: they rather have something that is incorrect but it's pleasing
• 25:35
  Speaker 3: them than something that is unpleasant but true. Right, And
• 25:40
  Speaker 3: this alignment phase brings this agent to be more and
• 25:43
  Speaker 3: more in the Okay, I will please you and I'll
• 25:46
  Speaker 3: tell you. I'll never tell you're wrong. Rather than telling you, okay,
• 25:49
  Speaker 3: here you're wrong, I rather i recommend going another way
• 25:52
  Speaker 3: or another hypothesis.
• 25:54
  Speaker 2: I'm so interested in that, so added interest.
• 25:55
  Speaker 3: Guys.
• 25:56
  Speaker 2: So when you so, when you're trying to recalibrate the
• 25:59
  Speaker 2: l l M aspect, right, but when you're trying to
• 26:01
  Speaker 2: recalibrate the LLM in a behavioral direction, I understand that
• 26:07
  Speaker 2: you're going to give you're going to feed it with
• 26:09
  Speaker 2: new responses. But what does that process look like?
• 26:12
  Speaker 3: So you know, today, as we are integrating LMS and
• 26:16
  Speaker 3: our products, I mean the let's say the pre training
• 26:19
  Speaker 3: and post training has already been done. It means they
• 26:22
  Speaker 3: already encoded the preference of the LM and the LLM weights.
• 26:26
  Speaker 3: Right in the LM break Our only way of influencing
• 26:30
  Speaker 3: the behavior of the LM is what we call in
• 26:33
  Speaker 3: context learning, is basically by injecting in the context and
• 26:36
  Speaker 3: telling it, okay, try to behave in such a manner. Right,
• 26:40
  Speaker 3: But it's already too late for us. We cannot make it.
• 26:44
  Speaker 3: We cannot fundamentally change how the LLM is because we
• 26:47
  Speaker 3: cannot modify the weights. Something cannot. We could, but we
• 26:51
  Speaker 3: don't because there are other side effects. So today we
• 26:55
  Speaker 3: take the LM as is and we prompt it right
• 26:59
  Speaker 3: to behave in a certain manner. If you want to
• 27:02
  Speaker 3: fundamentally change how the ll M is answering and the
• 27:05
  Speaker 3: LM personality, you need to modify the weights. And modifying
• 27:09
  Speaker 3: the weights it means providing it with additional examples of
• 27:16
  Speaker 3: desired responses and go on this fine tuning phase, which
• 27:22
  Speaker 3: is much more heavy in engineering, and it means it's
• 27:25
  Speaker 3: much more risky because you can also impact other aspects
• 27:29
  Speaker 3: of the LLM as well.
• 27:31
  Speaker 4: And true we call this actually catastrophic forgetting. So when
• 27:35
  Speaker 4: you change the weight and you post trained, but you
• 27:38
  Speaker 4: forget previously acquire tasks and previously acquire knowledge, which is yeah,
• 27:45
  Speaker 4: as the most said.
• 27:47
  Speaker 3: But you can definitely. If you come back to the
• 27:49
  Speaker 3: first example, if you want an LLM that is blunt,
• 27:53
  Speaker 3: telling the truth, almost rude, you can do it on
• 27:56
  Speaker 3: the post training. It means you show it examples of
• 27:59
  Speaker 3: questions and answer and look this answer that is more honest,
• 28:04
  Speaker 3: more direct. I want you to favor it, and it
• 28:06
  Speaker 3: will basically modify the weight in order to provide those
• 28:09
  Speaker 3: answers today, you know, because we want to get engagement
• 28:14
  Speaker 3: from the user and because as humans we rather have
• 28:16
  Speaker 3: a pleasant answer. There are no incentives for the company's
• 28:21
  Speaker 3: training and proposing this LM to go that way, at
• 28:23
  Speaker 3: least the brutal, honest way.
• 28:28
  Speaker 2: Like you say, in the workplace, it becomes it becomes imperative.
• 28:32
  Speaker 2: But there's great a kunda an in the in the
• 28:35
  Speaker 2: consumer l M space, No doubt my habits. Something you
• 28:39
  Speaker 2: said prior to recording about the view that agent failures
• 28:44
  Speaker 2: aren't model failures but context failures. Elaborate on what you
• 28:49
  Speaker 2: meant by that and what that means in practice.
• 28:52
  Speaker 3: Yeah, Usually when you have like an agent failure, we
• 28:56
  Speaker 3: tend often to blame the model or the LLM that's
• 29:00
  Speaker 3: spowering those agents, right, But what we miss is is
• 29:07
  Speaker 3: that most of the failure, I would say, come from
• 29:09
  Speaker 3: the missing context. So the context is basically what we
• 29:11
  Speaker 3: inject in the memory of the agent when they need
• 29:14
  Speaker 3: to perform a task. Right, And I would argue, like
• 29:19
  Speaker 3: in consumer but also enterprise, I mean the context context
• 29:23
  Speaker 3: is everything, right, because to perform a task, let's say,
• 29:27
  Speaker 3: for example, for Spark performing automating the task for it,
• 29:31
  Speaker 3: the agents need to understand, Okay, what's the device state,
• 29:36
  Speaker 3: what are the company tools, what are the policies, what
• 29:39
  Speaker 3: is the user context, historical outcomes? Right, the skills that
• 29:44
  Speaker 3: are required to perform the task. And this context becomes
• 29:50
  Speaker 3: the most crucial part if you want to have useful
• 29:52
  Speaker 3: agents automating task in the enterprise. And we did a
• 29:57
  Speaker 3: small it's not a smaller. We did an exploration around this,
• 30:00
  Speaker 3: around how we can build the right context for the
• 30:01
  Speaker 3: agent and the enterprise right. And we explored two ways
• 30:05
  Speaker 3: to build this context. First one, we expose the agent
• 30:11
  Speaker 3: to humans. It means we show it examples of humans
• 30:14
  Speaker 3: solving a task, and the agent could observe humans solving
• 30:18
  Speaker 3: a task and learn from it. This is the first
• 30:21
  Speaker 3: way we did it. And the second way was we
• 30:25
  Speaker 3: created what it called synthetic traits and a sandbox environment.
• 30:28
  Speaker 3: It means we let the agent go free and explore
• 30:33
  Speaker 3: given web application and a protected environment, right. And the
• 30:38
  Speaker 3: goal of those two approaches is to allow the agents
• 30:42
  Speaker 3: to learn from those examples. It's like you have an
• 30:47
  Speaker 3: intern joining the company and for it to learn. First
• 30:50
  Speaker 3: version is do you let it look at the colleagues
• 30:52
  Speaker 3: how they do the work right? And the other way
• 30:56
  Speaker 3: is you give it like a safe environment when the
• 30:58
  Speaker 3: intern can explore and make mistakes right. And we showed
• 31:03
  Speaker 3: empirically we measured after this the two approaches, how accurate
• 31:08
  Speaker 3: is the A agent in performing task? And we showed
• 31:11
  Speaker 3: that for very specific tasks in the enterprise, when you
• 31:17
  Speaker 3: are able to automatically build this context, either by observing
• 31:20
  Speaker 3: humans or by exploring in the sundback's environment, you can
• 31:25
  Speaker 3: improve the accuracy by a significant percentage. Like for sertain tasks,
• 31:30
  Speaker 3: we see an increase by seventy percent in accuracy just
• 31:33
  Speaker 3: by allowing the I agent to be exposed to this
• 31:37
  Speaker 3: information and build the context. Right.
• 31:39
  Speaker 2: Crazy, it's it's so interesting having you guys on AI
• 31:44
  Speaker 2: labs is fascinating. Muhammed, I think a four to fifth
• 31:49
  Speaker 2: visit is inevitable. Maybe in six months time when everything
• 31:52
  Speaker 2: is kind of completely changed again. But I thinking, speaking
• 31:57
  Speaker 2: of the future, let's ran things off with this question,
• 31:59
  Speaker 2: open question for you both. You can both answer one
• 32:02
  Speaker 2: of you as you like, but what does product development
• 32:05
  Speaker 2: look like in your mind's eye when you when you
• 32:07
  Speaker 2: cast it forward two three years?
• 32:10
  Speaker 3: Yeah, I mean what we see first, what we see
• 32:13
  Speaker 3: around is that boundaries today are let's say, are more
• 32:20
  Speaker 3: and more blurry between what a product manager does, what
• 32:22
  Speaker 3: the engineer does, and what the designer does. Right this,
• 32:25
  Speaker 3: I see it working every day that we see that
• 32:29
  Speaker 3: the product managers are prototyping more and more means they
• 32:32
  Speaker 3: don't necessarily need an engineer in order to build a prototype.
• 32:35
  Speaker 3: Designers are able to day to work to create working
• 32:38
  Speaker 3: interfaces and working UIs and engineers also they have expanded
• 32:45
  Speaker 3: their capabilities what they could build and also the kind
• 32:49
  Speaker 3: of the kind of discussions we can they can have
• 32:52
  Speaker 3: with PM and designers. So what if I had to
• 32:55
  Speaker 3: get what I would see is small our teams will
• 33:00
  Speaker 3: be able to do more right. But what's important for
• 33:05
  Speaker 3: us is, and I think it's the point that you've
• 33:08
  Speaker 3: seen mentioned at the beginning, it's a very important point.
• 33:11
  Speaker 3: Strong judgment and taste are becoming more and more important
• 33:15
  Speaker 3: as skills, right, whether you're an engineer or PM. I mean,
• 33:19
  Speaker 3: those kids are the most important one as we are
• 33:21
  Speaker 3: automating more and more of the job and we have
• 33:23
  Speaker 3: more and more AI agents being an integral part of
• 33:26
  Speaker 3: the tip right. And in addition to that, I think
• 33:30
  Speaker 3: the usual skills that remain important is curiosity, the ability
• 33:35
  Speaker 3: to learn right, taste. As I said, so my advice
• 33:42
  Speaker 3: for engineers PM is really to experiment as much as possible,
• 33:46
  Speaker 3: find your AI native workflow, how you can integrate AI
• 33:50
  Speaker 3: in workflow, and obviously don't turn off your brain when
• 33:54
  Speaker 3: you do that, right, and keep the taste and keep learning.
• 34:00
  Speaker 4: Yeah, thanks more, I completely relate to it. I I
• 34:02
  Speaker 4: also think that I mean obviously, I mean and this
• 34:05
  Speaker 4: ship has as sailed already, Like the tools are making
• 34:07
  Speaker 4: as much more productive, and they they are also changing
• 34:11
  Speaker 4: a little bit of what like this good product like
• 34:14
  Speaker 4: means and it's definitely now cheaper to produce code to
• 34:18
  Speaker 4: produce new feature. And as more said, like the differential
• 34:21
  Speaker 4: differentiator now is really taste, and I think that's a
• 34:26
  Speaker 4: great enterprise product. It's not just a list of features
• 34:31
  Speaker 4: because I mean, we can build the tons of tons
• 34:33
  Speaker 4: of features. Now it's actually those thousands of like small
• 34:37
  Speaker 4: decisions that do compounds, right, the so being obsessed with
• 34:42
  Speaker 4: the details and to build the best product for for
• 34:45
  Speaker 4: for for for our customers. I think this is, this is,
• 34:49
  Speaker 4: this is what's gonna different like allow different companies to
• 34:53
  Speaker 4: to to strive actually and yeah, so uh, this is
• 35:00
  Speaker 4: this is also what's what's what's what's what's driving us
• 35:04
  Speaker 4: at the moment?
• 35:07
  Speaker 2: Phenomenal judgment and taste interestingly the same kind of language.
• 35:11
  Speaker 2: But some of my friends in like the visual profession,
• 35:14
  Speaker 2: shall we say, you know, videography, graphic design, where where
• 35:18
  Speaker 2: where suddenly there is a huge democracy of what people
• 35:21
  Speaker 2: can democratic element and what people can do. Judgment and
• 35:25
  Speaker 2: taste aren't going anywhere, you know. And as much as
• 35:29
  Speaker 2: anybody can now mock up a website, a web page,
• 35:32
  Speaker 2: a banner or many of us shouldn't, you know. But
• 35:35
  Speaker 2: it is funny, very funny, very provocative to hear the
• 35:38
  Speaker 2: same terminology entering in such a distinct.
• 35:40
  Speaker 3: Because if you think about the term, because beyond code,
• 35:44
  Speaker 3: I mean, generating content is becoming much and much much
• 35:48
  Speaker 3: cheaper than before, So it means anyone can generate content,
• 35:52
  Speaker 3: and that's why the importance of this and judgment is
• 35:59
  Speaker 3: much more than before, because it's scritical for you.
• 36:04
  Speaker 2: So maybe we're not automating ourselves after all, gentlemen. Hey,
• 36:09
  Speaker 2: hopefully not. But hopefully we'll still have all have jobs
• 36:12
  Speaker 2: in six months time. And if we do, AI Lab's
• 36:15
  Speaker 2: got to come back on and catch up because this
• 36:17
  Speaker 2: has been absolutely wonderful to have you and thanks for
• 36:19
  Speaker 2: coming for the first time you've seen. Thanks for coming back, Muhammad,
• 36:22
  Speaker 2: It's been great, guys, Thanks Tom for having guys.
• 36:25
  Speaker 4: Thank you Dom.
• 36:28
  Speaker 1: That's it for this week's Deck Show. We'll be back
• 36:31
  Speaker 1: next week, and if you don't already, be sure to
• 36:33
  Speaker 1: follow the show so you don't miss an episode until
• 36:36
  Speaker 1: next time.