Latest / Agentic Conversations (formally mlops.community) / [Exclusive] QuantumBlack Round-table // Gen AI Buy vs Build, Commercial vs Open Source

Transcript

[Exclusive] QuantumBlack Round-table // Gen AI Buy vs Build, Commercial vs Open Source · Agentic Conversations (formally mlops.community)

• 0:05
  Hello and welcome everyone. We are here for a very special
• 0:13
  session and I'm so excited to be able to share this with you
• 0:19
  right now. I would love to know before we
• 0:22
  even get started and do anything, where you all are
• 0:27
  calling in from. I love getting to see how global
• 0:32
  this community is and today is no exception to that.
• 0:37
  We've got an we've got an exceptional chat that's about to
• 0:42
  come up, and if you know anything about how we do this,
• 0:46
  you probably know now is the moment that we get to have a
• 0:52
  little bit of a guitar intro. And I'm going to bring out our
• 0:57
  three friends from Quantum Black and McKinsey.
• 1:03
  But I'm gonna do it in a little bit of a different fashion than
• 1:08
  you probably are used to. We're here today to talk all
• 1:11
  about Gen AI and if you should be building or buying, so I'm
• 1:16
  going to use that as some fodder for the song that I want to
• 1:22
  create on the fly here. But I also love getting prompted
• 1:27
  by you all, so if you have some different ideas of what I should
• 1:37
  sing about, feel free to drop that in the chat and I will do
• 1:41
  my best to musically inspired improvised jam session right
• 1:48
  now. And I'm going to make it quick
• 1:49
  because we've got these incredible people waiting to
• 1:54
  jump in. I imagine you can hear that.
• 1:58
  Can you give me a thumbs up? If you hear that team, you hear
• 2:02
  the guitar. Alright, so this is where it
• 2:05
  gets dangerous. a lucky me you are are in the
• 2:58
  right place. You came to the right show.
• 3:02
  We're going to get it started right now.
• 3:05
  Let me bring out our guests of Bonner.
• 3:08
  We've got Moh. He's here calling in from
• 3:12
  London. Where you at MO?
• 3:16
  Where you have no where you have.
• 3:18
  Bring him on to the stage. There he is and Next up we've
• 3:23
  got now your not your coming straight from the UK.
• 3:30
  Last but not least, we've got Alonia Power ready to start this
• 3:40
  round table. What's happening folks?
• 3:46
  I just can't get enough of that intro, so thank you Dimitrios.
• 3:49
  It's so great to be here. That was fantastic dude.
• 3:53
  That's we should have recorded that I.
• 3:56
  Put it on Spotify. Don't you worry, it will be
• 3:59
  going up in a week and we'll send everyone a link to stream
• 4:04
  it, you know? But we didn't come here just for
• 4:07
  that. I know we came for all kinds of
• 4:10
  incredible things around building or buying in the Gen AI
• 4:13
  space. And since I took up a little bit
• 4:17
  of extra time on that intro, I want to jump right into it.
• 4:21
  I know there's lots of things that we can be talking about,
• 4:24
  but we should probably level set with just what we mean when
• 4:29
  we're talking about building or buying for Gen AI models are out
• 4:34
  there. What does this look like?
• 4:36
  How do you see this space? And Moe, I'll kick it off to you
• 4:39
  first to let us or lead us and bring us towards the light man.
• 4:45
  Where are we at? Yeah.
• 4:47
  No, definitely. Thank you Metric.
• 4:50
  Hello Vera, my name is Mohammed Abbas, said Amal.
• 4:53
  I'm senior solution architect, Quantum Black.
• 4:56
  I spend a lot of my time helping companies and organisations
• 5:01
  learn about analytics, how to deploy analytics, how to make
• 5:04
  the best of it, and how to develop in production at scale.
• 5:09
  I'm joined by my 2 colleagues looking Nair and Ilona.
• 5:14
  Of course. Hello everyone and thanks so
• 5:16
  much Moe. Hi everyone.
• 5:17
  Alona. Look Manova, I am based in New
• 5:19
  York. I'm so honoured to be here
• 5:21
  today. I have a dual role here at
• 5:22
  Mackenzie. So on the one side of it, I am
• 5:25
  managing council for Mackenzie Digital where I'm one of the
• 5:28
  leaders of our global technology legal team and I work with you
• 5:32
  know really anything related to emerging tech, any legal
• 5:35
  regulatory issues and advising our in-house, our internal
• 5:39
  consultants like the wonderful group that you see here today.
• 5:42
  And on the other side of it, I'm the Head of innovation for
• 5:44
  McKinsey Legal for our legal function globally.
• 5:47
  And that's where I get to really think about some of the the, the
• 5:50
  points that we're thinking about today.
• 5:51
  The builder by thinking about how we adopt and pilot and
• 5:55
  employ and implement legal technologies, thinking about
• 5:58
  generative AI and what it means for our workflows, for how
• 6:02
  really for how we do our everyday work and how we deliver
• 6:05
  our services more broadly. Mayor, Over to you.
• 6:09
  Fantastic. So hi everyone.
• 6:11
  I'm Mayor Khan. I'm a partner Quantum Black,
• 6:14
  based out of London. I've always on one topic, which
• 6:17
  is really how do you scale artificial intelligence?
• 6:20
  And that's not just running one or two models in production, but
• 6:23
  taking organisations on the journey to get to hundreds or
• 6:25
  thousands over the last year. I've also inherited mother that
• 6:29
  Jenny I'd stuff and for my sins that's how do I take this great
• 6:33
  amazing technology that Mo is going to introduce.
• 6:36
  But really make it real. And when I say make it real,
• 6:40
  there's lots of decisions that need to be made by organisations
• 6:42
  and we'll touch on some of the build versus buy and how she
• 6:45
  even thinking about it and this rate of change of what's going
• 6:49
  on in the industry. But super happy to be here with
• 6:52
  you all today. I think it's it's exciting
• 6:54
  times. We've got a lot of people who
• 6:56
  are interested in this topic ChatGPT started it all off and
• 7:01
  you know took the genie out of the bottle.
• 7:03
  We we see huge adoption of kind of the general I from
• 7:08
  researchers to public adoption and a huge spike or to millions
• 7:13
  of users. It's been discussed across all
• 7:15
  channels and mediums and 2023 was a special year for it.
• 7:20
  A huge number of models being released almost weekly if you
• 7:25
  count and and it's gone beyond just it's not text, it's gone
• 7:30
  beyond the text it's actually text images, videos and audio.
• 7:35
  And we've also seen many trusted and tools that we use day-to-day
• 7:41
  are getting intelligent and operating systems and starting
• 7:44
  to integrate a lot of these capabilities in it.
• 7:48
  And all of that brings a lot of questions and to to the people
• 7:53
  that we work with to companies and to the individuals we have
• 7:56
  here is that what does it mean for us?
• 7:58
  How do we use it? How do we interact with it and
• 8:01
  what does build versus buy, what is open source versus commercial
• 8:06
  and what do we do. The first thing I wanted to to
• 8:10
  to start us off is to ask Naya maybe what kind of models do we
• 8:13
  have available and to hear from you?
• 8:17
  And maybe this is just about framing and we'll we'll have a
• 8:20
  discussion, but it's good to get the framings right.
• 8:22
  And and I I'd buck it up, models in two categories.
• 8:26
  You've got the closed type models, ones that you call by
• 8:29
  API that are probably commercial in nature, things like Open AI
• 8:34
  or even Google's Gemini as an example.
• 8:38
  So I was like, why are called closed models?
• 8:39
  Because they're behind an API. Don't know what's going on.
• 8:43
  It's a black box boy, Ted. Suburbs.
• 8:45
  And it's on someone else's kit, arguments sake.
• 8:49
  Then you've got these open models.
• 8:50
  I know some people call them open source, but we can debate
• 8:53
  on the terminology. But let's call it open for time
• 8:55
  being. There's for example, you have
• 8:58
  mistrials, Mr AI, Mistral or mixed role recently, That's
• 9:03
  Apache too. You can use that.
• 9:04
  You can take that off the shelf and use that yourself and you
• 9:06
  don't worry about third party technology.
• 9:10
  Innovation Institute Falcon is another one Met aslama too,
• 9:14
  which is really, really popular. But probably not.
• 9:17
  Is not a patchy, has a certain commercial things you need to
• 9:20
  think about or restrictions. And if you go look at a hugging,
• 9:24
  hugging face, I mean like there's about 470,000 models up
• 9:28
  there at last count when I looked earlier.
• 9:32
  That's huge. There's lots of great things and
• 9:34
  so I'd back in two parts. Is the closed models and so it's
• 9:37
  not always open AI or GPT, that's a burst being I want to
• 9:40
  just point out. And then there's a tonne of
• 9:42
  stuff on open models and things that you could host yourself and
• 9:45
  do stuff with. That's how I'd back it up, the
• 9:48
  different types of and this is just really to to frame the the,
• 9:52
  the, the bill by would you use commercial, do you use open
• 9:55
  etcetera etcetera. Now I think there is another
• 9:58
  part of this like I think like how companies are are using
• 10:02
  these type of models and so I I would think about three
• 10:07
  different categories. But the first one you can tend
• 10:09
  to say look this is not something part of this
• 10:11
  discussion. There is existing software or
• 10:13
  tools that are adding generation Gen AI type capabilities or
• 10:17
  these large language model capabilities today.
• 10:19
  So Microsoft if you've got Windows 11 you will start to see
• 10:22
  copilots appear and eventually if you buy new laptop to start
• 10:26
  to see your keyboard key on your keyboard which is a code pilot
• 10:30
  key. Salesforce, SAP, GitHub to a
• 10:33
  degree all of these things are are adding LLM capabilities.
• 10:37
  So you might as well just this part that.
• 10:39
  So really the the the discussion comes down to to two parts,
• 10:41
  which is the do I buy a product or solution off the shelf and
• 10:48
  which has an LLM inside it or is uses an LM so that the product
• 10:53
  and that has two parts, the product or people use and then
• 10:56
  the engine, the LLM part, the part of the solution.
• 10:58
  You don't get choice of which LLM but it's there and this
• 11:01
  could be things like chat bots or text editing, video editing
• 11:04
  software. There's lots of things that
• 11:06
  exploded onto the scene. So these are buy off the shelf
• 11:09
  and you have everything. Then there's the other part of
• 11:11
  this which is you might want to build something custom for your
• 11:14
  organisation and and this is where you have choices, choices.
• 11:19
  Do I use these closed models we talked about earlier or do I use
• 11:22
  these open mode. So I just wanted to make sure we
• 11:24
  frame every everything on some of these these buckets and
• 11:27
  categories. But we'll we'll discuss I guess
• 11:29
  a lot around the do you buy something off the shelf and
• 11:32
  everything inside and you don't get a choice in the model and
• 11:35
  which model using how it does stuff behind it.
• 11:37
  So it's a black box, but it's software you can just get up and
• 11:39
  running quickly or do you start to build stuff inside your
• 11:43
  organisation And then how do you choose between different models
• 11:46
  and how do you make trade-offs and on a lot of these things.
• 11:49
  And this is where I think our loader will add light a little
• 11:51
  bit on on risk and be going and some of the consideration.
• 11:55
  Yeah. And there I'd I'd love.
• 11:56
  I think that that's really clear and that's really helpful to
• 11:59
  understand. And one thing that I, I wanted
• 12:00
  to just ask you a bit more about and this is where the, the, the
• 12:05
  risk considerations become really important, right.
• 12:07
  Because the first question that I think comes up at least in my
• 12:09
  experience in in these conversations is, well, you know
• 12:12
  what about the data security and issues or what about the
• 12:15
  confidentiality issues. And I think it's really
• 12:18
  important to understand the distinction between something
• 12:22
  that's closed in terms of being an enterprise model versus
• 12:25
  something that's open, right. So data flows, how does your
• 12:28
  data work when you're using any of these models?
• 12:30
  What kind of offerings can you get with any of these models,
• 12:34
  whether you're building or you're buying and how should you
• 12:36
  think about that? Because I think one
• 12:38
  organisations are thinking about this and when those in charge of
• 12:42
  this are thinking about that within organisations, right.
• 12:45
  There's really, there's really a focus on protecting that
• 12:49
  organisation data but there's also an equal parts of focus on
• 12:52
  getting the best and the the best form and the optimised
• 12:56
  generated insights from both that data and maybe from
• 13:00
  external sources too. And so I'm wondering how you
• 13:03
  think about this and will please feel free to chime into.
• 13:05
  You know, how do you understand closed enterprise versions of
• 13:09
  various offerings or or you know what various models can provide
• 13:12
  and what, what do you see in terms of what's available today?
• 13:17
  Yeah, I mean look before I answer I I think it's also good
• 13:20
  to be for you to kind of shed light and you spend a lot of
• 13:23
  time and putting your introduction aside but with
• 13:26
  clients and it's probably the first thing that clients are
• 13:30
  asking about. So maybe shed a little bit light
• 13:33
  on it because I think it helps. And the reason I'm asking about
• 13:36
  this is that traditionally if we talk about you know IT
• 13:40
  procurement or build versus buy and this, there's traditional
• 13:44
  frameworks for deciding. But with this technology and the
• 13:49
  way that this integrates into an existing enterprise, we've
• 13:51
  inside an organisation, they're all other risks which you don't
• 13:58
  typically have with some of the other traditional technologies
• 14:01
  we have today in enterprises. And this is what worries our
• 14:03
  customers maybe shed a little bit light on that then we can
• 14:05
  use that as an anchor of you know the build versus buy
• 14:08
  traditional frameworks etcetera, etcetera.
• 14:11
  Yeah, absolutely. So I think what's been
• 14:14
  interesting to me is really is probing what this question means
• 14:18
  for different organisations and for for those who ask it and are
• 14:21
  worried about it. And I think right, we need to
• 14:24
  understand how this comes together in different ways.
• 14:27
  So when you're thinking about an enterprise version, the way I
• 14:29
  think about it is really as just a closed circle.
• 14:32
  So if you're taking, if you're taking a capability, right, an
• 14:36
  LM driven capability and it's an enterprise version of that
• 14:40
  capability, you're essentially just working within that circle.
• 14:43
  So your data, whether it's on Prem or cloud based, whatever it
• 14:46
  might be, however it's hosted, it's not leaving your
• 14:50
  enterprise, it's not going outside and you're not drawing
• 14:53
  information from the outside world into your enterprise data
• 14:57
  source, right, or your data-driven insights and that is
• 15:01
  probably the safest approach. So you're essentially creating a
• 15:04
  universe where your data is, is read, right, it's iterated on
• 15:09
  and there there are insights generated based on that data
• 15:12
  only and you're then just working only with that.
• 15:16
  Then there's the next layer of, you know, maybe you have some
• 15:20
  level of output from either the tool itself and there are
• 15:23
  preferred data sources into the tool or maybe the tool to some
• 15:27
  extent pulls data from the outside world, right.
• 15:30
  But it's not feeding your data out into that world.
• 15:33
  So your data is not used to train, you know, the the broader
• 15:37
  LM in any way. So your data still stays within
• 15:39
  the circle, but then you're drawing insights from kind of a
• 15:42
  broader set of data points and data sources.
• 15:45
  What's interesting is that at this point in time, depending on
• 15:48
  the tool, depending on how it's built and how it's structured,
• 15:51
  you can mitigate what we call hallucinations.
• 15:54
  So hallucinations are really just how how AI gets it wrong,
• 15:59
  right? So it's generating insights
• 16:01
  where it just gets it wrong. It either makes stuff up or it's
• 16:05
  something that's not true, right.
• 16:07
  But the reason that it does that is because these generative
• 16:09
  technologies, their beauty and their power is in creativity
• 16:14
  itself. So they can be creative.
• 16:16
  That's what they do best and it's really interesting to think
• 16:18
  about to what their limitations are, right.
• 16:21
  So when we see what the limitations are on generative
• 16:24
  AI, it's really their ability to do multi step reasoning and
• 16:28
  that's why I'm doing math. It's such a good, you know, use
• 16:30
  case to show what it can't do because what it can do is be
• 16:34
  creative. But when it comes to multi step
• 16:36
  reasoning that has a degree of being right or wrong, that's
• 16:40
  where it's sometimes you know is not as accurate as it should be.
• 16:44
  And so bringing that all together, if you're OK with a
• 16:48
  bit less creativity but you'd really like to double down on
• 16:51
  safety. There's a way to do that
• 16:53
  depending on how you structure the tool, depending on whether
• 16:56
  you keep it more open or more closed, and it's really just a
• 17:00
  matter of what your internal strategy is and how you build
• 17:03
  it, and how correct you like that tool to be fit for your
• 17:06
  particular use case. From my.
• 17:09
  Yeah, now I, I, I I love that. I if I dig a little bit deeper,
• 17:13
  I think what you're trying to say is, look, it's the these LM
• 17:18
  or these engines call it whatever you want, they're
• 17:22
  great. But you also have to use your
• 17:23
  data which I think is a key key unlock.
• 17:26
  And I think once you try to explain to people actually look
• 17:29
  we're not using the LM's knowledge that unlock here for
• 17:32
  many organisations is their data and it and it's around the
• 17:36
  sensitivity of that they are which is one of the lenses you
• 17:39
  need to consider. If I understood correctly.
• 17:42
  Well, on a comfortable with that leaving the perimeter and going
• 17:46
  somewhere else and living in someone else's house, argument
• 17:48
  sake. Or do they want to keep that
• 17:50
  with inside their own boundaries, their own perimeter,
• 17:53
  in their own house and in managing that before we get on
• 17:56
  to hallucinations is that? Correct.
• 17:59
  Exactly right. Yep.
• 18:00
  Exactly. And I absolutely.
• 18:02
  And look I I I had another line on on top of the the the risk
• 18:06
  stuff and I think sure we'll dive a little bit more
• 18:08
  especially on hallucinations IP and there's a there's a tonne of
• 18:11
  stuff if I bring the IT way of thinking about you know build
• 18:15
  versus buy. I want to buy some because I
• 18:17
  want to try something very quickly like I or gain some kind
• 18:20
  of capability very, very quickly.
• 18:22
  But at the trade off that I can't customise it and I can't
• 18:27
  control it which is the risk. Lens customization is going to
• 18:30
  be important. I will dig into wireless
• 18:33
  important, but I might not be able to control it as well,
• 18:36
  which is I think what you're alluding to.
• 18:37
  Is that a huge risk going if I build something?
• 18:41
  Potentially this could be I build some software for some
• 18:43
  users internally. I get you know exactly what I
• 18:46
  want, but there's a greater expense here.
• 18:50
  Now build doesn't mean I have to build all the parts.
• 18:52
  I can take parts and this is where the open close comes into
• 18:54
  it or I have to make decisions, but I have a greater expense of
• 18:58
  actually building something what I want and it takes a lot more
• 19:01
  effort. And then there's more
• 19:02
  constraints on like do I have the right you know, people kind
• 19:05
  of skills do I right data infrastructure, GPU, which is a
• 19:10
  big big problem. Availability to do this.
• 19:12
  Can I, can I do this? Not everyone is Bloomberg and
• 19:15
  can build their own GPT. I'll pick on them as an exam but
• 19:19
  I'm I'd I'd always come back to higher level and whenever we're
• 19:24
  talking to organisations and just before we get into build
• 19:27
  respite what's what's the problem we're trying to solve
• 19:30
  like I'm going to borrow something from Steve Jobs and I
• 19:32
  I talk about this a lot at it and it sounds so common sense
• 19:36
  but what's the problem trying to solve why is it a problem and
• 19:40
  then backwards to what do you need to address that do you need
• 19:43
  a chat bot arguments say do you need an LLM is there other
• 19:47
  things better that you could do instead And sorry that might
• 19:51
  offend people I I'm not not for tech forwards where here's some
• 19:55
  technology and let's go find a problem I I never really sticks
• 19:59
  so it's if there is a a real need of an organisation it's
• 20:02
  working way backwards and then to your.
• 20:05
  To everything that you highlighted from now on.
• 20:08
  I love your your insight on this what's the risk lens what's the
• 20:12
  the the talent lens what's the the skill lens infrastructure
• 20:16
  and IT the list goes on and on and there's lots of things and
• 20:18
  so build versus buy it depends like that's the that's probably
• 20:22
  the punchline depends on the organisation it depends on a lot
• 20:25
  of things. Now, I'd love your take because
• 20:27
  I feel like I've been talking for a bit, but you know more
• 20:30
  than I do in the field. You always telling me off, so.
• 20:34
  No, no. And another talking you, you
• 20:36
  framed it well, right. I think that one of the other
• 20:39
  dimensions to add into this is that generally is a new field.
• 20:43
  So when the conversation comes to build versus buy and
• 20:47
  especially in the kind of the pre built product, it is a new
• 20:51
  field. A lot of the products are either
• 20:53
  just coming into the market or kind of being demoed and tested
• 20:57
  all kind of they haven't spent long enough for a lot of
• 20:59
  companies to feel comfortable at deploying these.
• 21:02
  So when you add that generally organisations don't necessarily
• 21:07
  go through this, they seem a product that makes sure that the
• 21:09
  market or there's support options and thereby they'll pick
• 21:12
  it up and select. So when you bring that lens and
• 21:16
  you say how mature is the product, how mature is the
• 21:19
  market, what am I looking to solve for and how it's going.
• 21:23
  I as a as an organisation, to make this decision I would have
• 21:28
  to think really hard, number one, especially with how rapid
• 21:33
  the field has been moving. A lot of organisations decided
• 21:37
  to build their own chat bots maybe beginning of 2023.
• 21:42
  And then like by mid 2023 we started to see you can get
• 21:46
  products that will allow you to spin up your own chart but
• 21:48
  without the engineering effort. And so there's a kind of a
• 21:52
  question being asked now. Should I wait three months, six
• 21:55
  months or build it myself And what are kind of the questions
• 22:01
  that I have to take on and how do I know it will be built?
• 22:04
  And so maybe a better question some, some people ask is that is
• 22:09
  it something I should be building myself or is it
• 22:11
  something I should expect the market to provide for me?
• 22:15
  And one of the lens I kind of, well, one of the insights I
• 22:19
  concluded on is that if you believe you're building a
• 22:23
  product that is core to your IP, it is an essential part of your
• 22:28
  business or it's an essential part of your product.
• 22:31
  And there's there's a very strong reason to kind of
• 22:34
  explore, should I build it and should I do this because as you
• 22:37
  said, you will have your data in it.
• 22:39
  You may have more than just your data, your policy, your your
• 22:42
  constraints and control other things like for example,
• 22:47
  Copilot. I don't think every company
• 22:48
  should be developing their own GitHub Copilot through
• 22:51
  accelerated developers. I think a few options would be
• 22:54
  enough, but not enough for every company to build it on that.
• 22:58
  So I wanted to say something I I just am jumping in because I
• 23:02
  love these points that you're making.
• 23:04
  And there's something that Luna mentioned that I think is
• 23:07
  fascinating when it comes to the degrees of that ability for you
• 23:15
  to have this buy selection where maybe it's it's just your
• 23:20
  someone who's like a regular person going and hitting the GPT
• 23:25
  for API or maybe. And then you don't have any idea
• 23:30
  what's happening with your data behind the scenes.
• 23:32
  Are they training anything with it or not?
• 23:35
  And then so you have that kind of like spectrum, I think, which
• 23:38
  is fascinating where you can say, OK, now I know I'm still
• 23:42
  buying, but I know what's happening with my data and I'm
• 23:45
  able to say that this is not being used for training
• 23:50
  purposes. And then Moe, what you were
• 23:52
  saying is something that I've been hearing a lot of people
• 23:54
  talk about as far as thinking about forward compatibility and
• 23:59
  how can I build something now for the future and future proof
• 24:04
  it in case a new model comes out.
• 24:06
  Everybody's talking about multimodal now.
• 24:09
  Can I build a system that is ready for the multimodal model
• 24:13
  that comes out the Luma 3 model next month when that drops?
• 24:16
  Like how can I be forward compatible and backwards
• 24:20
  compatible? So there's my two cents.
• 24:22
  I'll get off stage now and that you all get back to.
• 24:24
  It and I'll add to that too. I love the idea of thinking
• 24:28
  about compatibility and forward and backward, but there's also
• 24:30
  regulatory compatibility, right, and making sure that you're not,
• 24:33
  that you're avoiding regulatory debt.
• 24:35
  And we're certainly not here to give legal advice.
• 24:37
  But I think different providers and different build and buy
• 24:40
  considerations really should embed that as an analysis and a
• 24:43
  point of consideration. So if you're building something,
• 24:46
  embedding responsible AI principles in by design and
• 24:49
  making sure that you're thinking about, you know, working with
• 24:53
  your legal and compliance teams to think about where the
• 24:55
  regulatory space is headed to make sure that if you're
• 24:57
  building, you're not putting yourself into a position of, you
• 25:00
  know, falling into regulatory debt down the line.
• 25:03
  On the flip side, if you're buying, understanding the right
• 25:05
  questions to ask your vendors and really think about, you know
• 25:08
  are your vendors or whoever you're you're buying from
• 25:11
  aligned with certain principles, aligned with your own
• 25:13
  organisational positioning on responsible AI issues, right?
• 25:16
  And how you approach it and how you embed it in by design?
• 25:19
  I think these are all really important considerations.
• 25:22
  Andrew said but I wanna I wanna build on that and and the
• 25:25
  comment I made earlier about you know what's the problem you're
• 25:27
  solving and work your way backwards like from from the use
• 25:29
  case perspective and it's Demetrius touches a little bit
• 25:33
  on on you know someone just chooses GPT but for some use
• 25:38
  cases they're so trivial you don't need the gold standard I I
• 25:42
  I don't believe you do you so in some use cases are sold trivial
• 25:45
  with inside organisation you could get away with something
• 25:48
  very very small that's not GBT it it doesn't always have to be
• 25:53
  GT4 arguments 8. It could be one of the the
• 25:56
  simple open source open models and I can see a situation as it
• 26:01
  started to emerge that you eventually may have but we'll
• 26:04
  talk about the risk of the risk suddenly is magnified lots of
• 26:08
  different solutions or use cases we've inside your organisation
• 26:12
  which will be using different LMS and that's a different ball
• 26:16
  game from a you were a cheap data officer CIO CTO you'll be
• 26:20
  worried at night like where is my day going how how am I
• 26:23
  protecting it. But I can see that situation
• 26:25
  already, especially with a lot of these off the shelf solutions
• 26:28
  which have LMS as features or generative AI parts as features.
• 26:34
  You don't know what model that's using.
• 26:38
  It's not written anywhere and and and and that that's
• 26:42
  happening at the moment. You know we're we're seeing that
• 26:44
  at the moment so I just wanted to build on that.
• 26:45
  I always start from the the I personally you'll start from the
• 26:49
  use case point. You work your way backwards and
• 26:50
  figure out what you need and sometimes it might not but that
• 26:53
  needs to be considered it might not be the GPT of the world.
• 26:56
  You could use something trivial and small, very small, lovely
• 26:59
  hosted. There, there's an awesome
• 27:01
  question that just came through the chat that I want to grab
• 27:04
  real fast while we are on this topic.
• 27:06
  And it is asking about if there are products that folks are
• 27:11
  seeing which provide transparency for the users or
• 27:15
  clients into how the model was trained or for that matter what
• 27:19
  model it is. Because maybe if we can just get
• 27:22
  what model it is, then we can say OK, we agree or don't agree
• 27:26
  with this model's training and the way that they've done
• 27:29
  things. Is, is that off the shelf
• 27:32
  products, Yeah. That are.
• 27:37
  So is there products that provide the transparency for how
• 27:41
  the model was trained or which model is being used behind the
• 27:44
  scenes like you were saying like a lot of times we don't
• 27:47
  necessarily get that. Yeah, we we, we don't get, I
• 27:50
  mean, even with the, let's say, let's say the open models, we
• 27:53
  don't know how we could definitely fine tune these open
• 27:56
  models, but we don't know how they were trained.
• 27:59
  We don't know exactly the data sets that were used, even though
• 28:02
  some of them could you reproduce?
• 28:04
  If I use the proper definition or open source, I can take the
• 28:07
  source code and I can go and get the data, get my own hardware.
• 28:10
  I can reproduce exactly. But I don't think you can do
• 28:12
  that today. And with some of these black
• 28:14
  boxes, I don't think you can off the shelf pack products.
• 28:19
  You could, you know, you have the full transparency of what's
• 28:21
  available or what's being used behind the scenes.
• 28:24
  Well, I don't, I don't know if you've seen differently or if
• 28:26
  you've seen differently like product evaluations, but I don't
• 28:30
  I if I can look now I don't see anything on terms and conditions
• 28:33
  that we use these type of models and this is how we use your data
• 28:36
  and this is how it was train. I've seen it differently.
• 28:39
  I think I was trying to cheat a little bit and say, uh, big
• 28:43
  organisations that may use managed services will have some
• 28:48
  of their data agreements that, you know, we share your data
• 28:51
  with X&Y And there are some companies that I think I may
• 28:55
  have spotted it, but I don't think it's enough of a
• 28:57
  transparency to say here as a user how you can go and learn
• 29:01
  about it. I think that's right.
• 29:02
  And I'll just add that you know it's helpful to keep in mind, I
• 29:06
  don't know that we get this level of transparency, but
• 29:08
  keeping in mind that different LM providers have different sets
• 29:12
  of ethos behind. They have a different ethos,
• 29:15
  right, each one of them. So anthropic versus cohere
• 29:17
  versus others, right, versus open AI, there are just
• 29:21
  different considerations and different ways in which the the
• 29:24
  functionality operates and you can you can argue right,
• 29:28
  different effectiveness levels. Some are better than others for
• 29:31
  certain things, right there they have, they're starting to, we're
• 29:35
  starting to see different types of features and different types
• 29:38
  of kind of guiding and foundational principles emerge,
• 29:42
  which I think is a really interesting space to watch.
• 29:44
  But again, we can really only understand that if we have
• 29:46
  transparency into which LM something is using in the first.
• 29:50
  Maybe we we went through some examples like some fictional
• 29:54
  examples is like you know some of the decision and so on would
• 29:56
  need to do that would have been the way just had to anchor on on
• 30:02
  on on things. Hey, I'm Vishnu.
• 30:05
  I'm a data scientist at first hand and I definitely think that
• 30:09
  you should subscribe to the ML OPS Coffee Sessions podcast.
• 30:12
  It's the best podcast out there to stay on top of what MLP's
• 30:16
  actually is, to talk to the true thought leaders in the space.
• 30:19
  And oh by the way, Demetrius is absolutely hilarious.
• 30:23
  What a weird guy. You should definitely subscribe
• 30:25
  to the podcast. I've said I this used to chat
• 30:34
  bot as an example and I chat bots everywhere.
• 30:36
  But maybe it isn't an organisation who wants to build
• 30:39
  a chatbot and and one of the you know maybe they want to prove
• 30:45
  value quickly. Does this have value now?
• 30:48
  If I framed it right earlier that there's two parts, it's the
• 30:51
  product what people use and then there's a model part on the
• 30:54
  product side of the the chatbot, the front end, they really using
• 30:58
  service now. So we can ignore that for now.
• 31:00
  So, so we'll just focus on the model and focus on the
• 31:02
  integration. Now in this fictional example,
• 31:05
  maybe because I have to build this back end needs to build an
• 31:09
  API and have an LLM. This fictional organisation
• 31:12
  might have Azure and so it's not too much of A stretch because
• 31:15
  they just wanna prove values differ concept maybe to use
• 31:18
  Azure Open AI as an example, even though we've been saying
• 31:21
  you don't have to always use them but I'm just using that as
• 31:23
  an example. And because they probably have
• 31:26
  Azure they've also got a contract in place and probably
• 31:28
  guidelines around you know the data will never leak out and
• 31:30
  what not. And a lot of comment here on you
• 31:32
  know what what should look at that, that's the the model part.
• 31:36
  But then there is actually a build part because it doesn't.
• 31:39
  That's not all we have to do. We talked about this service,
• 31:42
  now we we've chosen the model, we actually have to build
• 31:44
  something and this is that API. Somehow we have to integrate
• 31:48
  that with service. Now we have to use our data
• 31:50
  somehow. So now we need to talk about,
• 31:52
  OK, you need to build a vector store.
• 31:54
  So you need to put that somewhere, you need that back
• 31:57
  end up and running, you need the infrastructure and then you need
• 32:00
  to work on the ways of integrating backwards and
• 32:02
  forwards service now. But that's the kind of decision
• 32:05
  making that you might go for. Let's say an internal basing
• 32:10
  customer chatbot an internal base.
• 32:13
  But maybe then because you are wanna make it external facing
• 32:19
  and now you're talking about, well, I don't want it to be a
• 32:21
  proof of concept. I've proven that the the team
• 32:24
  can use it and it's solving a problem we talked about earlier.
• 32:28
  It's a bit sensitive data. There's customer data here.
• 32:30
  I don't I don't fully wanna put it into OK, I I I so now it's
• 32:34
  about taking that model part and saying OK I wanna host that
• 32:37
  myself. I don't wanna, I don't want
• 32:39
  someone elses, I don't want it to be in someone else's
• 32:41
  infrastructure. I want to host it myself.
• 32:43
  So now you have options on some of the open models.
• 32:47
  That's curve of from using an API or someone's API or
• 32:52
  someone's model, something from a commercial point of view and
• 32:54
  in pivoting later as you start to move towards production and
• 32:57
  scaling this out and you have more risk worries, That's what I
• 33:01
  tend to see. But maybe, maybe I'll just pause
• 33:03
  there and say that, does that resonate with you 2?
• 33:05
  It really does. Yeah.
• 33:06
  And and I might even go one step back a bit I think that was you
• 33:10
  said it so well in there. And I I.
• 33:12
  The question that I see that I've been getting most often or
• 33:15
  that I've seen, you know in the industry or in roundtables
• 33:17
  that's been talked about is what do we do first, right, in terms
• 33:21
  of thinking about use cases whether it's legal services or
• 33:24
  or other industries right. When organisations are thinking
• 33:28
  about how to approach generative AI from an assistive
• 33:30
  perspective, right when they're augmenting, accelerating,
• 33:33
  empowering, what they're doing, how do they identify the use
• 33:36
  cases and what should they do 1st?
• 33:38
  And I think the chat bot to me is the high value, high, highly
• 33:43
  scalable use case, right? And then the question is if
• 33:45
  you're building that. And when we say chatbot, the way
• 33:47
  that I understand chatbot is it's really it.
• 33:51
  It chat about sounds kind of funny, it sounds like it's a
• 33:53
  little robot doing lots of different things.
• 33:54
  But I think the way to understand it in very simple.
• 33:57
  Terms. Exactly.
• 34:00
  It's something that you know, it's like a really smart
• 34:03
  enhanced Google in a lot of ways, right.
• 34:06
  It's Google that can do lots of different things and and even
• 34:08
  more broadly it can summarise information for you.
• 34:11
  It can generate insights based on your, you know, everything
• 34:15
  that you have internally in terms of your knowledge base,
• 34:17
  right. Whether it's policies and
• 34:19
  procedures or data that you've collected or insights that
• 34:22
  you've collected or you know, in financial services or in in, you
• 34:27
  know, it might be investment summaries or in the legal space.
• 34:30
  That's contracts, right. Whatever it is that you'd like
• 34:33
  to summarise or aggregate and generate insights on internally,
• 34:36
  it can do that for you. It can also paraphrase emails.
• 34:40
  It can shorten your emails and make them clearer.
• 34:42
  It can summarise things that you feed into it, right.
• 34:45
  It's basically your right hand assistive tool to do a lot of
• 34:48
  things organizationally faster than you can.
• 34:51
  Then you can just ordinarily do them, you know, at human speed.
• 34:54
  And that to me is what chatbot means and that's high value and
• 34:57
  highly scalable across your organisation.
• 35:00
  And the build versus by consideration that I think is
• 35:03
  really interesting is when you're thinking about build
• 35:07
  versus buy, if you have the capability to build, I think
• 35:11
  what's really interesting for organisations to think about is
• 35:14
  that they know their data best, right?
• 35:16
  You as an organisation, you know the kind of data that you have.
• 35:20
  And so you know and you may understand that tension of how
• 35:24
  you treat that data and how you want insights generated on that
• 35:28
  data versus having the risk of hallucination, for example.
• 35:32
  Right, That you are uniquely positioned to understand the
• 35:35
  tension of that risk and what it means.
• 35:38
  Whereas something that's off the shelf may not get that, may not
• 35:43
  get that tension exactly right. Maybe it's too risky, maybe it's
• 35:47
  about risky enough. But you as an organisation you
• 35:50
  understand your own strategic priorities.
• 35:52
  You know what kind of tooling will really feed your needs
• 35:56
  best. And mayor, this is to your
• 35:57
  point, right? Don't have the tech identify the
• 36:00
  problem for you. Identify the problem 1st and
• 36:02
  address it. And that, I think, is really the
• 36:04
  heart of the matter. It's about understanding what
• 36:07
  your problem statement is, what your pain points are, and how
• 36:09
  you address it, and then considering whether you can
• 36:13
  tailor something towards that by building or buying.
• 36:18
  Another lens to it as well that when you're using a service such
• 36:24
  as one of the managed services cloud open AI and what have you
• 36:30
  is that it's not just the model, right.
• 36:32
  We've noticed that for example when model least at earlier
• 36:36
  stages, people have tried to get them to say inappropriate thing,
• 36:40
  get them to kind of do behaviour they were not intending to do
• 36:43
  and so on. And all the time those suppliers
• 36:46
  did add guardrails, did add enforcement layers and layers in
• 36:50
  front of it. So what you get as a user, it's
• 36:55
  not just the model, it's the model plus whatever values
• 36:58
  certain organisation may kind of deem the right for them.
• 37:03
  So one of the things that I generally float when we say the
• 37:09
  build versus buy, it's really easy for us maybe here in the UK
• 37:13
  and in the US because the UK is so close kind of to the US
• 37:16
  culture to a degree. But like a lot of the models
• 37:18
  have been coming from the US and kind of from a Western culture.
• 37:23
  So a lot of the guardrails are all of the behaviours that are
• 37:26
  being ingrained in them are very suitable to this.
• 37:29
  And when you start to ask can I deploy?
• 37:34
  I'm an organisation, I'm an international organisation.
• 37:36
  If I built a chatbot that I wanted to serve to my
• 37:40
  organisation internally or to my customer base, that's global.
• 37:44
  Will their behaviour and the success that I get when I run it
• 37:46
  in the English and I start to roll it out French and other
• 37:49
  European languages or even taken to Africa and only the Eastern
• 37:51
  and then Asian things be it applicable?
• 37:55
  If this is something that has to worry about because I have built
• 37:58
  the product, I've demonstrated the value I I know it worked.
• 38:01
  But the small detail is that do I now have to worry that when I
• 38:07
  do it in different markets I can expect different behaviours
• 38:10
  because an individual sitting in Southeast Asia may ask the
• 38:14
  question differently and then somebody who will ask it in in
• 38:19
  the UK and maybe they'll ask it even in their local language.
• 38:22
  I love that and and look he's he's not quite the same thing
• 38:25
  but I I noticed someone recently took the and this is probably
• 38:28
  one of the big benefits or open models.
• 38:32
  Someone recently took the Llama to model and fine-tuned it for
• 38:38
  Odia. Audio is an Indic languages of a
• 38:42
  place in the Odisha, in the Indian state of Odisha.
• 38:46
  And what they wanted is they wanted additional domain
• 38:50
  knowledge of the local area and culture.
• 38:52
  So like food, recipes, historical places and temples
• 38:56
  and some of the health rituals, art, culture, famous people of
• 39:00
  the region. You're not going to get that
• 39:03
  with probably a commercial model that's built in the US Like if
• 39:07
  you if you're in India and if you're in the Middle East, you
• 39:09
  probably want something local as well.
• 39:10
  If you're in Far East, you probably want something.
• 39:13
  So I think one of the benefits and this is also why yeah like
• 39:17
  goes back to it depends what what are you, what are you
• 39:20
  building for what do you need to do now.
• 39:23
  But to fine tune models and and do that, I think we'll we'll
• 39:26
  we'll dive into that is that you still you will need additional
• 39:28
  talent and you all skills and you need infrastructure and you
• 39:31
  need GPUs and whatnot but you can get something that works for
• 39:35
  your, for you, for your as an organisation.
• 39:38
  I think it's pretty cool story. I think there's a paper out,
• 39:41
  maybe I can share it on the chat if I can find it, but it's just
• 39:45
  a nice benefit of OK, This is why open open models are are
• 39:48
  are, are are pretty useful because I wouldn't expect one of
• 39:53
  the the big vendors to do this. There are some really cool
• 39:56
  questions coming through in the chat, and one that I've been
• 40:00
  noodling on for the last minute is.
• 40:04
  How do you all differentiate the build versus buy question for
• 40:09
  Gen AI versus just ML, traditional ML, I guess we could
• 40:14
  call it these days if at all? I'll, I'll try.
• 40:18
  I'll try to take a stab at it because they're not the same for
• 40:20
  you. So that's that's the first thing
• 40:22
  and and they complement each other very, very nicely.
• 40:26
  Look if I use the example as as as a customer interaction type
• 40:33
  of tool maybe for example maybe you you like your your a
• 40:38
  retailer maybe there's a they've got purchase history.
• 40:41
  Maybe you wanna do recommendations to an individual
• 40:43
  and maybe they wanna use Gen AI to customise those
• 40:47
  recommendations in a language or tone, especially for the end
• 40:50
  individual. So maybe Mo likes black
• 40:52
  T-shirts, maybe a loaner likes T-shirts with bit sparkle, I
• 40:55
  don't know. But but as a as a as a retailer
• 40:58
  I I I know all of it. I have all this data because
• 41:01
  I've got the purchase history and I see what they're doing and
• 41:03
  what. But I also know you know where
• 41:05
  they respond. Do they respond to emails or
• 41:07
  Slack messages or text messages or what that calculation,
• 41:11
  although it's analytic models to decide which is the best channel
• 41:16
  to talk to them, what's the appropriate messaging and what
• 41:18
  not, That's traditional AI and you still need to have that.
• 41:22
  And then what you could do is take a lot of that information
• 41:24
  and then pass it to get AI, which will then come out with
• 41:28
  something like, I don't know, tomorrow would be very, very,
• 41:32
  very formal type of like this is available and here's a discount
• 41:36
  or what not. And so maybe to alone would be
• 41:38
  very very hip and kind of text message, very short punchy to
• 41:42
  the point and that could be the genie layer.
• 41:44
  But I I see them as compliment. They don't replace each other.
• 41:48
  But then here comes back to the the, the, the technology and and
• 41:52
  figure out all these different parts.
• 41:54
  There's a Gen AI part for sure and the input that is not in
• 41:58
  what we've been talking about which is rag.
• 42:00
  The input of that is the output of some analytic model which is
• 42:04
  saying send message at this time of this channel to this person.
• 42:07
  And so you work your way backwards and to train that
• 42:10
  analytic model you still need your data platform or your
• 42:13
  customer 360. You still need your traditional,
• 42:17
  you still need to build some kind of infrastructure to do
• 42:19
  that. I'll just pause there, but I
• 42:21
  don't open model only. You have anything else to add on
• 42:24
  to that? But it's just that bring it to
• 42:25
  life that there are Jenny doesn't replace it.
• 42:28
  Compliments. Yeah, it doesn't.
• 42:32
  That England, I would almost say it's in the name.
• 42:36
  It's the generative nature of it.
• 42:38
  Like if you look at what people use, generative AI raise the
• 42:42
  most popular cases. It isn't something that we've
• 42:45
  traditionally done with regular about.
• 42:47
  We may have done it with robotics automations and except
• 42:49
  for example chat bots isn't something that ML kind of had
• 42:54
  superiority on. And a lot of the popular ones
• 42:57
  have always been ruled based and keyword based and
• 42:59
  acknowledgement. And when Gen AI came about or
• 43:03
  when you know there's more than just L&M's in this space that is
• 43:06
  picture and so on and we should probably speak about those in
• 43:09
  another session because they deserve their own session.
• 43:13
  The LLM in this space, I'm using it to either speaker corpus,
• 43:19
  extract something out of it, summarise something out of it,
• 43:22
  process something out of it, maybe I give it some hints and I
• 43:25
  ask it to generate something of it.
• 43:27
  Now the applicability where I can continue as you said at the
• 43:30
  player, let me find the insight that I want to reach my user to.
• 43:34
  And now that I found out, I wanna reach there the cool spiky
• 43:39
  kid for those black T-shirts and I want to reach him on WhatsApp
• 43:44
  now to frame a message that will allow me to engage him in a
• 43:49
  particular toll and so on. I'll have to go and build a new
• 43:52
  product and so on and well if I built it for Nair, I'll have to
• 43:55
  build a different one for an older.
• 43:56
  And so generative AI is kind of unlocked a certain space.
• 44:01
  We we had very robotic. Michael have been engaged in
• 44:03
  technology with that rather than creatively.
• 44:06
  How can I just give it enough prompts and ask and and get it
• 44:09
  to surprise me. I think that's where Laura was
• 44:11
  saying earlier and very good at just creativity where I I know
• 44:16
  what I want or at least I know what I want to know but I don't
• 44:18
  know how to deliver it. Can you help?
• 44:20
  And that splittism even though it kind of is, could be scary in
• 44:25
  some use cases. The nature of it is what
• 44:30
  generally is adding to the to the to the entire conversation.
• 44:33
  What do you think about? No, you said I that's exactly
• 44:36
  what I was going to index on. I I wanted to focus on that
• 44:39
  generative quality and I think you said it really nicely both
• 44:41
  of you. So it's the generative quality
• 44:43
  and it's the spontaneity, right. And that's where so much of the
• 44:46
  interesting value is in this tech.
• 44:49
  The one thing I would add and this is really just adding and
• 44:51
  building on on everything that you're saying in terms of a
• 44:53
  consideration for for organisations thinking about the
• 44:57
  data readiness for generative AI.
• 44:59
  I like to think about it in in terms of nutrition, right.
• 45:02
  What does a specific type of Gen AI technology, if you're
• 45:06
  building or buying, what kind of nutrition, is it, you know, fed
• 45:10
  on it? Does it sustain?
• 45:11
  Is it Word documents, Is it PDFs?
• 45:13
  Is it, you know, is it structured data, unstructured
• 45:16
  data? I think this is important to
• 45:18
  think about. And I think this is a bit
• 45:19
  different than more traditional AI when you're, you know,
• 45:22
  building, when you're feeding data into an algorithm.
• 45:25
  I think generative AI, at least at this stage is a bit more
• 45:28
  sensitive to the data type and format that it feeds on.
• 45:32
  And so that's just important to think about in terms of getting
• 45:34
  the output that you want and you know, the level of creative.
• 45:37
  You're looking for. I I want to and I I love that I
• 45:41
  I wanna build on something you said earlier Alona and and this
• 45:45
  just finishes off the point on the the A it's not the generic
• 45:49
  technology is great but it's not great numbers not great Max like
• 45:53
  you you can trade it all your lights and couldn't get them
• 45:56
  back and where there's some kind of financial stuff going on and
• 46:01
  this is where your traditional is gonna really kickass it is if
• 46:05
  you're going to need it you're just gonna keep going backwards
• 46:08
  and backwards you go OK damn. I need to do traditional
• 46:11
  analytics I need to do traditional A I need to get that
• 46:13
  right. I need to build that in place.
• 46:14
  So lots of those investments in MLPS and having all those
• 46:20
  processes in place, ways of working, they'll still valid,
• 46:23
  they don't go away. You just now have added
• 46:25
  additional what Morrowind alone have added additional things.
• 46:29
  You think about in terms of how do I treat this unstructured
• 46:32
  data very well, How do I govern that?
• 46:36
  You know, with my existing existing modern data stacks, how
• 46:40
  do I have a proper kind of CICD, some kind of testing, but how do
• 46:44
  I test this stuff on top of what I also I've got in place?
• 46:47
  It doesn't. It just complements, it just
• 46:49
  adds on top. And if I take the lens of
• 46:53
  someone who manages a lot of this stuff, it's complex, it's
• 46:56
  not easy, it will get better over time, and I'm sure vendors
• 47:00
  will pull out all in one solution.
• 47:02
  But right now, today, it's stitching lots of different
• 47:04
  technologies and tech together, which is a which is an
• 47:08
  interesting challenge that we don't talk about enough.
• 47:11
  I think the the last sentence you said it's stitching.
• 47:14
  I think one of the interesting things open till before.
• 47:19
  So if you look at the history of Dani like or or LMS, there were
• 47:24
  being developed for quite a few years before they kind of
• 47:26
  reached a a a crazy popularity out of nowhere and it was the
• 47:30
  application. It's like how we starting this
• 47:32
  conversation. What is the I don't know if
• 47:34
  open. I knew they were very active
• 47:36
  even though I had some blogs that it was kind of a a random
• 47:40
  success story. But the objective is once you
• 47:43
  have triggered the successful use case where it clicks for
• 47:47
  people how to use it, suddenly you've got a bloom of adoption
• 47:51
  out of lower. And if you look at businesses
• 47:55
  and users who are asking to use Gemini now, right 11th in this
• 48:00
  degree, like how many of them are declaring it?
• 48:02
  How many of them are people that understand the data science and
• 48:06
  kind of and understand how these models are but they don't
• 48:10
  actually you. You'll notice that a lot of our
• 48:12
  business users that have understood enough how to
• 48:15
  interact with the product to build their own products or
• 48:18
  solve their own problems and thereby, like I would say in a
• 48:22
  funny story where and grandparents are using it to
• 48:26
  summarise books or to generate bad bedtime stories.
• 48:30
  Like could you have told me in 2015 or or before kind of that
• 48:34
  there will be people who are not even classed under digital scale
• 48:38
  of adoption using these technologies in such a free
• 48:42
  form? And out of that document, yeah.
• 48:45
  That's the applicability of that kind of where where one of the
• 48:50
  also the interesting niches I found Jenny B.
• 48:52
  News for. It's been used as a stitch
• 48:53
  between many things like it is so quick and easy and dirty to
• 48:58
  be able to grab an output from a website, do something for it,
• 49:01
  send it somewhere else, pick it up somewhere else.
• 49:03
  And it's like traditional A I always suffered from the data
• 49:07
  processing. You need really good data, good
• 49:10
  shape. That's what informed the
• 49:12
  decision making. With Janai you could now upload
• 49:16
  your Excel and ask it questions. There's in in some of the
• 49:18
  examples in the ChatGPT for example, and they will be able
• 49:22
  to do things for you. Yes, it's not just the model,
• 49:24
  there's more, but you can see the application.
• 49:29
  So I think I think we're we're converging on on time soon but I
• 49:33
  think as I, as I hear this look there's a top level message of
• 49:37
  it depends there's a there's a message of you know what are you
• 49:41
  solving for. There's a whole lot of different
• 49:44
  types of risk you need to you need to navigate and that it's
• 49:49
  more than just calling an API like going for this conversation
• 49:53
  it's more than just calling an API whether that you're hosting
• 49:55
  that API or someone else's hosting API.
• 49:59
  There's lots of things you need to think about including risk
• 50:03
  increasing legal including IP and a bunch of things.
• 50:05
  So I'm, I don't know, Demetrius. Like are we, are we, are we, are
• 50:10
  we confused everyone? I think this has been absolutely
• 50:16
  incredible. I love the activity and the
• 50:18
  chat. I just got to say that it's
• 50:20
  really cool to see everyone chiming in and asking questions.
• 50:24
  And so there's been another question that is actually
• 50:28
  something that I would love to get y'all's expertise on because
• 50:34
  it's a common pattern that I've seen which is coming from a mic.
• 50:37
  And he's asking how do you deal with model interoperability
• 50:44
  basically starting with Open AI to get something out there fast,
• 50:48
  quick, dirty OC and then you're like, OK, we've got some ROI on
• 50:54
  this, we need a productionized it, but we got to have a little
• 50:59
  bit more transparency, we got to have more control, we want to
• 51:03
  move to an open source thing. So we can we can have that,
• 51:07
  those levers that we need. Are there any guiding principles
• 51:11
  to follow to avoid huge migration risks?
• 51:16
  Yeah, this is there is a really good question and and I think I
• 51:20
  did a talk on this a while ago with maybe more or someone else
• 51:24
  we we did a thing on this. Look, there's no easy way.
• 51:28
  Like if you change the engine, he's like the engine of your
• 51:32
  car. He changed the engine of your
• 51:33
  car. Like you might find that
• 51:35
  something doesn't like maybe it was petrol with what diesel
• 51:37
  would prompts. Prompts are very sensitive to
• 51:40
  the to the models and you'll find that even within the same A
• 51:45
  family model Sophie that's Open AI for example between 3.5 and
• 51:48
  four the might not work. And we've seen this even between
• 51:53
  4:00 and no one tells you that something has changed.
• 51:55
  The prompt stopped working for whatever reason.
• 51:57
  Like there's a sensitivity and so this is where you know we
• 52:01
  haven't delved into this conversation but it's having
• 52:03
  good technology principles abstracting the prompts for
• 52:06
  storing metadata that this one is valid and it works on this
• 52:10
  particular model and this particular version of the
• 52:13
  particular model. But you're going to have to test
• 52:15
  even when you're doing your POC and you know at some point I'm
• 52:18
  going to use maybe an open model, bring it in house, you're
• 52:23
  gonna have to test. Now I know some of our team have
• 52:26
  managed to get away with problems that work across
• 52:28
  models, but that's only because they're testing and we've got
• 52:33
  you know they've they've built a very an indirection.
• 52:36
  So I look at like an API gateway, model gateway, so they
• 52:40
  could just quickly flip between the different ones and just
• 52:43
  test. So I I would think about these
• 52:45
  kind of things versus to your what I'm guessing is that you've
• 52:48
  just realised that our owners of migration effort and Oh my God
• 52:52
  there's thousands of thousands of prompts.
• 52:54
  How do how do I work my way through this and it's maybe lots
• 52:57
  and lots of hours or manual testing how do I get through
• 53:00
  this? So try and think about those
• 53:02
  architectural principles. This space will change at a
• 53:04
  ridiculous pace, so having the good hygiene or develops the
• 53:10
  foundational dev OPS principles that ultimately as much as you
• 53:12
  can are are going to pay dividends later on.
• 53:16
  Yeah, I think what I'll add to it is don't try and work in the
• 53:22
  middle of the night trying to get this bump to work.
• 53:24
  And then once you are happy with the answer submitted and
• 53:26
  expected, it will continue to make sure you have written
• 53:31
  enough prompts and recorded those responses and kind of
• 53:34
  committed some sort of a mechanism to constantly evaluate
• 53:38
  that what you are running against.
• 53:40
  Because these are services are. Because these are, especially
• 53:42
  when they're managed by somebody else, that they're constantly
• 53:46
  given at least an answer within the range you accept, whether
• 53:49
  it's an individual or an organisation or and so on.
• 53:52
  And you notice it like you notice like in in Twitter or
• 53:55
  Excel when people start complaining that they used to
• 53:59
  ask it this way and now no longer responds.
• 54:02
  So that same behaviour. You should document it and make
• 54:06
  it more reproducible and testable.
• 54:09
  Folks, I feel like our time here is sadly coming to an end.
• 54:14
  There are so many other great questions that I have in the
• 54:18
  chat right now which is really cool to see and I appreciate
• 54:23
  everyone of you for asking these questions.
• 54:25
  What I will say is that these three lovely human beings that
• 54:29
  have been teaching us for the last hour or so or at least 40
• 54:33
  minutes, are all on LinkedIn. So that is the good part.
• 54:37
  Feel free to reach out to them on LinkedIn and continue the
• 54:41
  conversation. And while you are here, I want
• 54:45
  to share with you all. We're going to leave, but that
• 54:49
  doesn't mean that you have to, because we've got a whole
• 54:53
  section where you can you can go and you can meet people that are
• 54:58
  here with us. So if you want to go to the left
• 55:00
  side here and you click on that match button then you can meet
• 55:05
  with others who are matched with you.
• 55:09
  So you can see there's I think there's like 120 a 140 of us
• 55:13
  here right now that all maybe wanna start chatting with each
• 55:18
  other. And the other thing that I just
• 55:20
  had to point out was these folks, the Quantum Black team
• 55:25
  put together this how to build a Knowledge Assistant at scale.
• 55:29
  I know there were some questions coming through about grounding
• 55:32
  your AI in facts and all that fun stuff.
• 55:36
  This is an incredible blog post about that and all the
• 55:40
  considerations that you want to be thinking about.
• 55:43
  And spoiler alert, they're coming out with some more, so
• 55:47
  wait till that drops soon enough.
• 55:49
  This has been so cool. Thank you Mayor Lonia.
• 55:53
  And Moe you all have. You've outdone yourselves for
• 55:58
  sure. Thank you.
• 55:59
  Thank you so much. Thanks everyone.
• 56:03
  See ya. Hey, Lazlo here.
• 56:09
  If you're serious about Emma Loops, you hit subscribe, right?
• 56:12
  Now.