Latest / The Road to Accountable AI / Tanvi Singh, Ekta AI: The Case for Sovereign AI

Transcript

Tanvi Singh, Ekta AI: The Case for Sovereign AI · The Road to Accountable AI

• 0:09
  Kevin: Hi, I'm Kevin Warbeck, professor of Legal Studies and
• 0:11
  Business Ethics at the Wharton School of the University of Pennsylvania.
• 0:15
  For decades, I studied emerging technologies from broadband to blockchain.
• 0:19
  Today, AI is promising to transform our world, but AI needs accountability
• 0:26
  mechanisms to ensure it's developed and deployed in responsible,
• 0:29
  safe, and trustworthy ways.
• 0:32
  On this podcast, I speak with the experts leading the charge for accountable ai.
• 0:39
  Today's guest is Tan Vi Singh, a longtime financial services technology leader.
• 0:44
  Who's now focused on building sovereign AI solutions.
• 0:48
  We talk about what AI sovereignty means beyond control of data
• 0:53
  and locations of data centers.
• 0:55
  The connections between sovereignty and accountability for ai, where
• 1:00
  domain models will gain traction versus broader, large language models, and
• 1:06
  how AI sovereignty is potentially a bridge across ethics, regulation,
• 1:11
  governance, and brand alignment for ai.
• 1:16
  Tavi, pleasure to talk to you.
• 1:17
  Thank you so much for joining me on the Road to Accountable ai.
• 1:21
  Thank you for having me.
• 1:23
  This is actually my first public conversation since coming out
• 1:26
  stealth for our sovereign AI platform company that I co-founded, and I
• 1:30
  think this is the right forum to do it because ETO was born directly
• 1:34
  out of the accountability problem.
• 1:36
  So very excited to be on the podcast with you.
• 1:40
  So let's start with the backstory before we get to, uh, what you're
• 1:43
  doing now in, in the sovereign AI work.
• 1:46
  Um, you spent a number of years in financial services, uh, places like Credit
• 1:50
  Suisse, UBS, um, very regulated risk sensitive environments for dealing with,
• 1:56
  uh, AI and, and other new innovations.
• 1:59
  Um, can you talk about your experience in that world and, and how that
• 2:03
  shaped how you think about some of these AI governance issues?
• 2:06
  Absolutely Gavin.
• 2:07
  I spent over two decades building and governing large scale data and AI systems
• 2:12
  inside highly regulated industries, um, global banks, public interest
• 2:17
  organizations, pharmaceuticals, and in those environments, AI isn't experimental.
• 2:22
  If you're accountable to regulators, citizens markets history sometimes,
• 2:27
  and when systems fail, there is no abstraction layer to hide behind, right?
• 2:32
  So it's not about drawing pictures or creating audios or making videos.
• 2:36
  It's true accountability.
• 2:38
  So when I saw repeatedly, that is most AI systems today are very powerful, right?
• 2:44
  And but externally controlled institutions consume intelligence through models.
• 2:49
  They don't own data, they don't govern and infer its processes.
• 2:53
  They can't always explain.
• 2:55
  And that create a structural issue because you can't be accountable for decisions.
• 2:59
  You do not control.
• 3:01
  And that was the need and that was the reason why.
• 3:04
  Of the last couple of years, ever since I left UBS, I tried to hone into the
• 3:08
  problem, looked at different investments, different companies across the globe, and
• 3:12
  when I did not find one that was exactly, that could exactly solve the problems of
• 3:17
  the regulated industry I've been part of.
• 3:19
  I co-founded Atta with my other co-founders, one of them, um,
• 3:24
  ex Microsoft architect who has built up the Microsoft Fed before.
• 3:28
  So he is the one who was part of the sovereign AI built in the early
• 3:33
  2020s before sovereign AI became a topic of conversation in our
• 3:37
  regular, uh, discussions today.
• 3:41
  So for those who aren't familiar, maybe, uh, describe what sovereign AI means.
• 3:47
  It's a very interesting question.
• 3:48
  Um, you know, sovereignty is what makes accountability real.
• 3:52
  So for sovereign ai, as we understood it was sovereignty
• 3:57
  with infrastructure, right?
• 3:58
  So G 42 and many organizations said that we want to keep the infrastructure
• 4:03
  with us where our data would reside within our own infrastructure.
• 4:07
  So we can control, you know, our information, which is our biggest asset.
• 4:11
  Um.
• 4:12
  Uh, in, in terms of our operation as a software nation, we in banking have
• 4:17
  seen sovereignty in a way that we did not put all of our data onto the cloud.
• 4:21
  I remember the back and forth I have with all my business partners to say it's more
• 4:25
  effective to put your data onto a cloud.
• 4:27
  Um, it's more cost effective to do so, but.
• 4:30
  A lot of those data could not be put onto the cloud because
• 4:33
  of the sensitivity of that data.
• 4:35
  So think about nations, think about organizations like banks
• 4:39
  and any other regulated industry.
• 4:40
  Data is your most precious asset.
• 4:43
  So for the longest time, Soen was about where do you put your data?
• 4:47
  Now far fast forward gen ai, uh, you could do a lot with the data and the
• 4:51
  LLMs came in the OpenAI, Microsoft were very good partners and, um,
• 4:56
  UBS has been a very good partner.
• 4:58
  So we saw the evolution of what happened with the LLMs, but with the large
• 5:03
  language models, you have to give away your data to either to the rag or to the
• 5:07
  training of the data in order for that LLM to be effective in your organization.
• 5:13
  The moment you put your data into a large language model or an open
• 5:17
  source model, you lose the sovereignty of that data because you do not
• 5:21
  control how that model is formed, govern secured, and he forth run.
• 5:26
  So in total concept of sovereignty, it is the control of your infrastructure,
• 5:32
  it's control of your data, it's control of your mortals, and how
• 5:35
  do you govern and run those models.
• 5:38
  So in a nutshell, if I were to say that I am the owner.
• 5:41
  I'm the leader of an organization.
• 5:44
  This system should reflect my value, my laws, my culture,
• 5:48
  and one that I can prove it.
• 5:50
  And if you could do that, then you have a sovereign infrastructure.
• 5:55
  But in this, uh, generative AI era, there are only a limited number of
• 6:01
  frontier labs that can build these.
• 6:04
  Truly, um, leading foundation models.
• 6:08
  Uh, so doesn't that push away from an environment where each deployer
• 6:13
  of AI is going to have total control of their own infrastructure?
• 6:16
  That's a very fair question.
• 6:18
  Counterintuitively, no, um, as domain specific models have proven over and
• 6:23
  over again, and we have seen that in the more regulated healthcare
• 6:27
  industry, that they're often more accurate and far more cheaper.
• 6:31
  So if your intention as an organization is not to have one Einstein in your
• 6:36
  pocket that can do everything for you, and in that case, you should
• 6:40
  be working with the frontier models to your point, but your intention is
• 6:44
  to have that one employee that's the best in class employee for you in an
• 6:48
  organization who is an expert in your organization, your data and your topic.
• 6:53
  Then domain specific models turn out to be much more accurate towards your.
• 6:58
  Respective benchmarks much more secure, and you wouldn't believe it
• 7:02
  a lot more cheaper because you're not churning the whole inference of an LLM.
• 7:06
  So I say it, you don't need a bazooka to kill a fly.
• 7:08
  And most of the time in organizations, all we are trying to do is to kill a fly.
• 7:13
  Uh.
• 7:13
  In the focus of the activities that, that you're responsible for.
• 7:17
  So domain specific models are the ones that, uh, can be more effective and
• 7:22
  does not need to be state-of-the-art.
• 7:24
  That can solve every single problem that you associate with when
• 7:28
  you look at the frontier models.
• 7:31
  Just an aside, not, not a big thing.
• 7:32
  We'll cut this out, but I, I, I, I think you're, um, you're bumping the, the
• 7:36
  table a little with your hand as you're talking, which I, I, I do that too.
• 7:39
  Okay.
• 7:40
  Which I, I'm picking up a little on mic again, the editor will pull it out,
• 7:42
  but just, just, you know, if you can not, not a big thing, but I'm hearing
• 7:46
  a little bit of background noise there.
• 7:47
  Okay.
• 7:48
  Yeah.
• 7:48
  I keep that in mind now.
• 7:49
  Thank you.
• 7:50
  Thank you.
• 7:50
  Okay, good.
• 7:51
  Um, was okay?
• 7:52
  Uh, yeah.
• 7:53
  Um.
• 7:55
  Can you drill down on what exactly is the, the challenge
• 8:00
  that calls for that sovereignty?
• 8:01
  Because I can understand a company saying.
• 8:03
  I'm worried about security or access to my data.
• 8:07
  Certainly many companies are worried about the, the, the queries and the
• 8:11
  prompts that they use, getting used to, to train large language models.
• 8:15
  But if you've got a, a enterprise contract with an open AI or a Gemini or
• 8:21
  something like that, that's presumably going to restrict what they can do
• 8:25
  in terms of using the prompting.
• 8:27
  So, so where exactly is the problem that organizations are finding,
• 8:31
  not having the sovereignty.
• 8:33
  So one of the latest news that came out last week was, uh, the
• 8:37
  acting, uh, uh, head of law legal for President Trump has been using
• 8:43
  chat GPT for a lot of very sensitive legal data for the US government.
• 8:48
  Um, and I think that article in itself is a much.
• 8:50
  Better articulation of the problem to the question you just talked.
• 8:54
  Uh, because as much, as long as we say the frontier models are all safe
• 8:58
  and secure onto our platform, the moment you send over any sensitive
• 9:02
  documents through that interface, it becomes part of the, um, training
• 9:06
  corpus of, of that frontier model.
• 9:09
  Mm-hmm.
• 9:09
  So that's where it, it, it's absolutely okay if I'm trying
• 9:13
  to arrange for a travel, uh.
• 9:15
  On a private level, if I'm trying to learn a language, if I'm using this for anything
• 9:20
  and everything to get myself educated on.
• 9:23
  So that's where the frontier models are extremely helpful, are extremely powerful
• 9:27
  and have become, uh, the basic part of our lives because we have every teacher,
• 9:32
  every expert in the room that we can.
• 9:34
  To, but when it comes to regulated topic, regulated industry, national secrets, then
• 9:39
  those frontier models become a challenge on the other side because, uh, you could
• 9:44
  not use, uh, those frontier models for your very sensitive information, that
• 9:48
  by default it becomes a training corpus.
• 9:51
  So.
• 9:52
  That's a challenge that most of the leaders of organizations that
• 9:55
  care about patents care, care about IP care, about the sensitivity of
• 10:00
  information about their clients, are not using large language models for any
• 10:04
  external client facing, uh, use cases.
• 10:08
  Most of the LLMs in the frontier models that you see of the
• 10:10
  organizations are still used for operational topics internally.
• 10:14
  Dealing with NONSENSITIVE data and in order to change the equation, in order
• 10:18
  to have more sensitive topics, more high value use cases to be implemented, which
• 10:23
  obviously include a lot of sensitive information, the need for domain
• 10:27
  specific models makes a very strong play.
• 10:31
  Should every organization be looking to having this sovereignty, or is
• 10:37
  this primarily an issue either for big companies or regulated companies?
• 10:43
  So brand alignment, uh, Kevin becomes another topic, right?
• 10:46
  Uh, we talk about private data just before, but brand
• 10:49
  alignment becomes, uh, a topic.
• 10:51
  So few of our existing pilot design partners and clients we
• 10:56
  speak with, um, are very, um.
• 10:59
  Interested in working with us on sovereignty, on domain specific models
• 11:03
  because they would want their values, their laws, their culture to be reflected
• 11:08
  as the brand alignment of the models.
• 11:11
  So sensitivity remains to be one.
• 11:12
  Number two is use cases around brand alignment.
• 11:15
  So if you're a PR company and if you do a lot of work around.
• 11:19
  Relationships.
• 11:21
  Then again, you know, building a model that speaks your language and
• 11:24
  not the language of the internet, which is typically what a large
• 11:28
  language model, a frontier model is.
• 11:30
  It again, becomes a use case.
• 11:32
  So it depends on what you care about.
• 11:33
  If you have, uh, if you are a school and you care about teaching
• 11:37
  different languages to your student, any model would work.
• 11:41
  Um, if you are, um, a PR organization and you create, and you have a huge brand
• 11:46
  value and you want to create a use case that is your client facing use case, you
• 11:50
  would consider a domain specific model.
• 11:53
  And then I guess we have definitely talked about sensitive regulated
• 11:56
  industries and sovereign states who care about sovereignty.
• 12:00
  And henceforth would look at sovereign domain specific models as well.
• 12:05
  As you said, you're creating a company ETA that's working in this area.
• 12:09
  Can you explain more specifically what it is that you are building for
• 12:13
  this, uh, vision of sovereign ai?
• 12:16
  So what we are building is a platform that has four different layers that
• 12:20
  takes care of the data ingestion.
• 12:22
  Um, pre-training, the model, the model itself, and, uh, the, the final layer
• 12:28
  that takes care of security of your data as, as, as the four layered platform.
• 12:33
  And this is a platform that can be deployed into any infrastructure
• 12:37
  that you consider sovereign.
• 12:38
  So in many use cases, we've heard, uh, private cloud platforms within Microsoft,
• 12:44
  Google to be sovereign as defined by the clients or on the other side.
• 12:48
  We have, uh, worked with, uh, other design partners who think about air gapped
• 12:53
  environments to be sovereign, which it.
• 12:55
  Truly is sovereign.
• 12:56
  So depending on what your definition of sovereignty is, we can embed our platform
• 13:01
  into, into, into that infrastructure and then can use your data to train the model
• 13:05
  from scratch and can build any kind of application on top, uh, which is viable
• 13:10
  and useful for you across industries.
• 13:13
  And that is what we do as a license, a bigger license and a smaller
• 13:17
  services, um, uh, set of, uh, opportunity, uh, towards our clients.
• 13:24
  Do organizations know now what their definition of sovereignty is?
• 13:29
  Do you have to educate them to think about those different approaches?
• 13:34
  That's a great question.
• 13:35
  Kevin, my co-founder, uh, was the CTO of a company.
• 13:39
  Previously, uh, started in 2023 and they were pioneering the
• 13:43
  exercise of domain specific models.
• 13:45
  So when they went to potential clients, uh, I was absolutely, uh, trying
• 13:50
  to do an education from scratch.
• 13:52
  So they had a lot of people experimenting with them, but it was a very hard
• 13:56
  take, uh, with regulated industry and also with governments to work with.
• 14:00
  Um, but they, but they got acquired last year by Cisco, which was a
• 14:04
  great exit and a proof point of how great the technology is because, um.
• 14:09
  Eventually now in 2026, uh, doesn't need a lot of education.
• 14:12
  We are starting to see more and more enterprise, more and more regulated
• 14:16
  industry coming and talking to us and doing pilots and demos because they
• 14:20
  understand it's not just the sovereign play, it's also the domain specificity
• 14:24
  that gives them the benchmarks of accuracy, and more importantly,
• 14:28
  the cost of running the model.
• 14:30
  The cost of inference becomes a lot cheaper.
• 14:32
  When working with a domain model, then while running a bigger LLM.
• 14:36
  So when 2026 is all about applications, it's all about productivity,
• 14:40
  it's all about accountability.
• 14:42
  We find ourselves to be in a much better space, uh, that de cards
• 14:46
  clients and their education and their knowledge about what sovereign domain
• 14:50
  specific models can do for them.
• 14:52
  I guess what I was, was getting at WA was also, as you mentioned, uh, for
• 14:56
  some people, if it's in the cloud, it's not within your sovereign control.
• 15:01
  For some people.
• 15:02
  It is, uh, there's no bright line, uh, because obviously there
• 15:06
  are trade-offs at every point.
• 15:08
  So are organizations comfortable making those decisions?
• 15:13
  And, and obviously if we're talking about a, you know, a, a, you know, a
• 15:17
  highly regulated or national security use cases or they've got a whole
• 15:20
  established practice, but I would assume there's a lot of companies that.
• 15:24
  Uh, this vision appeals to them and, and they have exactly the needs you're
• 15:28
  describing, but they've been so used to thinking in these terms that everything
• 15:33
  is outsourced, that they're not really at the point of understanding.
• 15:36
  So I'm just, just curious if that's an issue that you're seeing or, or if there's
• 15:40
  already that awareness in the market.
• 15:42
  So the awareness does come, I think Europe and um, other nations
• 15:47
  you would assume would be more aware of what Soeren is about.
• 15:50
  But we are seeing more and more of traction just in Americas,
• 15:54
  within different states in America, and on the call for sovereignty.
• 15:57
  So for a lot of, uh, organizations and speaking to a few nations in
• 16:01
  Africa, it is about soen and its true a essence and having control
• 16:06
  as a nation about their data and not giving that information away in the
• 16:09
  hands of either an American frontier model or a Chinese open source model.
• 16:14
  Um, so.
• 16:15
  It's the pure definition of sovereignty.
• 16:17
  But as I said, a lot of, a lot of organizations that we are speaking
• 16:21
  in Americas also do not want to give a lot of their data to the American
• 16:25
  sovereign, uh, American frontier models.
• 16:27
  Mm-hmm.
• 16:28
  So we've seen a lot more awareness.
• 16:30
  On domain models, um, which wasn't there till about mid of last year.
• 16:35
  Uh, people still needed to be educated of why domain model is
• 16:38
  gonna be better than Microsoft, which they have been working with as a
• 16:42
  Microsoft license for over decades.
• 16:44
  Um, but not everyone, to your point, Kevin, but.
• 16:47
  Many, I think more than, more than enough that we could cater to in terms of our
• 16:52
  services have been, uh, much more aware on the concept of sovereignty, the concept
• 16:57
  of domain model, and the cost of running these models to be productive with ai.
• 17:02
  Because I guess the point in time where people were piloting.
• 17:05
  By people, I mean organizations, not individuals.
• 17:08
  We as individuals are much more mature on using ai, most of us.
• 17:12
  Um, but organizations have still been piloting and still been testing
• 17:16
  on what start of the possible is.
• 17:18
  But I see 2026 to be a year where a lot of these experiments are gonna get real.
• 17:22
  So technologies like us have become, uh, part of the value
• 17:26
  chain that they're evaluating.
• 17:29
  Usually when people talk about sovereign ai, they, they start with,
• 17:33
  uh, physical location of data centers.
• 17:36
  And, and as, as you were talking about before, sovereignty in the governmental
• 17:40
  sense, uh, that, you know, we, we, you know, we're a European company.
• 17:44
  We don't want to be using American cloud providers stored in the us.
• 17:48
  How does that, uh, element of sovereignty around AI relate
• 17:52
  to what you are describing?
• 17:55
  It very, very much has been, uh, the first stance when people think of in, uh,
• 18:00
  sovereignty is think about infrastructure.
• 18:02
  Um, I think McKinsey also helped us do the job, uh, very effectively
• 18:07
  on talking about sovereignty.
• 18:08
  They released report a couple of weeks before, I think earlier in January
• 18:11
  when they talked about three layers.
• 18:13
  Sovereignty.
• 18:15
  One to your point is infrastructure where the most amount of CapEx is, uh,
• 18:19
  because you have to not just, uh, provide for, uh, uh, the chips and, uh, the
• 18:24
  infrastructure to build, uh, the, the sovereign, uh, the sovereign data center.
• 18:30
  But you also need to be able to power it and then cool it.
• 18:32
  So it is.
• 18:34
  Company is many, many, many big investment funds like G 42 that are building this.
• 18:38
  Uh, but also, uh, different organizations in different nation states that are
• 18:42
  providing, um, the, the space for that, uh, data center built and also
• 18:47
  the electricity needed to fuel it, uh, to the point that India also made
• 18:51
  the use of nuclear power private to be able to, um, to be able to provide
• 18:56
  electricity to the data center.
• 18:57
  So different.
• 18:58
  Countries have been doing a lot.
• 19:00
  And so veen when it comes to infrastructure has been a big topic, um,
• 19:05
  uh, for the last year, year and a half since President Trump came to power.
• 19:09
  Um, I think that became a big topic for most nations, but when it comes to
• 19:13
  building your own models and building your own data, people recognized and a
• 19:16
  lot of, lot the states recognized that even if they have their infrastructure, if
• 19:21
  they're still using a Chinese model, it's still using an American frontier model.
• 19:25
  They're not totally sovereign.
• 19:27
  So it came with the process.
• 19:28
  First you have the infrastructure, then you have the data,
• 19:30
  then you build the models.
• 19:32
  Um, and a big proof point to it was also being last week, uh, when UAE that has
• 19:37
  been the biggest creator of sovereign infrastructure, also released their
• 19:41
  first sovereign, um, domain model.
• 19:45
  So from their, from their university in Abu Dhabi.
• 19:48
  So that is also a proving point that, uh, nations that have been on the forefront
• 19:53
  of building sovereign infrastructure are also now thinking about the next
• 19:57
  layers, which is the data and the models, uh, which is what we built at acta.
• 20:03
  Where ultimately do you see the dividing line between where these.
• 20:08
  General purpose, uh, big foundation models will get used and the, the
• 20:12
  more domain specific sovereign models because obviously there's some level
• 20:16
  of trade off, as you said there.
• 20:17
  There certainly are situations where you're better off with it with a
• 20:20
  smaller and more focused model.
• 20:22
  But how do you see the world of AI dividing?
• 20:27
  Um, um, unlike any, uh, unlike many other technologies where enterprises
• 20:31
  first and humans are individuals and citizens are later, right?
• 20:36
  We've been part of the evolving evolution of technology over years.
• 20:40
  So when, uh.
• 20:42
  All these different amazing technologies.
• 20:43
  Even when AI ML came in, right, big banks, big organizations
• 20:47
  like Amazon had access to that.
• 20:49
  And, uh, we as individuals or small and medium sized enterprise did not feel the
• 20:54
  benefit of what AI ML could do, right?
• 20:57
  It's just in the hands of how you do credit source scoring.
• 21:00
  How do you identify fraudulent transactions?
• 21:02
  How do you do surveillance on fraud?
• 21:04
  And all that good stuff was in the hands of bigger organization.
• 21:08
  It's for the first time.
• 21:09
  With Gen ai, that the power has been more in the hands of the individuals.
• 21:14
  Um, and that has been a really big reason why we see the euphoria
• 21:19
  on gen AI like never before.
• 21:21
  I mean, we have seen euphoria on blockchain before.
• 21:24
  Um, we keep getting frenzy for quantum computers, but never at
• 21:28
  the same altitude that we have seen this for, for AI to happen because
• 21:33
  now we are seeing the individuals, uh, the power of what AI can do.
• 21:37
  So the bigger frontier, l Lns would continue to capture, uh,
• 21:41
  the massive domestic markets, the massive retail market that exists.
• 21:45
  And there are lots of markets in terms of language arbitration.
• 21:49
  If you just look at India, uh, so many different language, so many different
• 21:53
  states that still needs penetration.
• 21:55
  If you look at the entire African continent, they don't have.
• 21:58
  Anything as a frontier model of their own.
• 22:00
  So there's a huge capacity for the bigger frontier model to continue to keep making
• 22:05
  that penetration into the retail market.
• 22:08
  But when it comes to enterprise, good enough is not good enough.
• 22:13
  And you need to be able to get almost a hundred percent accuracy and you
• 22:16
  should be able to have economies to run these models for you to actually
• 22:21
  start showing benefits of ai because so far it's costs, it's not benefits.
• 22:25
  And to, in order to show benefits, you need economies, and then you
• 22:28
  need to be able to run the models.
• 22:30
  Also at a price point that is viable for the business.
• 22:34
  And that's where we see a big amount of domain specific models making
• 22:37
  foray into the enterprise area.
• 22:40
  Having said that, I'm sure the bigger enterprise will continue to keep playing
• 22:43
  with the frontier models and, uh, the open eyes and the philanthropics and the
• 22:47
  mistrials of the awards will continue to keep evolving to cater to the enterprise.
• 22:51
  But we do see ourselves about $500 billion target addressable market for domain
• 22:57
  specific models for sovereign models.
• 22:59
  That are much more cost effective and can do the job that they're assigned to.
• 23:03
  So they might not be able to say who won the, um, Australian Open
• 23:07
  yesterday, but they can totally do the work that you have assigned
• 23:11
  them to do in your specific domain.
• 23:14
  Mm-hmm.
• 23:15
  And just broadly, uh, one of the things that I've always been concerned
• 23:18
  about in the dialogue in recent years about digital sovereignty more
• 23:22
  broadly, and this is primarily on on the government side as opposed to.
• 23:26
  The, the private models that you're talking about, um, is that
• 23:29
  it's a, a fragmented vision that, uh, if we have a world where.
• 23:34
  We have this technology, which is inherently global, which can draw on
• 23:39
  people and information across the entire world, but we wind up implementing
• 23:44
  it in a way that just boxes it into individual nation states, or even
• 23:49
  individual groups within nation states.
• 23:51
  Then I worry that there's something lost.
• 23:52
  There's both, uh, technically in terms of interoperability, but also in terms
• 23:56
  of shared awareness and communication.
• 24:00
  Um.
• 24:01
  Is that a concern in the area of sovereign ai?
• 24:06
  I think from a sovereignty perspective, I always give that answer, right?
• 24:10
  If you're making and coming in from a banking, uh, background, right?
• 24:13
  And that's the conundrum we always had with the FinTech, doing blockchain,
• 24:17
  that you have open access to everything.
• 24:19
  Every transaction is known to everyone on the blockchain
• 24:22
  versus the swift transactions that we all have to deal with.
• 24:25
  It's messy, it's difficult, it's very opaque in nature.
• 24:28
  You would not want your transactions to be known to everyone.
• 24:31
  Who are you transacting with, who you're sending money to?
• 24:33
  So there's certain use cases, even for a bank that remains to be very
• 24:38
  private, that remote remains to be very accurate, cost effective.
• 24:41
  That's the universe that is very apt for domain specific models.
• 24:45
  And they're gonna be use cases that needs to be interoperative.
• 24:49
  Different banks can talk to each other, the cross border
• 24:51
  transactions, and that's where.
• 24:53
  Those are the kind of use cases which would continue to be very,
• 24:56
  very adaptive for LLMs and bigger frontier models to deal with.
• 25:00
  But you know, you do not have one size fit all for every solution and
• 25:05
  from a regulatory perspective, right?
• 25:06
  The regulators would not feel comfortable for bigger frontier models to be
• 25:11
  good enough because they cannot say, did you follow the best practice?
• 25:14
  The regulators need to know, can you prove, prove what your AI did,
• 25:18
  why I did it, and who authorized it?
• 25:20
  Unfortunately, large language models and frontier models cannot answer
• 25:25
  yes to any of those three questions.
• 25:26
  Mm-hmm.
• 25:27
  So for Gen AI to become regulatory approved, compliant accountable,
• 25:32
  you need to have control over what you're building, what you're training
• 25:35
  it with, and who authorized it.
• 25:37
  And that cannot be done for the large language models.
• 25:40
  So in by default, uh, domain specific models will have to be
• 25:43
  the play if the enterprises have to benefit from the gen AI frenzy.
• 25:49
  Yeah.
• 25:49
  This ties back to what you mentioned at the beginning, the
• 25:51
  connection to accountability.
• 25:53
  Uh, because we, we've had a lot of debate in legal forums and
• 25:58
  regulatory forums around ai.
• 26:00
  Where something goes wrong at the end of the process.
• 26:05
  And the question is, well, should you regulate the one who's deploying
• 26:09
  that particular system, or should you regulate the, the model developer?
• 26:14
  Uh, and they both have a point, the, the user of that, uh, large language model.
• 26:19
  Says, well, I I don't control what's going into chat GPT.
• 26:23
  Exactly.
• 26:24
  And open AI says, I don't control what people are doing with chat GPT.
• 26:28
  Exactly.
• 26:28
  So it sounds like what you're describing is, is potentially
• 26:31
  a way to overcome that problem.
• 26:33
  Exactly, because I mean, coming in from banking, the biggest department
• 26:38
  that is extremely regulated is the model risk management, right?
• 26:41
  They, they're very, uh, you know, definitive, uh, set of algorithms.
• 26:46
  They're not probabilistic, they're very deterministic in nature.
• 26:49
  But each of these models that are used in banking and how you do trading,
• 26:52
  how do you do fraud detection?
• 26:54
  And the likes are governed with these model regulations.
• 26:57
  And none of the gen AI models can be regulated.
• 27:00
  In the Mr. M spectrum because it's probabilistic.
• 27:03
  That's number one because they're large language models and if they're
• 27:06
  not even built by you and you have no control of what data was fed and
• 27:09
  how the decisions are being made, uh, it'll never pass through MRM.
• 27:13
  And as much as I was very, uh, I found the mrm RM to be a huge friction for all
• 27:18
  the good stuff we wanted to implement.
• 27:20
  I understand the value of why this is important, because if you are.
• 27:23
  Responsible for the citizens, for the money that you hold on account of
• 27:27
  your citizens and for your clients.
• 27:30
  All of these regulations are extremely important to make sure
• 27:33
  that you're doing the right thing.
• 27:34
  So if you are in the regulated industry, I I, I do believe that this
• 27:39
  would not be an option, but a must have to be able to understand what
• 27:42
  your models do, and you can make the model accountable or the model builder
• 27:46
  accountable for what comes out of it.
• 27:48
  Just the way we do it for Mr. M in the deterministic
• 27:52
  algorithms in the back today.
• 27:55
  One other thing I want to ask you about, uh, is that I, I understand you've
• 27:58
  been engaged in some of the efforts that the, the Vatican, uh, has been
• 28:02
  doing around AI ethics and governance, and Pope Leo has, has spoken a lot,
• 28:08
  uh, since becoming Pope about ai.
• 28:11
  So c can you talk about, um, the, the pieces of that that, that you've
• 28:15
  engaged with and, and what role can a religious leader, uh, have in this area?
• 28:23
  Uh, it's an excellent question.
• 28:24
  My co-founder has been a big part of that exercise, uh, working with the
• 28:29
  previous Pope, but I think our engagement was more with the chancellor of the
• 28:33
  Academy of Sciences and, um, uh, they, they have been a spokesperson on the
• 28:39
  fact that AI has to be accountable as.
• 28:42
  Exactly the ethics of your podcast and the control on who is building the models and
• 28:46
  who's running the model cannot be in the hands of the few, which is what the church
• 28:50
  does say because Church of the People.
• 28:53
  Um, I think our relationship with them is more in terms of, uh, you
• 28:57
  know, the ethics and the principles on which AI should be developed.
• 29:00
  And, uh, that is what we do under the guidance of, uh, the
• 29:03
  chancellor of Academy of Sciences.
• 29:05
  So we have done some work in terms of, um.
• 29:08
  Getting to a point where benchmarks could speak to the brand alignment, because
• 29:12
  you can imagine the brand alignment for church is extremely important.
• 29:16
  It cannot have a woke American model, uh, woke frontier model,
• 29:19
  so to say, to speak on its behalf.
• 29:21
  So a lot of work that we have done in that space is more in terms of understanding
• 29:27
  what is accountability all about.
• 29:28
  How do you add in the ethics and the governance that Pop Pops is
• 29:32
  responsible for with regards, uh, that.
• 29:35
  Set of alignment in that we could loop in and build it into our platform.
• 29:39
  So it's been a very exciting set of relationship because, um, we feel that
• 29:44
  we are trying to bring in the ethics, uh, from, from a brand alignment perspective.
• 29:48
  And, uh, one of the brands that has passed every test of time
• 29:53
  is the brand of the church.
• 29:56
  Yeah, it's fascinating how you pull these different pieces together because
• 29:59
  I, I, I often remark that there's conversations about AI ethics, there's
• 30:03
  conversations about AI governance, there's conversations about risks,
• 30:06
  there's conversations about regulation.
• 30:08
  Uh, and, and there's conversations about brand and, and business needs.
• 30:13
  Uh, and I, I. Try to frame accountability as something that, that cuts across them.
• 30:18
  But, uh, it, it, it's really interesting the, the way that you see
• 30:22
  all these pieces coming together and, and the, the, the sovereign models
• 30:26
  really being at the center of it.
• 30:28
  Oh, that's fascinating.
• 30:29
  That's why I'm very excited about, um, bringing ACTA to life.
• 30:33
  Um, and after having worked in banks and regulated industry, I
• 30:37
  do understand the need with which we built the act, ACTA four.
• 30:41
  And uh, also with my co-founders experience, it just is extremely
• 30:44
  important that, um, unlike many other competitors of ours who are very.
• 30:49
  Much aligned in Theca Mix and the research side of things.
• 30:53
  What we are trying to build with ACTA is the experience side of things, right?
• 30:56
  So if you've experienced building, um, uh, a, a huge platform that's sovereign,
• 31:02
  um, from, from, from the times in early 2020 years when sovereign AI wasn't the
• 31:07
  thing to actually having experience.
• 31:10
  Building MRM and regulated industry, what exactly does the industry need
• 31:14
  in order for it to accept as a model, get it convinced, get it approved by
• 31:18
  the regulators, get it approved by the compliance to put it into play.
• 31:23
  Otherwise, unfortunately, we see ourselves in the same conundrum
• 31:26
  when it comes to enterprise.
• 31:27
  I always say.
• 31:28
  Us as in the enterprise to just experiment and pilot and see
• 31:33
  what's the art of the possible.
• 31:34
  And the most use cases that we have seen from enterprise is
• 31:37
  internal information, right?
• 31:39
  Uh, can you answer questions on what information is stored where
• 31:43
  for your internal employees, it has not been deployed into
• 31:46
  a lot of mission critical, um.
• 31:48
  Topics and use cases because of this very reason.
• 31:51
  So that's where we feel that we can help support a lot of enterprise, a
• 31:55
  lot of sovereigns to be compliant, to be regulated, rightfully so, because
• 31:59
  then you could answer the question of who's accountable for this ai
• 32:04
  Tandi.
• 32:04
  Thank you so much for this conversation.
• 32:07
  Thank you, Kevin.
• 32:07
  My pleasure.
• 32:10
  This has been The
• 32:11
  Road to Accountable ai.
• 32:12
  If you like what you're hearing, please give us a good review and
• 32:15
  check out my substack for more insights on AI accountability.
• 32:19
  Thank you for listening.
• 32:23
  If you want to go deeper on AI governance, trust and responsibility with me.
• 32:28
  Another distinguished faculty of the world's top business school.
• 32:31
  Sign up for the next cohort of Wharton's Strategies for Accountable
• 32:34
  AI Online Executive education program, featuring live interaction
• 32:39
  with faculty expert interviews and custom designed asynchronous content.
• 32:43
  Join fellow business leaders to learn valuable skills you can
• 32:46
  put to work in your organiz.
• 32:49
  Visit exec ed.warden.upen.edu/acai for full details.
• 32:56
  I hope to see you there.