Latest / CXOTalk / AI Snake Oil: Princeton Professor Exposes AI Truths | #867
Transcript
- 0:00AI is a mess of truths, half truths, lies, and genius.
- 0:07So what do we do? Our guest today on episode 866
- 0:14of CXO Talk just wrote the book on this topic.
- 0:19It's called AI Snake Oil. Arvind Narayanan is a researcher
- 0:25and professor of computer science at Princeton University,
- 0:28where he's also the director of Princeton Center for Information
- 0:32Technology Policy. Arvind, I've been waiting to
- 0:36hear about AI snake oil. AI snake oil is AI that does not
- 0:42and probably cannot work as advertised.
- 0:44There is a lot of genuine progress in AI, but there are
- 0:48also many AI products that claim to do things that, as far as we
- 0:53know, are simply not achievable. Things like hiring automation
- 0:57software that claims to use videos of a job candidate
- 1:01talking and analyze not even what they're saying about their
- 1:04job qualifications, but their body language, facial
- 1:08expressions, that sort of thing in order to determine their fit
- 1:11for a position. There is a lot of this going on
- 1:15in parallel with the genuine AI advances.
- 1:18And we thought my co-author Syed Kapoor and I that many people
- 1:21need a better way to tell apart what's real from what's not.
- 1:25And that's why we wrote this book.
- 1:26You're not talking about deception, or are you?
- 1:29Are you? Or are you talking about
- 1:31technology limitations? Give us a sense of that.
- 1:35It's a mix of those things. In some rare cases I think there
- 1:38is deception. One example of this is a company
- 1:42Do Not Pay that got into trouble with the Federal Trade
- 1:45Commission for making claims to have built a robot lawyer.
- 1:49There was no robot lawyer. And they further claimed that
- 1:53any attorney who wore an earpiece to the Supreme Court
- 1:57and used their robot lawyers to argue a case would be paid $1
- 2:00million as a way for the company to show how amazing their
- 2:03technology was. I mean, it had to have been at
- 2:07least somewhat deceptive. They had to have known that it
- 2:09was a publicity stunt because electronic devices are not even
- 2:13allowed in the Supreme Court. They knew that they weren't
- 2:15going to be able to go through it with this.
- 2:17And to this day, a couple of years after that stunt, there is
- 2:20no evidence that they have built something like this.
- 2:22You can look at the Federal Trade Commission's complaint.
- 2:24It's full of juicy details of how they made things up.
- 2:28So that's one category. But much more common is you take
- 2:31a kernel of something that works well, but then hype it up
- 2:34because that's what you think clients want to hear.
- 2:37That's what investors want to invest in.
- 2:39In many cases, it's like centuries old statistics, you
- 2:42know, regression being rebranded as AI.
- 2:45And while it's true that these models can pick up some patterns
- 2:49in the data, they're not perfect oracles.
- 2:53And when you sell hiring automation software and make
- 2:56claims about video analysis, that's when it crosses the line
- 2:59for me. I'm in the enterprise software
- 3:01business and every enterprise software product is now based on
- 3:06AI. Well, it's not really based on
- 3:08AI, but it may have some AI features.
- 3:10But you're not. You're not talking about that,
- 3:12are you? Not mainly.
- 3:13There is certainly a lot of that going on as well.
- 3:16So the things we're really interested in in the book and in
- 3:19what we want to communicate to people is there are some
- 3:22categories of AI that are probably not going to work at
- 3:25all. It's not even worth buying the
- 3:27product. In other cases, there is so much
- 3:32hype that you really have to evaluate for yourself how well
- 3:35the AI product is going to work for you.
- 3:37You can't just go based on the marketing materials.
- 3:41And it's not a matter of simple exaggeration.
- 3:44AI performance can be very dependent on our specific data
- 3:47set, the specific queries that we might want to put into it,
- 3:50that in many cases customers end up very disappointed and find
- 3:53that AI doesn't do what they thought it would do.
- 3:56And then there's another category where AI might really
- 3:59work well. And this is less about the
- 4:01enterprise setting and more for a society broadly.
- 4:03And we might want to push back on certain applications of AI
- 4:07precisely because it works so well.
- 4:09And a good example of this is facial recognition.
- 4:12And to me, while there are some concerns about facial
- 4:14recognition making mistakes, the primary ethical concern is that
- 4:18it can be used for mass surveillance, especially by
- 4:21authoritarian governments. We know it has been used this
- 4:23way both in China and Russia. And it's dangerous because it
- 4:27works so well. You can pick out a face in a
- 4:29large crowd, for instance. So those are some things to be
- 4:32aware of. Those are reasons why we might
- 4:34not want to deploy AI or what might want to push back against
- 4:37AI deployments. In your book, you make a
- 4:40distinction between predictive AI and generative AI.
- 4:47Can you tell us about that as a kind of foundation for our
- 4:52further discussion? Absolutely.
- 4:54And this is where a lot of confusion around AI comes from.
- 4:57AI as of course not one single technology or application.
- 5:01It's an umbrella term for many kind of loosely related
- 5:04technologies and applications. So chat GPC on the one hand, and
- 5:07generative AI overall is very different from quote UN quote AI
- 5:12that banks might use for assessing credit risk, for
- 5:15instance. So the latter is a type of
- 5:17predictive AI. It's making predictions about
- 5:20the future, particularly a person's future, in order to
- 5:23make decisions about them. Is this person going to pay back
- 5:26alone? Is this person going to perform
- 5:28well at this job if we hire them?
- 5:30Is this person in the criminal justice system going to commit a
- 5:33crime? And on the basis of those
- 5:35predictions, enormously consequential decisions might be
- 5:39made about those people's lives, such as pretrial detention, that
- 5:43is, denying bail, which might be months or years until your trial
- 5:47date. And so this is the kind of area
- 5:49where we're very skeptical. And the evidence shows that
- 5:53these tools are, you know, slightly better than random
- 5:55number generators. And that's very different from
- 5:59generative AI, which is about producing patterns as opposed to
- 6:03making predictions about what people might do in the future
- 6:06future. So you're not throwing
- 6:08predictive AI and predictive analytics under the bus because
- 6:11there are many also very positive examples of these
- 6:16technologies. So again, how do you draw the
- 6:18distinction between what works and what is what you call AI
- 6:23snake oil? Let's talk about some positive
- 6:26examples. If we want to use statistics and
- 6:29analytics in the hiring process, there are many great ways to do
- 6:32it. We can look at the data that we
- 6:35have about what helps people perform well versus not perform
- 6:40well, and we might discover things about the applicants, but
- 6:43we might also discover things about our own workplace.
- 6:46Maybe a particular manager and their style of managing is a
- 6:49poor fit for a certain kind of candidate.
- 6:52And in fact, those kinds of insights can be much more
- 6:54actionable because you can change something in your
- 6:57organization that might benefit your hiring for many years to
- 7:01come as opposed to making decisions about individual
- 7:04candidates. So that's one thing, right?
- 7:06Analytics is different from automated decision making.
- 7:08That's the the lesson there. In some cases, I talked about
- 7:11banks, you know, predicting credit risk.
- 7:15Yes, those technologies don't work perfectly, but still banks
- 7:18do have to make those determinations despite those
- 7:20limitations. If they didn't assess risk at
- 7:23all in lending, they would go out of business.
- 7:25So that makes sense to me as well.
- 7:27Again, although there are reasons to be cautious, here's
- 7:30another example. The IRS might do audits and they
- 7:35might use AI techniques to identify who might be at higher
- 7:39risk of committing tax fraud. So I think that's a well
- 7:43motivated application as well. Here.
- 7:46The difference is that you're not making a life changing
- 7:48decision about someone. Yes, an audit is annoying for a
- 7:52lot of people, but ultimately you only get punished.
- 7:54You only get fined if you have actually cheated on your taxes.
- 7:57So the moral calculus there is different as well.
- 7:59So both because of technical reasons and because of moral
- 8:02reasons, I think some applications of predictive AI
- 8:05are much more justifiable than other ones.
- 8:09Clearly certain applications will have more profound impacts
- 8:13on the people who are involved and they have no say.
- 8:18The targets have no say in those outcomes.
- 8:22And very often, unfortunately, there's little to no recourse as
- 8:25well when there's an automated decision.
- 8:27I think a problem with a lot of our existing systems already and
- 8:31the presence of AI in the mix perhaps exacerbates that
- 8:35problem. It's already the case that's
- 8:37when a job applicant is rejected, they get no feedback
- 8:40that can help them improve for future applications.
- 8:44And so we talk about this a lot in the book.
- 8:47A lot of the time what AI does, it exposes something that's
- 8:50already broken about some of our processes or organizations or
- 8:54institutions. And I think those are
- 8:55opportunities to kind of use AI as a mirror to see maybe what's
- 9:00not working well and fix it at a deeper level as opposed to
- 9:03figuring out how to tinker with the technology itself.
- 9:06Subscribe to our newsletter, go to cxotalk.com.
- 9:11Check out our upcoming shows where you have great, great
- 9:14shows coming up. We have some questions that are
- 9:17coming in on both Twitter and LinkedIn, so let's jump there.
- 9:21And the first question is from Arsalan Khan on Twitter.
- 9:27He's a regular listener. He asks great questions and he
- 9:29says this as AI is pushed more and more directly and indirectly
- 9:35towards consumers, Should consumers have a way to opt out?
- 9:42It's a policy question. It depends on exactly what the
- 9:45application is. Opt outs can be in various ways.
- 9:49So when we're talking about generative AI, for instance,
- 9:52many artists and other kinds of creators want to opt out of
- 9:56their data being used for training AI.
- 9:59And many creative people say that opt outs actually don't go
- 10:03far enough. And there is a reason, in fact,
- 10:06to argue that there needs to be some sort of collective
- 10:09bargaining and opt out is, if you think about it in terms of
- 10:13an economic negotiation, it's between one individual and $1
- 10:16trillion company. The power asymmetry is so huge
- 10:19that it's, you know, it, it, it ends up not very much in, in the
- 10:25artist's favor because opting out really doesn't benefit them
- 10:31to a significant degree because there are millions of other
- 10:34artists whose information is also used.
- 10:36And so it doesn't really hurt the AI model.
- 10:37And so the AI company has no reason to sort of be afraid of
- 10:41artists opting out. And so that process doesn't
- 10:44really work well. So yes, that's one case where
- 10:46opt outs or maybe even something stronger should be required.
- 10:50Or you can think about recommendation algorithms on
- 10:52social media, which have been blamed for a lot of things.
- 10:55The evidence there is very much still in progress, but even
- 10:59based on what we know so far, many people want choice over the
- 11:02algorithm or being able to revert to a chronological feed.
- 11:06So those are a couple of examples where opt outs can be a
- 11:08very important intervention. Profound policy implications
- 11:14here, because it's one thing if Netflix recommends the wrong
- 11:18movie or Amazon recommends a product that you're not
- 11:21interested in. Something altogether different
- 11:23if you're applying for a loan and the algorithm incorrectly
- 11:28identifies you as being a poor credit risk and you can't buy
- 11:33the house, or the criminal justice system identifies you as
- 11:37being a having committed a crime when in fact that's not the
- 11:41case. The answer to this should be
- 11:43contestability, which you already brought up an ability to
- 11:46appeal a decision and ability to have recourse.
- 11:49And when we look at, you know, that that seems so obvious to
- 11:53say, why don't these systems already have this?
- 11:56And I think the answer to that is nuanced.
- 11:58I think what is going on in a lot of cases is AI vendors are
- 12:02coming to these decision makers that are usually in a financial
- 12:06crunch and want to save money on their decision making processes.
- 12:10And they're saying, look, you can automate your process, you
- 12:13can save a lot of money. But then when mistakes
- 12:16inevitably happen, when people complain, the AI vendors are
- 12:19going to retreat to the fine print that says a human always
- 12:22needs to be in the loop. And so if you always have a
- 12:25human in the loop, are you really realizing the efficiency
- 12:28gains? I do think there are ways to
- 12:30improve efficiency while also preserving the benefits of
- 12:34having human processes, but that actually requires innovation.
- 12:37So in, in many existing systems, it's, it's kind of one or the
- 12:41other. There's no way to have a sweet
- 12:44spot in the middle, but I I do think we can get there.
- 12:47And of course we all submit to the shrink wrap license
- 12:51agreements that we or the click through agreements that we use.
- 12:54For example, we're streaming to LinkedIn, but how many of us
- 12:59actually read those agreements? But they do provide outs for the
- 13:05technology vendors. We have another question from
- 13:09now from LinkedIn, and this is from Edith Joaquin Pillai.
- 13:16And she says, can you describe some of the reasons why people
- 13:21are so susceptible to this AI snake oil now and what the
- 13:27average person can do to be better informed to avoid AI
- 13:34snake oil? And she continues, Also, we
- 13:36don't want people to be too fearful of AI adoption either.
- 13:40So how do we encourage folks to adopt positive applications and
- 13:45I'll just add and therefore protect themselves at the same
- 13:48time? I think it depends on the kind
- 13:50of AI. Let's talk about generative AIA
- 13:52little bit, which we haven't talked about much so far.
- 13:56Generative AI is a rapidly advancing technology, while at
- 14:00the same time there's also a lot of snake oil.
- 14:02For instance, AI for detecting when students are cheating on
- 14:06their homework by using AI. Unfortunately, as far as I've
- 14:10seen, those detection tools don't work.
- 14:12So a lot of generative AI is very useful.
- 14:16So how can we encourage consumers to explore those while
- 14:19staying clear of the ones that don't work?
- 14:21So what I'll say to this is I think because of the way that
- 14:26technologists, tech companies are portrayed in the media,
- 14:29there's a lot of deference to whatever tech CE OS are saying
- 14:33today. And I think we should avoid
- 14:36deferring to them so much. I think that they are, of course
- 14:40very smart, no question there, but they also have vested
- 14:43interests to say whatever it is that's going to make them more
- 14:45money. A good example of this is until
- 14:48two months ago, virtually every tech CEO was saying they're
- 14:53going to scale these models to be bigger and bigger, smarter
- 14:55and smarter, perhaps all the way to AGI.
- 14:57And then around the time of the Nureps conference in November,
- 15:01the script suddenly flipped. Apparently now model scaling is
- 15:04over and now they're doing something called inference
- 15:06scaling or a test time compute scaling.
- 15:09So I'm not saying they were right before or they're right
- 15:11now. But the point is, if without any
- 15:14new evidence, you can suddenly, you know, completely flip your
- 15:18script because now it's in your financial interest to do so.
- 15:21Now they're, you know, some of them are saying this in my view,
- 15:24because they want to raise money for other purposes.
- 15:26It clearly shows that we can't really trust their forecasts,
- 15:29whatever they claim about their own products, etc.
- 15:32And so I think we should evaluate this independently.
- 15:34The good news with generative AI is that in my experience, a few
- 15:38hours of playing with these products will give you much
- 15:41better information about how it's going to work for your use
- 15:44case as opposed to believing what companies claim or media
- 15:48reports or frankly, in many cases even peer reviewed
- 15:51academic literature. So I would encourage you to
- 15:53trust your own evaluations. You know, maybe not to the
- 15:58exclusion of any other way of evaluating AI, but use that as
- 16:01the primary way in which you decide if generative AI products
- 16:05are useful for you. So I would encourage a culture
- 16:07of experimentation as well as a culture of having a little bit
- 16:10of skepticism about what is coming down to us from
- 16:12companies, from media, from researchers.
- 16:15Large companies, even small companies, have a vested
- 16:19interest. I mean, just look at Facebook's
- 16:21recent about face in terms of the how they evaluate potential
- 16:28fake news. Facebook wants to sell users,
- 16:33attract users, sell their service, give their service
- 16:36away. So of course there's a vested
- 16:39interest here. That's absolutely right.
- 16:41And in fact, we have a whole chapter in our AI snake Oil book
- 16:45talking about AI that's used on social media for content
- 16:49moderation, you know, deciding what content stays on and what
- 16:51doesn't, as well As for as well as recommendation algorithms.
- 16:55And one of the things we concluded was that a big
- 16:59challenge that social media companies face is that they
- 17:02don't have the legitimacy in the eyes of the public to be the
- 17:06arbiters of the public square of all of the conversations that
- 17:08we're having. And I think that's one reason
- 17:11why whatever decision they make is subject to fierce pushback
- 17:15from, you know, one side or the other, perhaps both sides.
- 17:18That's been a consistent feature over the last more than a decade
- 17:21at this point. And adding AI to the mix, yes,
- 17:24it might save them some labor in human content moderation, but
- 17:28ultimately it doesn't help that score legitimacy problem.
- 17:31In fact, if anything, it makes it worse because people are
- 17:34treating AI as well with a lot of skepticism.
- 17:36And when it's deployed by companies who are not very
- 17:39trusted, that only makes things worse.
- 17:42And I think what we're seeing now is very consistent with
- 17:44that. Because of all of the backlash
- 17:47around the inevitable mistakes that AI makes, the role of AI
- 17:50has been dialed down. But also there is a lot of
- 17:54blowing whichever going whichever way the political
- 17:56winds blow. And perhaps, you know, social
- 18:00media companies are doomed to doing that sort of thing as
- 18:04opposed to making principles decisions because they're not
- 18:07trusted by the public. Folks, if you're watching, this
- 18:10would be an excellent time to subscribe to the CXO Talk
- 18:14newsletter. Just go to cxotalk.com and we
- 18:16will keep you up to date on the amazing shows that we have
- 18:19coming up. And right now, you can ask your
- 18:22questions. Take advantage of this
- 18:25opportunity. If you're watching on Twitter X,
- 18:30use the hashtag CXO talk. If you're watching on LinkedIn,
- 18:35just pop your question into the chat.
- 18:37And truly, when else will you have the chance to ask Arvind
- 18:41Narayanan pretty much whatever you want.
- 18:45So take advantage of it. And we have a question from Greg
- 18:49Walters on LinkedIn. And he says because AI can see
- 18:56so much more than IMDBBD for for instance, don't you see AI
- 19:03having the ability to deanonymize all data?
- 19:07I think the issue here is the extent to which data is
- 19:12anonymized, deanonymized, and what that set of issues is.
- 19:18There are definitely ways to use AI to try to breach people's
- 19:22privacy, to try to link their data across one website or app
- 19:26to another, one data set to another.
- 19:28Certainly. And a lot of my research when I
- 19:31was in Graduate School, you know, 15 plus years ago was on
- 19:35exactly this topic, coming up with algorithms to show there
- 19:38are these big privacy vulnerabilities.
- 19:41Ultimately the conclusion that I came to is that part of the
- 19:44solution is technical, but I think primarily the solution to
- 19:48privacy concerns, Some of these privacy concerns has to be
- 19:52social, economic and legal. And yes, it's possible that AI
- 19:59is going to make it easier for bad actors to, to create these
- 20:05privacy violations. But generally a lot of the
- 20:08people who are doing this are commercially motivated.
- 20:10They're doing it because you know, they want to sell you more
- 20:13products or something like that. It's rarely the case that it's a
- 20:16foreign nation state adversary that is coming after a specific
- 20:20person's. Data and information, there are
- 20:23some of those people who are at higher risk as well.
- 20:26But for the majority of people it's a matter of having
- 20:29regulation out there that sets some round rules and removes the
- 20:35incentives for these deeply privacy violating business
- 20:39models. I do think we've made a lot of
- 20:40progress on that. There's no comprehensive federal
- 20:43privacy legislation in the US, but at at the state level a lot
- 20:47has happened. And of course in the EU there is
- 20:50there are things like the GDP PR which many people are concerned
- 20:53about with respect to its effect on innovation.
- 20:55So those are two different approaches that these two
- 20:57regions are following. But overall, I would say there
- 21:01has been progress on the privacy front.
- 21:03And let me just mention for folks that are interested in
- 21:06this international angle, we have a show coming up, I believe
- 21:10in April with two members of the House of Lords who have been
- 21:14very focused on online harms. And the topic of AI is going to
- 21:19center very prominently in our discussion.
- 21:22So if you're interested in that, check out the CXO talk site and
- 21:25you'll find the you'll find the date.
- 21:28We have a really interesting question from Arsalan Khan, and
- 21:31I think it follows from the earlier one from Edith.
- 21:35And Arsalan says, how can normal people identify AI snake oil and
- 21:42who should create the moral and ethical authority?
- 21:47Is it? Is it the government, the profit
- 21:50seeking organizations, nonprofits, the public square in
- 21:54in some way? So just two questions.
- 21:56How do we identify snake oil and who creates the moral authority
- 22:00around what is snake oil? What's acceptable?
- 22:03What's not? Breaking down different types of
- 22:05AI when it comes to the generative, AII think each of us
- 22:08is the expert on whether it works for us or not.
- 22:11And you know, whenever I see on Twitter, right, or LinkedIn or
- 22:16anywhere, just an everyday person who might not be an
- 22:19expert on AI but is an expert in their own domain.
- 22:22They're a lawyer or a doctor or whatever it is trying out some
- 22:26of these claimed AI products for their own purposes and posts
- 22:30their experience. Either it worked well and here
- 22:33are the ways that they're productively using it, or it
- 22:36kind of works and here are some pitfalls, or it didn't work at
- 22:39all and here are the reasons why it's very different from the way
- 22:43it was portrayed in marketing. I think those are excellent kind
- 22:47of evidence. I often find myself going to
- 22:50those kinds of experience reports.
- 22:52Again, personally I find that more valuable than either
- 22:56marketing materials or in many cases peer reviewed studies
- 22:59which have to kind of uniformly apply AI across every user.
- 23:03And so miss a lot of the nuances of the ways in which your AI
- 23:07application might be different from anyone else in another
- 23:11industry or even the same industry or a different company,
- 23:13etcetera, etcetera, etcetera. When it comes to predictive, AII
- 23:17think it's a lot harder. It can be harder for everyday
- 23:21people to do this, not because of a lack of expertise.
- 23:24That's not the issue at all, but because predictive AI is not a
- 23:27consumer technology, right? That's the big difference here.
- 23:29It's deployed by companies. You know, it might be an HR
- 23:33department, it could be the criminal justice system.
- 23:36So we don't have access, right? And so I think one step that's
- 23:40really important there is through regulation or by clients
- 23:46of AI tools, negotiating with the vendors to insist that we
- 23:50have rigorous data on how these tools were evaluated.
- 23:54That goes beyond a simple headline number like 97%
- 23:57accuracy and really digging into what data set was used in what
- 24:00context was it deployed and asking to see that data and
- 24:04judging for oneself. And in terms of who has the
- 24:07moral authority here, I mean, I would say in that sense, AI is
- 24:11no different than the way we regulate really anything else.
- 24:14I mean, it's regulation is something ultimately that flows
- 24:18from the collective will of the public and that is channeled
- 24:20through various ways. The media, for instance, and
- 24:23exposing certain problems, nonprofits and advocating for
- 24:27certain things, Companies, of course, in leading by example
- 24:31or, or or in various other ways and then ultimately regulators
- 24:35coming in and creating or enforcing policy on the basis of
- 24:40all of this. We have some interesting
- 24:43questions now coming up about industries, so let's jump to
- 24:46those. And the first one, I'm just
- 24:48taking these in order. Greg Walters asks about snake
- 24:54oil inside the Quote Education Complex.
- 24:59Yes, there is a a lot of that. I do think there is a lot of
- 25:01potential as well. But even if we go back before
- 25:05the recent wave of excitement over AII think Edtech has been
- 25:10you know a a graveyard of failed products.
- 25:13There are so, so many. We talked about some of them in
- 25:16our book, and we have an article on our newsletter also called
- 25:21the AI Snake Oil, which is titled something like 18
- 25:24Pitfalls and AI Journalism. And we go through how certain
- 25:28edtech products were portrayed in the press, including in The
- 25:32New York Times, and how a lot of the claims made by them were
- 25:35uncritically reported. And looking back a couple of
- 25:38years after those claims were made, many of those products are
- 25:41now dead. So, so why does this keep
- 25:44happening? First of all, yes, there is
- 25:45potential. There are many products that are
- 25:47valuable. I mean, as an educator, we use
- 25:49certainly a lot of tech products and increasingly some of them do
- 25:52have AI, at least simple forms of AI.
- 25:54For instance, we use grade scope for grading, which uses some
- 25:58degree of AI to cluster different student responses to
- 26:01make it more efficient for the greater to grade.
- 26:05It's not AI doing the grading, but it can help the grader.
- 26:07So those are examples of small but nonetheless useful ways in
- 26:11which you can do it. And there's a lot of research
- 26:13coming out around AI tutoring as well.
- 26:16And again, done right, I think that can be very useful.
- 26:19And when I'm in learning mode, I often use AI myself.
- 26:23There are pitfalls, but I do find it a useful technology.
- 26:26All of that said, I think the fundamental structural reason
- 26:29perhaps why there is so much snake oil in the space is that I
- 26:33think the key bottleneck to better education is not better
- 26:40technology, but kind of the social preconditions of
- 26:43learning. That's been my experience at
- 26:45least. For instance, 10 years ago there
- 26:48was the assumption that, oh, all of these famous professors are
- 26:52putting those video lectures online and they're on Coursera
- 26:55or other online online sources. And so college is going to
- 27:00become obsolete. You can hear from all of these
- 27:02experts directly on the Internet.
- 27:04Of course, it didn't happen. It turns out that the value of a
- 27:07college education is not the information that you get that
- 27:10is, of course freely available online, but again, creating the
- 27:13social conditions in which learning happens, the
- 27:15motivation, the commitments that you get and the the one-on-one
- 27:20interactions, all those sorts of things.
- 27:22And so when the bottlenecks are those kinds of social problems,
- 27:26putting too much trust in technology is almost inevitably
- 27:29bound to fail. So to what extent are the
- 27:33problems associated with predictive AIA function of, say,
- 27:39immature technology, immature algorithms, immature data sets
- 27:45versus vested interest, and the bias that's created when
- 27:51somebody has a predetermined goal in mind, whether it's to
- 27:55sell products or gain eyeballs? It's both.
- 27:58I think it's more of the latter. So the the incentives and those
- 28:01kinds of factors. But let's talk about the
- 28:03technology a little bit. I do think the technology is
- 28:06somewhat of a limitation, but not in the way that one might
- 28:09assume. It's not that in improving these
- 28:12models is going to improve anything.
- 28:14That's not where the limitations come from.
- 28:17The problem is that ultimately as long as you're taking a
- 28:22supervised machine learning approach, as as long as you're
- 28:24just using a model that is built to match patterns, no matter how
- 28:28good it is at at matching patterns, right?
- 28:30And using that to make decisions which are ultimately about
- 28:34causal reasoning, right? So, you know, if we hire this
- 28:38person, how will that affect their performance?
- 28:40That's the causal question we're trying to ask.
- 28:42And similarly in healthcare, etcetera.
- 28:44And So what we're doing in deploying AI is we're reducing
- 28:47all of these causal questions to pure prediction questions.
- 28:51And that's a way in which the technology today is
- 28:53fundamentally limited. There are lots of researchers
- 28:56working on this, including some of my colleagues here at
- 28:58Princeton. And there are ways in which we
- 29:01can integrate causal inference techniques into machine
- 29:04learning, and that can be a way forward.
- 29:08It's not going to be a panacea, but I do think it can result in
- 29:11better decision making systems based on data than the ones we
- 29:15have today. With all of that said, again, a
- 29:18lot of the fundamental problems are not about the technology.
- 29:21One example we discuss in the book is software for predicting
- 29:26which students in college might be at risk of dropping out.
- 29:30Now this is, I'm not calling the snake oil.
- 29:32It's it's well-intentioned. So you can go help those
- 29:34students and help them do better, spend more resources on
- 29:37them. And I haven't looked into
- 29:40exactly how accurate this is, but I wouldn't be surprised if
- 29:42it's accurate enough to at least be useful.
- 29:44It doesn't have to be perfectly accurate.
- 29:46The problem was how certain colleges were using this.
- 29:50There was one investigative report of a college, and we're
- 29:53talking about, you know, the long tail of colleges, which are
- 29:55under a lot of financial pressure these days.
- 29:58And the way they were using it is they were trying to use it
- 30:01right at the beginning of the semester when students had
- 30:04enrolled as freshmen in order to predict who might drop out in
- 30:09order to then encourage them to quit the program preemptively.
- 30:13So why were they doing this in such a student hostile way?
- 30:16It's because they figured that if the students quit the program
- 30:20very soon, they wouldn't count against their enrollment numbers
- 30:23and therefore their graduation rates would look better, right?
- 30:26And so if you're, you know, even if the product is built with
- 30:29good intentions in mind, even if it works really well, if you're
- 30:32using it in a way that ultimately harms your decision
- 30:36subjects instead of helps them, that's not a technology problem.
- 30:40Yeah, you have the adaptation of the technology, the use of the
- 30:44technology, and people put their finger on the scale and as a
- 30:50result you can end up with unintended consequences.
- 30:54That's right. I mean, sometimes they're
- 30:56intended consequences, but yeah, very often they're unintended
- 30:58consequences. We have another really
- 31:01interesting question now from LinkedIn, from Shale Chiara.
- 31:06And, he says, what role could academia play in holding the AI
- 31:13industry accountable for exaggerated claims and unfair
- 31:18practices? Think about the pharma industry
- 31:21where you know there has been have been a lot of exaggerated
- 31:24claims and unfair practices. And then you have the medical
- 31:29community practicing doctors as well as academic medical
- 31:32researchers. And there is a strong sense that
- 31:35while a lot of medical research is ultimately towards the goal
- 31:38of improving our medications, doctors are not automatically
- 31:44aligned with the industry. In fact, there are strong rules
- 31:46around conflicts of interest, and independence is fiercely
- 31:49valued. There's a lot of focus on where
- 31:51funding comes from now in computer science, academia, in
- 31:56contrast, of course, academia is much wider than that.
- 31:58But in computer science, among the people building AI, there
- 32:01has been no such recognition. Historically, the academic field
- 32:05of computer science grew up as really as a way to help the
- 32:08industry, as a way to build more technical, you know, prototype
- 32:12ideas and build proof of concept products that can then be
- 32:16adopted and commercialized by the industry.
- 32:18There is no wall. And in fact, going back and
- 32:21forth between the industry and academia is highly valued.
- 32:24People have multiple affiliations.
- 32:25They perceive no conflict between them.
- 32:28I'm not saying this is necessarily bad.
- 32:30I think there are reasons why it is this way, but I do think we
- 32:34need some strong subset of the academic computer science
- 32:38community whose identity is less about helping the industry and
- 32:43more about being a counterweight to the industry to, you know,
- 32:47Fact Check the claims that are coming from the industry.
- 32:49And historically, computer science does not have that
- 32:52culture. Now it is starting to develop,
- 32:54and I'm proud to be part of a very small minority of computer
- 32:57scientists who play this role. Academia is, of course, broader.
- 33:01There are so many people in the humanities, for instance, who
- 33:03are very critical of the tech industry.
- 33:05I think that's really good. And I think that kind of those
- 33:09critical voices can be much more effective if they're more
- 33:12informed by the technology. I think.
- 33:15I mean, I, I think they're doing their best, but it can be even
- 33:17more productive if they're more technically informed.
- 33:20That can be in collaboration with with computer scientists,
- 33:23for instance. So that's another way in which
- 33:25academia can be more effective at playing this role.
- 33:28Academia obviously has expertise, and one hopes that
- 33:35academics also have a more neutral perspective than the
- 33:40software vendors will have. That's right.
- 33:42And again, in most of computer science that has not been the
- 33:45case so far. And in many other cases outside
- 33:50computer science as well. The people using machine
- 33:52learning to make advances in whatever fields, whether it's
- 33:55chemistry, political science, medicine, etcetera, are often
- 33:58very prone to hype and exaggeration.
- 34:00They're not in it to make money. But excuse me, but you know,
- 34:05hyped papers tend to be more successful in the marketplace of
- 34:10ideas. So unfortunately, a lot of those
- 34:12same bad incentives exist there as well.
- 34:15And I do think academia needs to police itself so that we can be
- 34:19we can start to deserve a reputation for being more
- 34:21neutral and responsible, which I don't think we currently
- 34:24deserve. This goes to the heart of human
- 34:28nature and really has nothing to do with AI specifically, but it
- 34:32has to do with human ambition, human goals and so forth.
- 34:36And as you said earlier, AI is written by people and so AI is
- 34:40going to suffer the same slings and arrows as anything else.
- 34:46I think that's a fair observation, yeah.
- 34:48We have an interesting question. I'm going to jump to Paula
- 34:52Rooney, who is a senior writer at CIO Magazine and let's see if
- 34:58we can give her an answer that to this question that she can
- 35:00use in an article. And the question that Paula has
- 35:04is this, what are the top three misunderstandings and or
- 35:09falsehoods spread by AI vendors that experienced technologists
- 35:15such as CIOs believe? I think there are a lot of
- 35:19things that CIOs might believe because they want to believe,
- 35:23even though, you know, theoretically they should know
- 35:26better. And so one of those is claiming
- 35:32that this is this is AI or it's AI agents, which is of course
- 35:36the buzzword du jour when it's in fact traditional automation.
- 35:40SO1 trend I've observed in the industry is a lot of companies,
- 35:44you know, after a ChatGPT came out, there's obviously a huge
- 35:48fear of missing out. And so they all decided to make
- 35:50big AI investments. It's been 2 years now.
- 35:54Many companies, you know, boards and investors are starting to
- 35:58ask where the where the returns on this investment?
- 36:03What is all this AI done for the company?
- 36:06And So what a lot of CI OS are incentivized to do in this
- 36:09situation is take traditional automation and kind of rebrand
- 36:13it as AI agents. I'm not saying AI agents are
- 36:16snake oil, but they are very much overhyped, at least at this
- 36:18time. And so that, you know, kind of
- 36:22lets them fool themselves, but also in a way fool the the
- 36:26investors who've put in money into exploring AI use cases at
- 36:30the company. So yeah, that's that's at least
- 36:33one example of how CI OS in some sense fall for AI sneak oil.
- 36:38To what extent do you think that AI agents are the next coming of
- 36:43greatness? In some sense they're already
- 36:46real. So if we look at chat bots, for
- 36:48instance, when the early chat bots were released, they were
- 36:51just kind of web wrappers around large language models, right?
- 36:54You take the, you take a large language model, I mean you fine
- 36:56tune it to give polite responses, etc, and then you
- 36:59just pretty much let it loose on the web.
- 37:01So that's how chat bots started. That's not what they are today.
- 37:05They are full-fledged products. They have memory, they have the
- 37:09ability to write and execute code, they have the ability to
- 37:12search the web and find information.
- 37:14So many others things, you know, dozens of features that make
- 37:17them more useful than vanilla LLMS or large language models.
- 37:22And I think it's reasonable to call these features, many of
- 37:25these features agentic. So in some sense, when we're
- 37:28using chat bots, we're already using AI agents.
- 37:30So that's one example of AI agents that are working.
- 37:33Another example I will give is ways to do better research
- 37:41online. Not scientific research, not
- 37:43coming up with new ideas, but, you know, compile a lot of
- 37:46information about a particular topic very efficiently.
- 37:50Or compile big lists of things by doing, you know, dozens of
- 37:54different web searches and tools that try to do this, like Google
- 37:57Deep research, for instance, I have found to be quite useful.
- 38:01That's very much agentic. Those are success cases of
- 38:04agentic AI, Some agentic use cases in coding as well.
- 38:08So there are some small but important successes.
- 38:11I think the hype is around things like AI agents being a
- 38:15drop in replacement for human workers.
- 38:18Frankly, I think that's a little bit silly.
- 38:19I don't understand how people fall for that.
- 38:21When you think about your job, or really anyone else's job, all
- 38:25the dozens, perhaps hundreds of little tasks that need to go in
- 38:28into it in the course of the day.
- 38:31You know, the idea that agents are going to learn to do every
- 38:34one of them just by being trained ultimately on text from
- 38:37the Internet, I think that is very, very implausible.
- 38:40The only way to train them to do these subtle tasks in these
- 38:44enterprise contexts is putting them into practice in those
- 38:48situations and having them learn from mistakes.
- 38:50And that's going to be really a years long process, maybe a
- 38:53decades long process. And it's not going to be a
- 38:55matter of just training these agents in, you know, in, in a
- 38:59vacuum and then letting, letting them loose.
- 39:02So agents are not going to be able to automate everything that
- 39:04we do, but they are already doing some small but important
- 39:07useful things. We have a question from Tikka
- 39:11Nagi on LinkedIn Who says what is your take on super
- 39:16intelligence through AII? Think this idea of super
- 39:20intelligence that, you know that we're going to build this Galaxy
- 39:23brain. It relies on certain
- 39:26assumptions. It relies on this assumption
- 39:29that it's going to be US versus AI.
- 39:33And when we look at the history of AI, that has not been the
- 39:36case at all. As AI gets smarter, we're able
- 39:40to incorporate that smartness into our own workflows and
- 39:44through our intelligence, at least for now, being far more
- 39:47general and more flexible. By incorporating AI capabilities
- 39:50into our workflows, it's actually increasing our
- 39:53intelligence. So right now there's no reason
- 39:56to think that any time in the foreseeable future we're going
- 39:59to get into a worker versus AI, you know, a human versus AI
- 40:03dynamic as opposed to. You know, regardless of how
- 40:06small the model is, ultimately it makes us smarter.
- 40:10And so the whole notion of talking about it as super
- 40:13intelligence as opposed to a super useful automation tool
- 40:17presumes that we're going to let it loose in a way that it's not
- 40:20controlled, in a way that it's not doing things for us, but
- 40:22kind of decides on its own what to do.
- 40:25And that's a normative choice. And that's the key point that I
- 40:27want to make, whether we build super intelligence, it's really
- 40:30a question not about the capabilities of the technology,
- 40:33but about how we choose to deploy it.
- 40:35And we can choose to deploy it in a way that it's not the Super
- 40:39intelligence, which will likely be harmful simply because it's
- 40:41so unpredictable, but rather we can choose to deploy it in a way
- 40:45that augments human capabilities, again, regardless
- 40:48of how smart that it gets. Do you ever get accused by
- 40:53people in the AI business and AI vendors of kind of kind of
- 40:59throwing cold water on all of the magnificent greatness that
- 41:06super intelligence, which is so close will provide us?
- 41:09Do you ever get accused of that? We have people who don't like
- 41:12our message from a lot of different perspectives, and some
- 41:15of them are AI boosters. Some of them are people who are
- 41:19very concerned about AI and existential risk and think that
- 41:22we're trying to minimize those risks.
- 41:25We're not trying to minimize the risks.
- 41:26We just have particular views on what the policy responses should
- 41:30be. And there are people who are
- 41:31very concerned about AI ethics and the whole capitalistic model
- 41:34of developing AI and think that we should actually be much more,
- 41:40you know, vehement in our opposition to all AI, which
- 41:43we're certainly not. We're talking about specific
- 41:46types of AI not living up to their promises.
- 41:48And so in a way we see it as positive that we have people
- 41:52yelling at us from different directions instead of all from
- 41:55One Direction. So it seems like we're in some
- 41:58healthy middle ground perhaps, and I don't mind being in that
- 42:01position. The healthy middle ground where
- 42:04instead of one side attacking you or the other side attacking
- 42:07you, everybody's attacking you. That's right.
- 42:10So one of the things you discuss in the book is the the nature of
- 42:17institutions and how flawed institutions can contribute or
- 42:23do contribute to this AI snake oil problem.
- 42:26Can you talk about that? So the classic example is hiring
- 42:30an HR. So earlier I was critical of
- 42:33some of these flawed products in that space.
- 42:35But I think the reason those products are able to be so
- 42:38successful is that anyone with a job application out there, you
- 42:42know, is getting hundreds or perhaps 1000 applications per
- 42:46position. And it's just not an option to
- 42:48manually read through all of those applications and interview
- 42:51people and so on. And therefore, when an AI vendor
- 42:53comes in and says we can automate most of this problem
- 42:56for you, that seems very, very compelling.
- 42:58Even if we kind of know in the back of our minds that the
- 43:01product might not live up to the hype, it just feels very
- 43:04tempting to put our scepticism aside so that we can benefit
- 43:09from this efficiency gain. But the problem is if you have
- 43:12an underlying situation, like, you know, too many applicants
- 43:14per position, that's the problem with your institutional
- 43:17organization. Technology is unlikely to fix it
- 43:20because what's happening now is that job candidates are also
- 43:24using high to massively increase the number of opportunities they
- 43:27can apply to. So that's only creating an arms
- 43:29race. And I don't think more
- 43:31technology is going to get us out of this arms race.
- 43:34So instead I think we should be thinking about reforming how we
- 43:38do hiring. And there are so many great
- 43:40ideas out there. I know some software companies
- 43:42that hire, you know, not just based on CVS and interviews, but
- 43:46by working on a two week or one month paid project with someone
- 43:49so that they can much more deeply assess their skills and
- 43:53they're fit for the position. So that's just one idea.
- 43:55There are many other such ideas. I'm really a fan of partial
- 43:59lotteries, for instance. The idea is that we can do some
- 44:02basic evaluation of someone's qualifications for a position,
- 44:05but beyond that, trying to rank applicants is just an exercise
- 44:09in fooling ourselves because how someone's going to perform is so
- 44:13uncertain. The variation in people's
- 44:15performance in a job is not a result of one person being more
- 44:19competent than another. I mean, that explains small part
- 44:22of the variation, but a lot of it is because, I don't know,
- 44:24maybe there weren't a fit for this manager and those sorts of
- 44:27things that aren't even determined at the time you're
- 44:29trying to hire someone. So the best we can do is do some
- 44:32basic filtering and then and then select randomly.
- 44:36And if we were forthright about that, I think that would
- 44:39dramatically simplify the process and actually bring some
- 44:42clarity and Peace of Mind for applicants as well.
- 44:45And could, you know, save us a lot of trouble in the hiring
- 44:47process without decreasing the quality of applicants?
- 44:51Arvind, you're describing one specific use case and how to
- 44:57improve that process. Of course, it's going to be very
- 45:02different depending on the industry, the process, what's
- 45:06what you're looking at? Is there a way to look at AI
- 45:13itself, how we manage AI, how we potentially regulate AI from a
- 45:19policy perspective in order to drive underlying changes that
- 45:24cut across all of these use cases?
- 45:27And I, and I'm asking this because it has deep implications
- 45:33for policy efforts, whether overarching AI controls and
- 45:39policies will actually do anything in the face of this use
- 45:44case or application specific set of issues.
- 45:48I think the majority of policy and regulation should be
- 45:51industry specific, should be use case specific, but I think there
- 45:54are a couple of cases where it makes sense to at least think
- 45:58through AI wide controls. And then one is the issue of
- 46:02releasing model weights openly. Should that be prohibited?
- 46:05Should that, should the policy makers be neutral about that or
- 46:10should they in fact encourage it, invest in publicly, you
- 46:13know, using public money, building and releasing open
- 46:17weight models. That is of course a very
- 46:19contentious issue. We lean towards the open side of
- 46:23things. We've written a lot about why we
- 46:26take that perspective. So that's one example.
- 46:29And you know the the benefits of openness there are also so
- 46:32risks, but the benefits of openness I think will be
- 46:34realized across every single industry because it makes it
- 46:37much easier to take a model, customize them for yourself,
- 46:41potentially lowers costs because you can run it on premises,
- 46:44potentially better for privacy for the same reason etcetera,
- 46:47etcetera. So it can bring cross cutting
- 46:49benefits. Another one is labor.
- 46:52So if, if you don't mind, let me give a historical analogy.
- 46:57In the wake of the Industrial Revolution, eventually living
- 47:00standards for everybody went up. You know, it was kind of the
- 47:02best thing that happened. But for a good 30 or 40 years,
- 47:05it led to horrible labor conditions because people
- 47:08migrated from rural areas to the cities where safety conditions
- 47:12were horrible work hours, you know, 16 hours a day work,
- 47:16workplace safety was not there. There was no collective
- 47:18bargaining. And the modern labor movement
- 47:21grew out of that, right? And so today there is a concern
- 47:24that AI is now creating a massive reconfiguration of the
- 47:28relationship between capital and labor.
- 47:31There might be massive job losses even in areas where jobs
- 47:35are not necessarily lost. You know, AI might make the job
- 47:40involve much more drudge free because maybe now your job is
- 47:43just data labeling for AI all day.
- 47:45And and a lot of those jobs, we know how those work, those are
- 47:47outsourced to lower income countries where people are
- 47:51complaining about really horrible working conditions.
- 47:54And so maybe we need a new labor movement for the age of AI and
- 47:59that again can be really cross cutting and might potentially
- 48:02help every worker in some way. So what you're saying is at this
- 48:07moment in time, we are in this interim period between we, we
- 48:14can call it the pre AI to the mature AI where the social
- 48:21issues and job displacement issues and economic models have
- 48:26been worked out. And so we're in this this kind
- 48:28of messy middle right now. Exactly right.
- 48:31So this begs the question, what should we do?
- 48:36What are there? First, let's start with users,
- 48:42folks, business leaders. Are there frameworks that we can
- 48:46use to evaluate what's snake oil and what should we be doing
- 48:51inside our companies about this? One thing I can suggest is I
- 48:55mentioned an article earlier called 18 Pitfalls and AI
- 48:59Journalism, and that's something that's not just for journalists,
- 49:02but for anyone who, you know, watches the news or reads the
- 49:05news or whatever and is getting a lot of information about new
- 49:09AI products. You don't have to look at our
- 49:11specific list, right, but have some way in mind of knowing
- 49:16which pitfalls to look out for, because some of these are just
- 49:19recurring issues. You might see a claim like 97%
- 49:23accuracy with no context on exactly how it was evaluated,
- 49:27which doesn't allow you to reason about whether that
- 49:30evaluation is actually applicable to your particular
- 49:33situation. So that's one thing when you're
- 49:36encountering information about AI, what can you do?
- 49:38A second thing is what is the culture in your organization of
- 49:43experimenting with piloting, deploying AI, putting guardrails
- 49:47around AI? There's a role for individual
- 49:50workers here. There's a bottom up aspect, but
- 49:52there's also an important top down aspect.
- 49:54And from a top down perspective, I think companies need to do a
- 49:58bunch of things. They need to set the right
- 50:00guardrails. Privacy and confidentiality
- 50:02can't be left to individual workers.
- 50:04Those have to be enforced at a company level.
- 50:07And there are some more subtle issues as well.
- 50:10So for instance, Warden Professor Ethan Mollick has
- 50:13written about how in a lot of cases, when employees experiment
- 50:17with AI based innovations, ways to make their work flow faster,
- 50:21they may be reluctant to share it throughout the company
- 50:24because they're worried that now they're just going to get more
- 50:27work and they're not going to get any credit for this
- 50:29innovation they've introduced to their colleagues in the company.
- 50:32Right. So that again, has to be
- 50:33addressed in a top down way. How are you giving people the
- 50:37freedom to experiment, come up with new things and then reap
- 50:41some of the rewards of of doing that?
- 50:44So those are a couple of examples.
- 50:46There are a lot of other things, but I don't want to, yeah, I
- 50:49want to make sure we get to all the questions.
- 50:51And how about from a marketing perspective?
- 50:55Because when we talk about snake oil, it seems to me that you're
- 51:01really talking about the delta between what is quote, UN quote
- 51:06advertised and the expectations that that creates between that
- 51:12and the reality of the outcomes. When we look at marketing, there
- 51:16are so many different ways in which things are over hyped,
- 51:20starting with just the decision to call something AI, right?
- 51:23And you know, there, there, there's such a huge incentive
- 51:26for that. So maybe on that one, the market
- 51:29will correct itself a little bit.
- 51:30There's, you know, starting to be a bit of a backlash to
- 51:35everything being called AI. So maybe people will stop
- 51:39slapping the AI label on everything.
- 51:41But if you're do going to call something AI, one really basic
- 51:44thing or maybe 2 basic things I would expect is OK, which part
- 51:48of this is AI? What, what kind of AI and what
- 51:50task is the AI being asked to solve?
- 51:53Just, you know, being clear about that would bring a lot of
- 51:56clarity so people can assess for themselves.
- 51:58Do we do we think this is even a task that AI is capable of
- 52:02doing? And then going a step further
- 52:04and showing the evidence behind it, right?
- 52:05I think those two steps will address really the majority of
- 52:11AI related hype out there, but I know it's easier said than done
- 52:14because there are so many of these misaligned incentives.
- 52:18Well, there's a lot of pressures inside software companies to get
- 52:22those eyeballs to sell those products.
- 52:27And AI is this black box in many cases.
- 52:33And so it's very, very tempting to indicate, even if you don't
- 52:38directly say that, hey, you know, we have this, this thing
- 52:43and it'll solve your problems. Couldn't agree more.
- 52:46What about policy makers? Where do you stand on government
- 52:53policy, should it happen, and what should it consist of and
- 52:58how should we get there? Yeah, a lot of people like to
- 53:01poke fun at policy makers or complain about them, but I'll
- 53:05stick my neck out and say that so far they're doing a pretty
- 53:08decent job on AI. Certainly I have my complaints
- 53:10as well, but I do work with the policy makers in some capacity,
- 53:15you know, on a weekly basis at least.
- 53:17And I've gotten to observe them up close.
- 53:20And I'll say a few positive things and I'll say where some
- 53:24areas where things are not working so well.
- 53:26So it's not at all true that AI regulation is a Wild West.
- 53:30In 2024 alone, somewhere close to 1000 AI bills were introduced
- 53:34in the 50 state legislatures in the US and really dozens of them
- 53:39have passed. And in fact, many people are
- 53:42concerned about over regulation. So this Wild West idea is not
- 53:47true at all. And I think a lot of AI
- 53:49regulation has been focused on how people use it as opposed to
- 53:53the technology itself. And in the US, we generally take
- 53:57a sector by sector approach and I think that's broadly the right
- 54:00way to go about it. Another area where there's a lot
- 54:04of concern is policy makers are not very tech savvy.
- 54:08And that's very much true. But the politicians you see on
- 54:11TV are not actually the ones making policy, right?
- 54:13It's there staffers, it's agencies, it's attorneys general
- 54:17and so on and so forth. And those places, the amount of
- 54:21tech expertise has actually massively increased in the last
- 54:23couple of years because of the concerns around, you know, chat
- 54:26to GPT and AI risk and so on and so forth.
- 54:28So those are all areas of progress.
- 54:30I think there is still a long way to go.
- 54:33I think there are concerns, for instance, when you compare the
- 54:38US and EU, some people will say we're far behind compared to
- 54:41them. We don't have much legislation
- 54:43at the federal level. And there are concerns about
- 54:47that. Maybe those 50 states approach.
- 54:48It's not the right way. It just imposes too many
- 54:51regulatory burdens on companies. So those are all things to
- 54:54complain about. You know, things aren't perfect,
- 54:56but I think we have to start from a recognition that at a
- 54:58high level, things are going OK. It's not a disaster in the
- 55:01policy space. Arslan Khan says how to create
- 55:06an ideal AI ready organization? Do legacy organizations or
- 55:12start-ups have a better chance at it?
- 55:15I don't know if it's legacy organizations or start-ups, but
- 55:17I would say I think it really has to be oriented around your
- 55:21industry and your use case, right?
- 55:23I think, you know, AI is often looked at as a as a magic
- 55:27bullet, but do you have to start with what your problems are,
- 55:31right? And then look for how AI can
- 55:33help in your existing workflows as opposed to seeing it as a
- 55:37kind of one stop shop solution. So I would kind of flip it
- 55:40around. I would start from What are the
- 55:42problems you need solving? How can boards of directors
- 55:46evaluate ethical reputation, financial vulnerabilities, and
- 55:51technologies like AI which are highly technical and very
- 55:57rapidly changing? The good news there is that
- 56:00while it's true that AI is highly technical and rapidly
- 56:03changing, when we think about what are the aspects of AI that
- 56:08gives give rise to harms or concerns or a reputational
- 56:12risks, those are known quantities, right?
- 56:14So for instance, when we were talking about much simpler forms
- 56:17of AI before simpler machine learning models, bias was a big
- 56:20issue and it's a big issue with generative AI models as well,
- 56:23right? And we kind of know how to audit
- 56:25for bias, and it doesn't depend too, too much on the specifics
- 56:28of the model. And there's a whole community of
- 56:31AI bias auditors and they're working on more recent AI models
- 56:35as well. And so if we break it down, not
- 56:37so much by the technology, but by what the harms and risks are,
- 56:40what are the ways things can go wrong?
- 56:42What are the processes we've put into place to minimize those
- 56:44risks? Then we're not so susceptible to
- 56:46the whims of the technology. And I'll also just mention here,
- 56:51we were talking earlier about academia and there are academics
- 56:56who are really looking at this bias question and I'm sure that
- 57:00they can be of help as well to to business people.
- 57:03Definitely. And with that, Arvind Narayanan,
- 57:08thank you so much for taking time to be with us.
- 57:11I'm very grateful for your time. It's been very interesting
- 57:13discussion. Thank you so much for having me.
- 57:15Loved all the questions from you in the audience.
- 57:17And a huge thank you to everybody in the audience.
- 57:20Before you go, subscribe to our newsletter, go to cxotalk.com.
- 57:26Check out our upcoming shows. We have great, great shows
- 57:30coming up. Everybody, I hope you have a
- 57:32great day and we'll see you again next time.
- 57:34Take care.