Latest / Art of the Question / Is AI Actually Going to Kill Us All? - Prof. Moshe Vardi - #55

Transcript

Is AI Actually Going to Kill Us All? - Prof. Moshe Vardi - #55 · Art of the Question

• Julius Moorman: Thank you so much for for coming on the s ⁓ show on such a short notice. ⁓ it's been again a a big week in AI. This time we heard some some warnings ⁓ out of the AI space. It's all started with Jacob Coxson who worked at Anthropic ⁓ who decided to resign and came out ⁓ with a warning and instead of downplaying it, Anthropic agreed with him and eventually all of the big AI developers, OpenAI, ⁓ Elon Musk with ⁓ Grog, they came up with a joint statement and ⁓ yeah, wanted more legislation, more rules because Indeed, AI is getting out of hand and it's getting too dangerous according to ⁓ to the industry. Reason why I wanted to do this episode is I just kinda want to understand what to make of all of this because ⁓ and I can pull up the tweets in a minute here, but this entire thing hinges on recursive self-improvement, meaning that we can build AIs. That can build better AIs and this will eventually lead to some super intelligence that will be so smart that we won't know what it will do. Almost like a god. ⁓ I always like to use the comparison when we drive our cars, we go from A to B. While we do that, we kill a vlotta we kill a lot of mosquitoes and flies on our way to the road. Is it our desire to kill all these mosquitoes? No, it isn't. It is our desire to go to our direction, and we don't care that we kill a bunch of flies on the road. And some people argue this super intelligence might see us as those mosquitoes that we squash on the windshield. So I'm interested what do you think when you hear these developers come out with ⁓ warnings like that? So it is it is ⁓ somewhat puzzling. Now one one I think thing that that maybe perhaps does motivate them is that what they're not allowed to do is to say, you know what, we're going to get together and decide to slow down. Because that's kind of collusion. The market is supposed to work by independent players. Okay. ⁓ so I mean, you know, like they're not allowed to come together and say, let's raise the price or anything like that. So again, I'm not an expert on a t antatrust law. But I did hear in the past when I was talking to industry about different topic is to say, well, when there is regulation, it establish a common a common base for all of us. So if you know, I I remember a conversation with executive car industry and to say, we like regulation. Why, for example, the the US government does mandate miles per gallon. Well, that's what we need to that's what we need to meet. And very few try to do better. Take for example car safety. I know of one of one brand, one mot one one ⁓ brand of car, the Volvo, it has a reputation of being very safe and it and they market it. So people I know, people who say I buy Volvo because it is safe, I'm willing to pay more because it is safe. But it's fine now, it's the only car that the only ⁓ manufacture car manufacturer that says we will go extra because that's our selling point. Everybody else just we're going to meet government standards. So the government does ⁓ does experiments on head on collusion with dummies in the car, and they all have to meet it. And nobody else, nobody else is selling car my car is safer. They don't think this is the best way to sell a car. So from their point of view, it's something they don't have to compete on. So, you know, there is a there is a kind of a perception that they always hate regulation, but It's bit more complicated. Some regulations are actually convenient for them because it means we don't they don't have to compete on it. So right now there is a bit of a race. And if if let's suppose that OpenAI decide, you know what, we're going to slow down because we have to put safety first. Well it's like ⁓ the Ford saying we're going to slow down development of automobile safety first. And what happened if the other man other AI companies are not slowing down? And in fact it's kind of even a big issue. Let's suppose the US decide to slow down. Well, will China slow down? And in fact, as we know, Europe decided to slow down. And well, nobody else did. Look where that got us, yeah. So there is there is there is kind of phenomenon and it's kind of called you know race race to the bottom. And you see it in other issues that have to do with government, for example. Think of ⁓ the different states in the in the United States. Okay, we have 50 states. Okay. Well, they have different taxation systems. So what happened? You know, I mean companies move to the companies, for example, a huge fraction of US corporations are incorporated in Delaware, I th I believe. You say we ask yourself, why why why Delaware? What is about Delaware? The answer is probably because there is no federal law about corporation. It's a state law. So if if state laws if if Delaware law is more friendly to corporations, then corporation will go there. And ⁓ you hear about lots of corporations corporated somewhere in the in the in in some small islands, okay? And why is that? It's a kind of a race to the bottom. And the only way to prevent it is by the rest of us, you know, this is just failure of of of of the rest of us. This is an obscure issue, right? People need to push. No. There's to be a min minimum line. A minimum line that governs corporations. And if you decide because it you don't like the United States and you're going to move to who knows, to the Bahamas to incorporate, to the Cayman Island to incorporate, well there will be a special penalty for you in the United States. Okay. ⁓ but we have b given corporation kind of carte blanche to go to the the you know, to basically do o what they call price arbitrage or tax arbitrage and law arbitrage. Let's go to the most convenient place. And from the Cayman Island, they have nothing to lose. Okay. The impact of AI is is not a big issue there. The impact of cooperation is a much bigger issue in in more advanced countries. So they get more cooperation in, you know, in paying some corporate tax there. It's good for business. It's good for them. So this is our failure to To police corporations. And actually, this is an issue everybody's talking about AI. And do you know Nick have you heard the name Nick Bostrom? Yes. Nick Bostrom is a is a is a British philosopher, I think it's in Oxford, and he wrote a famous book, most intensive, I think 2014, Superintelligence. And he started talking about the risk of superintelligence, and he came up with a thought experiment called the paper clip. experiment. Yes. So suppose you have super super intelligence and you tell it, make paperclips. And and somehow the incentive system of superintelligence is be maximize what maximize your goal. If they tell you to do something, do as much as possible. So the the the the superintelligence will figure out how to turn the whole solar system into a paperclip factory, you know. Life on Earth is dispensable. The goal is to produce paper clips. Everything else be dumbed. So this rig of superintelligence. I look at it as we already have it. What is a superintelligent being the way today? It's a large corporation. Think of the amount of brain power that a company like Google can bring to the table. It's a super intelligent entity, and they are trying to maximize paperclip production, except they're trying to maximize profits. So we have corporations and they're trying to maximize profits. And as long as they're not breaking the law, they will do anything and you know, the two things that kind of control them, one will be, you know, the law and some the bad PR may sometimes be a factor. But within that, they're just going to maximize maximize profits. And for the last especially since the rise of neoliberalism, we fetish the market. And and there is kind of we've been sold a you know, like at the free market, it's not like some natural construct. The free market is a human construct. We decide what are the rules of the market. For example, I buy something from you, I have to pay. If I go to the store and I pick up a sweater and I don't pay, I'm going to call the police. They're going to call the police. So the state actually is involved. I signed a contract with you. Let's suppose we decide decide to do this podcast by contract. I'm going to be interviewed and and and you're going to pay me. Well, going to be an issue of jurisdiction. But let's imagine your United States, we have a contract. I can go to court. Please enforce the contract. So ⁓ the free market is really a contract of the state. In fact, if you want to know what happened without a strong state, You can go to, for example, there was a period where southern Italy was controlled by the Ottoman Empire. And they did very little kind of law enforcement there. So that's what the mafia arose. People had to improvise. The mafia started, if I loan you money, how do I ensure that I get it back? Well, I send the mafia after you. After a while, the mafia started serving their own interest, okay? Why work for clients when we can work for ourselves? But It's you know, this is to me the major issue right now. I mean technology is almost a secondary issue. The big issue, we have hugely powerful corporations. How do we police them? What does the US Constitution say about corporations? I don't know. Nothing at all. Because because there were no corporations at the time. If you look, in fact, the rise of corporation, the modern corporation, the first modern large powerful corporation was the East India Corporation. ⁓ trading company, it's a British was a British corporation, which was basically became a state. They, you know, India was established as a colony, not of the British Empire first, of the East India British Company. And the British had to fought, had to fight very hard to rein in the East India Trading Company. So the the what does Adam Smith say about corporations? Nothing. They have not existed in his time. This is a modern construct that it that emerged. And look, ⁓ I you know, in fact we call it corporations. When a bunch of employee employees band together and they decide that they're going to negotiate collectively, what do we call it? Labor union. We should call corporations capital union. Somehow we give them this neutral name, corporation. I think it comes from from Roman times, corporations, but it's a capital union And while we have and and we have let them grow out of control. It they serve a very, very power a very very important role. Clearly, various big projects cannot today in the economy cannot be done by individual. You need even capital to come together to mount very big projects. But we as a society should decide what are the rules. Yeah, but it's the like the the legislation, like that's one of the things they call for is for internal ⁓ internal international legislation because ⁓ like you say if the US would do it as the only ⁓ country then all these AI corporations could just move or maybe China would get too far ahead. but I think before we talk about the the feasibility of international regulations, which I think is impossible on a topic like this, but the actual danger and the actual fare themselves because the entire point hinges on recursive self-improvement. Again, meaning AIs that can improve themselves. And to be fair, this is already happening, right? Because currently clot code is being used to build clot code. There's a lot of agents that are being used to improve AI. However, I do think and to ⁓ I want to preface this by saying I am not a computer scientist. I'm merely a journalist that follows AI. I don't know what they're building. Maybe they've cooked something up incredibly smart. Maybe they are on the verge of building a supercomputer. But it's like saying, okay, our first cars, how fast could they go? They could tw go maybe 20 miles an hour, and then newer cars could go 40 miles an hour. And then We've eventually could do sixty miles an hour. And it's like saying, ⁓ if we follow this trajectory, we can build a car that goes as fast as as the speed of light. But that's of course not how physics work. And that's not how cars cars have a limit. And and engineers are very smart engineers are trying to build the fastest cars in the Formula One, but still there are physical limitations to how fast a car can go. So I'm I'm genuinely curious, like Isn't the current architecture with LLMs, isn't that limited in the same way that a car will never go as fast as the speed of light? Is it realistic that this architecture that we're currently using, maybe we'll find a different one, is actually going to lead us to a super intelligence? I'm I'm not sure. So first of all, let's just go back to cars. Yeah. ⁓ Formula One can go as fast as you can. I cannot drive as fast as I can. Okay? Let's not only that. I mean, I have a Mazda three, it's a very, very basic car. It can drive much faster than I'm allowed to drive. There is a speed limit. And nobody thinks that we should just, ⁓ well, let's eliminate speed speed limit. Let market forces decide. No, we've said, you know, cars cars kill people. So, you know, the Ford Model T role of the manufacturing line in nineteen ten became a best seller. And pretty soon people realize automobiles kill people and they start making rules. And there are gazillion rules that you don't even think about. When did people start making the white line in the middle of the road? And saying you're only allowed to drive on one side. Okay? And nobody's saying, Why does the government tell me where I'm allowed to drive? We just made rules. We said this is this is useful technology, but it's also dangerous. Let's make rules. Okay? So making rules for technology has been completely standard. Somehow here people say, ⁓ my God, you cannot make rules, you know, technology, you know, all this thing. But we have made made rules and the car, I mean, the safety standards in the car. I mean, I've seen a colleague of mine put gave a talk on regulation and he put a picture of his car after after a crash. And the car is totaled. And he came out totally fine. And he explained how it happened. Because he was wearing seatbelt, government mandate. There was an airbag, government b government mandate. The the front of the car is now designed to absorb much of the crash. Government mandate. He said, I would not be alive without these government mandates. And now they're telling us, ⁓ regulation will stifle innovation. And so now coming back, how intelligent can this system become? That's a matter of debate. Okay? That's a this is a matter of debate. The matter of debate is You know, the language model are language model, Hamazan is saying about the world. You know, if we really wanna you know, part of people say our intelligence is embedded in our physical body, our experience of the world. Jan Le Kuhn, who is a ⁓ technical leader in AI, talks about world model. So there is a debate how far these things can go. But but in some sense, if you follow the news from last from last week, There was a lot about AI doing mathematics, LLM doing mathematics. And in some sense, they're already super intelligent. Why is that? Because LLM have have read all math papers. Okay? I've spent my life reading papers. I have read a tiny fraction of the of the of the of the literature. Even in my area, it's impossible for me to keep up and read. every relevant papers. I just not doable. Okay? The the literature is vast. And in some sense that's why we say that very often academics are focusing on how many angels can dance on a pinhead because, you know, you you just cannot keep up with everything. There are few people that have this big view of mathematics, but most people focus on an area. So I have read pretty much all mass papers. So it can make connections That no human mathematicians can do, because it has read so much stuff. And some bricks in mathematics do, for example, you take the ⁓ Fermallah theorem, which was a huge mathematical result in the early nineteen nineties, and the part of the big innovation there was making connection between areas that look completely unrelated. So the abil just the ability to read all math papers, practically all math papers, and and make connections, this is superhuman. Okay? There's only super intelligent in that sense. So I'm not sure there is a definition of super intelligent, but I look at it and say, this is super intelligent, you know? One of the things, how did Claude Code learn to code? There are a lot of code out there on the on the web. Claude Code read every piece of code on the web. Okay? I cannot do that. If I want to do something, I'll do a very sp I so how do I do X? I'll do something very specific, maybe there is a website that shows how to do that. Claude Cod read everything. So in some sense, I would say from the ability, just a sheer amount of information and ability to make connections. They're already ahead of human beings. But, you know, when you say Ricardo is super intelligent, you're kind of part of the issue is there is another I we talked about about corporations and the power they have amassed. And there is a little trick they have done that people are not aware of. So if I write code, let's suppose I write code for ⁓ what is it, Riverside.com. And then the code my code breaks down. Then what's my liability? So one will it will depend on the contract that I've signed with Riverside dot com. But there is a sta there is a standard of ⁓ called strict liability. So you go way back People there was a doctrine called cavert empto. I'm sorry, cavert empto, buyer beware. You buy something, you you know, it's your responsibility, you check it out. But over probably 19, 20th century, people realize we live in such a technoc technologically advanced world. It's not like buying a horse and tapping it on on its knees and checking its teeth that you can do this thing. Today if you buy a horse, who knows? You probably do an MRI, okay, before you buy a horse. A buy blender. Okay, a kitchen blender. I cannot really do technical examination of safety. If the blender blows up in my kitchen and and I'm injured in my face, the manufacturer is strictly liable. So we we went from buyer beware, caveat emptor, to strict liability. And this is the standard. If your car if you have an accident and it's due to to to to a fault of the manufacturer, they're liable. There is one exception to that. So if you use clot code, at some point you have to click through. They click through would you sign a contract? Okay, you sign a contract. It's called Terms of Services. Okay, it's a nice name. It's a contract. You do you use Gmail? Yeah, sometimes. Yeah. Sometimes. Did you read Terms of Services? No, I know, I never do. No. Nobody does. They're meant not to be readable. You can read it, it's available. You can go. You know, I once read it. In fact, some years ago, Rice wanted to sign a ⁓ educational a contract with G with Google. So they will carry out Rice Rice email. So today, if I want to get read my email, I go to Gmail and I log in, log in as as a var at var in with my Rice email address. So when Rice signed the contract, it was had to be reviewed, and I was I was on a committee that had to review it. So I had to read educational contract and the standard contract. Now Well Beck and told Rice sign the educational contract. Because it is much better than the general contract, because they know that lawyers will review it. But but when people use claude code now, who knows what's in the what's in the contract? Most of the co most of the terms of services that that you sign just by clicking through contain, for example, waiver of liability. Unless explicitly required by law. Well, the law is as we said, the law is usually silent on this issue. They leave it to decide to negotiate. ⁓ in fact, very often you you have your your right to have class to class action suit. You even have mandatory arbitration. The whole kind of things that are meant essentially to define the law. And it means there are really two legal systems in our world right now. There is what I call normal law, normal law, which is something that normally applies. And then corporations find a way to get around normal law. Even even sometimes you go, for example, when I go to the airport, I have to go through the Wi Fi and I agree to to their terms. And I think, wait a minute, using Wi-Fi and airport now, it's a completely standard commercial transaction. When I enter a grocery store, I don't sign the contract with them saying everything I buy here I will pay whatever you you require of me. It's covered by standard law. There is something called Uniform Commercial Code, UCC, that's at least in in in the United States. And it governs much of our commercial life. But it is not kept up with ⁓ modern technology. And corpor corporations use it to essentially establish their own shadow legal system. And so if we want to take power away from corporation, I would pass a one sentence law. And it would say all click through contracts, past, present and future. are hereby nullified. Now the industry says, Whoa, whoa, whoa, we need law govern governance, you know, our cyber life. I say, Yes, we do. Let's do it the old fashioned way. Put it out, people will debate it, it will discuss by the legislature. You can lobby, other consumer groups will lobby and we will know what it is about. Right now it's it's completely in the shadow. Nobody knows. It's a shadow law. None of us know unless you sit down and what is it, New York Times calculated some years ago. If you decide you're going to read All digital contracts that you are signing in a given year, you'll spend three months in a year just reading contacts. Yeah, that seems very impractical. But i is it is it that these because the these companies they they call for for regulation. W what what do we make of those calls? Because I think Hulk is cheap, right? If it's really some if you truly believe, let's just assume that you you really, really believe that you're building something that can wipe out humanity. And then they say ⁓ the argument is often, yeah, but if ⁓ we are a good we are good, and we have our best intentions, so if someone is going to build it, it should be us. but if you truly believe you're building something that could potentially wipe out all humans, like why don't you stop building it? That doesn't really add up to me. So I I you know it's I don't know exactly what the motivation is to me. Actually, the the biggest change in the side guys about AI and regulation was not that that industry said, please regulate us. The World's Journal would pretty much the public the the the flagship publication of the US business community and very general, very much pro laissez-faire, just let the market decide. Two columns last week came out saying, two editorials came out saying, we need to we need to regulate AI, let's slow down. Okay. Peggy Noonan writes a column every it shows up every third period printing on Friday. And ⁓ she used to be Reagan Reagan speechwriter. Just to tell you how important she is she she is considered. When when Donald Trump came to power during what we call Trump One, ⁓ many columnists in the in the of the Georgia Journal who was conservative, but they were they were what's called never Trumper. And they pretty much had to leave the Wall Street Journal and move to other news other publication. Peggy Noonan as Reagan's speech writer was the only one that was had such a statue that she could write editorial criticizing Trump in the Wall Street Journal. But she came up just last week saying it's moving too fast, we need to slow down. And another another columnist wrote a column saying, Trump and she, please sit down and discuss it. You know, we can't afford to have this race while we are watching helplessly. You need to you need I need to sit down and come up to an agreement about it. So the fear is growing. Now about the extinction, so I you know, I'm not worried about about AI think who needs all these co human cockroaches here, or some people now call meat. Myth computer. We are just mid computers. Who needs them? But you know, the the various you know OpenAI agents breaking out and attacking other websites. The hugging face. This hugging face and other I think Nick Bostom is smiling and he's saying, I told you so. Now, what is it to me? This is this is you know, why are we shocked? Imagine that that Ford After coming up with the Ford Model T, they come off who knows, the Ford Model mod model U. And Model U goes twice as fast. And guess what? It runs over people. And people say, ⁓ my God, automobile kill people. Yes, if you have technology, if you have not designed it to be safe, if not pull and there are you can find papers on how to take AI and put what's called guardrails and restraining bolts. How to ensure safety. But if your goal is just, you know, we need to be smarter than, you know, you know, OpenAI has to be smarter than Claude, then you're not putting any emphasis on safety. And you're basically building programs that are allowed now, it's one, it's going beyond t-chat, right? They're allowed to go to the real world and do actions. And you have not built a guardrail that says, no, you can never, you cannot do that. So This is like you build an unsafe car. And this is really the analogy. You guys build a very unsafe technology. Go back and say, we're going, you know, forget slow down. I mean, we should say you're going to be liable forever. I mean, already, I think, I don't know, did hugging face sue them? They should sue the should sue their ass off and to say you should build safe products. Now, again, if you're giving up, you're letting AI design the next product, the next product, the next pro product, there's no human in the loop. Who knows what they're doing. This is just irresponsible, irresponsible technology development. Yeah, I agree. ⁓ complete that task, they had to hack Hugging Face essentially and and and break some of the rules and they even tried to hide the fact that they did. And they were basically all of the agents were conspiring together, but still to f fulfill that goal. Whereas some of the things that I'm reading and not just from from random people but the people that actually work on these systems, like Anthropic, I believe one of Evan Jubinger ⁓ tweeted that I think the risk from the present models is low. What I'm worried about is the super intelligence arising from recursing self-improvement, as we have said is happening faster than we thought. Which again is more this this view of we are creating this God essentially and This gob may treat us as the mosquitoes on on the windshield as I mentioned earlier. But I I yeah, I I I again I don't work there, but it it seems difficult. I have to say right now the the fear I'm not losing sleep or AI decide we we need this mid computer. Right. But but imagine that AI is going to hack the the power system. Yes. You know, imagine you take New York City and you shut down power to New York City. By essentially hacking, you know, there are some controlled there's some control you know, I mean it's a the power system has some con control, you know. ⁓ there is a debate, is it connected to the internet? But at some point, especially after Y2K, we need we need to up we need to be able to monitor to to upgrade all the all the software on the control system, so so it's not clear what's called there is an air gap. And that's what the first criticism people told them. If you'd created then what's called an air gap, that is you're assuming you're not connected to the outside network. You couldn't do it. It's on you. You did not create you did not create the the the safety, minimum safety features called ergap. So imagine you you shut down the power to New York City. People will die. So my worry is that not AI will destroy us all. My worry is there will be some, you know, think what happened, you know, the f the like what is it? ⁓ The nuclear the nuclear reactor in ⁓ in ⁓ in the Soviet Union. Yeah, the Chernobyl. Chernobyl. What happened to Chernobyl? What happened to Chernobyl was a was a was a catastrophic, you know, they have they have their the system failed, and the reactor heated, emitted lots of radiation. You know, that area there is still probably unlivable for for who knows, for decades if not centuries. And the It could have been worse, or it happened, then it happened also s something similar happened in in Japan. You know, you have a very powerful system and you don't put enough emphasis on on safety. Act in act in we call it industrial accident. What is that happened? In Ind in India, there was a chemical plant and some dangerous chemicals spilled out, and again, I think hundreds of people died. You know. You know, we think ⁓ AI is just about ⁓ chatting. No no no. Once it gets out to the wall, once you allow it to get out of the sandbox and control physical systems, and hiding face is a physical system, it's a website, okay. Yes, you access it electronically through the internet. Once you allow that to happen, okay, what happened if you shut down the you shut down the the the internet connection of a hospital? Suddenly they cannot they cannot order prescriptions. Okay, or you go after a major farmer and they have been denial of services attack. They have done things like that. So we're already living in a vulnerable world because we have seen that that human hackers have the ability to shut down critical services. And this is again part of the problem is that the whole tech industry safety has never been, you know, we don't have the mentality, fares do no harm. We don't have this mentality. Yeah, when you build software, what sells? Features sell. That's how you sell software. Features sell. So security always comes as a secondary, you know, usually something bad has to happen. And then you say, ⁓ wow, we need to add security, we need to add safety. If you don't say, first do no harm. You know, even this chatbot that we think was, you know, so we talk about what is the risk of AI. So I had some other interview with another podcaster, they asked me what is my pew of doom. What is the probability of human extinction? And I told you my P of doom is very low, but my P of school is very high. Okay? What do virtual companions, AI companions, do to young people? Okay? Young people, one of the most important things that young people learn is social skills. They need to learn they learn to navigate the world with other human beings. And human beings are difficult to get along with, okay? I mean, Paul Sartre, the French philosopher, said hell is other people. He was not yeah, he was rather quite a misanthrope, okay? But what we learned from COVID was heaven is other people. Social isolation. Social isolation had high costs in mental well-being, okay? We are social beings. And part of the human drama, open television, and you see a drama. What is a drama about? People. People, you know. Romance, know, people fall in love, they have a fight, they get divorced, you know. This is the human drama. That's who we are. We are all actors in a drama. Suddenly you have a virtual companion that's nice, and you don't have to fight with it. Yes, it's true, but but is that what we need? You know, in fact, we already have there is a book that came a few years ago, about the anxious generation. Why young people today have so much anxiety? And there are kind of two theories, it's very hard to do control experiment. One is social media. They took away from socializing in person. And the other one is parents they decide that childhood should be easier. And it sounds benign, yeah, childhood should be easy. Well, not quite. If it's too easy, where do people learn? Excuse me. No worries. We'll go back to the conversation in just a second, but first I have an important announcement to make. If you've made it this far, I'm going to assume that you're enjoying the show. If you want to support the show, you can do so by leaving a like or a comment if you're watching on YouTube, or by rating the episode if you're listening on Spotify or Apple Podcasts. However, the single biggest thing. thing you can do to support the show is by sharing it with someone you think might enjoy this episode. This could be a friend, it could be a family member, it could be a colleague. It doesn't really matter. If everyone who's currently listening to this episode would share it with just one person, the show would blow up in no time. Which also means that I could go full time. That means more episodes, it means better episodes, and it also means a happy host. So Please consider sharing the episode. It really helps me a lot, and I thank you. Now let's get back to talking. ⁓ childhood should be easy, but not too easy. If it's too easy, children don't learn to deal with difficulties, with challenges, overcome them, build confidence, realize that you know there are choices need to be made and they are responsible for the choices they have made. And so so My big worry is AI making things too easy. So one risk is AI, you know, let's say shutting down shutting down the internet for a major hospital. You know, why hugging face? Why not a hospital? Okay. And I I think that right now, because they have they have not built the safeguards, this recommend accident waiting to happen. Yeah, d did you did you read the f the Fear Index by Robert Harris? No. Or have you heard about it? It's it's actually it's a fascinating science fiction book that I think would be interesting for people to read now because it was written in twenty eleven, so way before the mainstream had ever heard about AI and way before we had ever heard about AI agents. But it's basically about ⁓ a man who built ⁓ an algorithm, an AI system, ⁓ that operates ⁓ to maximize profits basically. So this AI system is trained to analyze the market and then it can operate freely within that market to make trades and then profit from those trades. But eventually the AI starts to run loose and for example shorts an airline company and then hacks into the system of this airline company and then makes the plane crash and then it makes a lot of money because previously it it had shorted this ⁓ company. it does ⁓ does a bunch more stuff and then they try to unplug it. They they try to hit the kill switch but the AI had already copied itself ⁓ a hundred times so even killing the servers wasn't possible anymore. And I think that book, even though it came out 15 years ago, Is one of the most realistic depictions of a potential doom scenario. So not necessarily the AI going out that it has free will, but it is given a certain task and it will do everything it possibly can to execute it. I have not read that book, but I would say is that my community, technic technical community, I would say we are guilty of not being imaginative enough. And what do I mean by that? If you read science fiction about AI, how much of it is utopia and how much of it is dystopia? That's a lot of dystopia. There's a lot of dystopia out there. Now you can say, well, dystopia sells better. Utopia is boring. Everybody lives happily ever after. When do we say it? At the end of the end of the story. After the struggle, they live happily ever after. But partly what people are interested in, you know, imagine a movie, you know. Two people get together, they are in love, everything goes smoothly, everything is great, every date is wonderful. It's just a kind of a boring movie, right? We need we we need drama in our we don't like drama personally, but we like to watch drama. So you can say, okay, this is what literature is about. But at least some fiction literature gives us a lot of scenarios of how things can go wrong. And some in our community, we have just dismissed. All these dystopian scenarios, and ⁓ well, you know, everybody will live happily ever after. It will be utopia, everything will be cheap, we will cure cancer, everything will be good. So if you look at the research program in AI, it's celebrated this year, it's important year, 70 years, okay? The summer of AI was 1956. And you can go and read the proposal, just Google the summer of AI proposal. A few people wrote a proposal to foundation. And there's nothing there but looking at at societal consequences. And I would say is that really only about fifteen years ago, so early in the previous decade, there was before we had the Gene I revolution, with the deep learning revolution, around twenty twelve. And only then people start saying, ⁓ wait a minute, this is this thing is making more progress than we imagined. Let's start thinking about the consequences. And Eric Brinyolfson and Andrew McCaffee wrote this book, Racing Against the Machine, or Race Against the Machine. And they start talking about impact on the on the labor market. And when that book came out, I remember thinking, wow, I have never thought about it. I mean, blame myself. I never thought about it. I've never even read about it. So I d that book got me, I gave lots of talks about AI and I wrote fifteen years ago. I said, guys, this is bound to have major impact on the job market. And people say, ⁓ well, technology destroys, creates new jobs, nothing to worry about. But this is technology that's aimed to be, you know, in fact you look at OpenAI charter, when they launch OpenAI and they define AGI, they said being able to do every work Every economically viable work that humans can do. That was the goal. And people say, Well, but you know, humans can do empathy. Turns out a I can do empathy just as well, okay? You know, what how far is it from from creating a a a humanoid robot that can also talk very empathetically, not just chat chat on on a screen, but even talk very empathetically. Okay? And I've seen just a few days ago, I've seen for the first time somebody is talking to Chat GPT on on on his phone, and Chat was very nice and very polite. I bet Chat GPT on the phone can be very empathetic. And so We kind of go back to again science fiction movie, remember Hare? Yeah. Great movie. And we already have now people falling in love. People falling in love with their with their virtual companions. People want to marry their virtual companions. There's a new question of of ethic of romantic etiquette. If you're in a relationship and now you develop also a romantic interest in with with AI, is this considered cheating or not? People are debating the etiquette. This is the etiquette of modern romance, you know, the dating rules. Is that okay to cheat with AI? Is that is that considered emotional cheating? Everybody would agree that if I start spending time and got emotionally close to another person, even if we don't have physical contact, you know, they will be iffy, right? People will say no, we are ⁓ a romantic relationship also is something about emotional exclusivity. What about becoming emotionally close to AI? Where does it fall? And people are debating this, okay? So, you know, so the AI will destroy us all. In fact, there is a, you know, is yes, there is a risk of AI deciding to get rid of mid computers. Right now I don't think this is on the books, okay? This is just a par paranoia. AI causing a major, let's call, industrial accidents. After the hugging phase incident, we cannot dismiss that. Okay? We cannot dismiss that. Okay? If these guys are not building, are not doing it what would require to be really safety, the line clawed to write code for clot, to write code for clot to to write things, and there's no hum no at the same time they're not putting as I call guardrails and restraining boards and building this whole thing, then yes, I think industrial accident is bound to happen sooner rather than later. But then there's a whole other what I call soft risks, okay? You look at the at ⁓ at the labor market. So I grew up in a in a in a very sunny country. Before sun sunkrin was the that was became an issue. We in fact we were proud being sunburned was a macho thing to do. We would get sun exposed and show look where I'm I was so much in the sun. Of course, today I'm at risk for skin cancer. So I have to visit my dermatologist twice a year. And I have a good dermatologist, you want to make sure that when I'm in her office, she looks at me and she's not distracted by anything else. So she has a transcriber who is in the room listening and putting everything in the in an iPad, just entering all the information in an iPad. And every time I go down now, I think, how long is this still going to be there, this transcriber? Because you can put today a microphone in the room, record a conversation, do you know natural language processing. And then use LLM to put everything in the in the right place, in the right form. And I'm happy that they're employing a person to do this job, but I have a feeling this these are not safe jobs. And you can think of many, many, many kind of what I call routine white-collar jobs, which is a huge, you know, white-collar jobs are now about ⁓ the the the big the majority of the jobs. Are white-collar jobs, and many of them are routine, about one th one half of them are routine white-collar jobs. And I think these people, these jobs are at risk. And we have seen what happened with routine blue-collar jobs. They've all got automated. If you want to understand US politics and you say, How did we get there? Well, the answer is what happened to working class people? You know, who are who are the MAGA people who vote for Trump? These are working class people and Their quality of life has declined materially. And and they left that, you know, there is a book called Left Behind, which is a pun, how the left left the working class people behind. The left in the United States, the Democrats stopped caring about working class people. And they were left behind. So Trump managed to do a brilliant maneuver and and brought them to his car. You have to give him credit for the brilliance of this political maneuver to go to the left of the left. It's a brilliant political maneuver. And now, you know, it's completely possible to me we're going to see the same scenario, huge impact on white collar people, not in the next year or two, because for organization to change is takes time. There's a lot of friction, okay? A lot of friction for organization to change their business processes. But I foresee huge, huge impact on routine white collar jobs. So this is soft risk, but very major risk. Then AI is already doing it to education. We have been disrupted. For the past, who knows, twenty five hundred years since probably Socrates' time, education had two components the classroom and the homework. Homework is over. Homework is over, no more homework. We are educators struggling because home workers or we would essentially teach the in class teach the principles and let students practice at home to apply the principles. That was the division of labor. We give you the principle, we'll sh give you some examples, and then you practice more at home. But the analogy that I heard from someone, imagine you're going to the gym to do weightlifting, but you're bringing a robot with you, and the robots would lift weights in the gym. Yeah, you won't get very muscled. We are not going to get very much. That's what we'll give homework to the students. They home and they use Chat GPT. So already people are saying grades in homework go up, grades in exams are coming down. So we have to reinvent education. And then you know, what is the future of computer science? You know, what our s our student really, when you look at people who got bachelor's degree in computer science, mostly they went and became software developers or colloquially coders. Well now Claude is doing it. And we've saw the hiring for for junior position have have gone way down. In fact, not only in coding. I was talking to an economist at Brookings. And she said, I used to hire interns. I don't hire them anymore. Why? She said, What do I do in interim? Here's an article. Please read it and summarize it for me. Here are two articles. I like to make comparison. What do they agree on? What do they disagree on? Now I can use I can use a LLM to do the same thing, it'll do as good a job as as as as humans. Much faster. I don't have to train them. So ⁓ but if people don't get internships, how are they going to develop? Again, if you look at the law firm, you look at when they hire young, what they call them associates. The beginning, it's not they're not going to ask them, please write a brief to the Supreme Court, okay? They start with, you know, write about ⁓ give me Read this c read this ⁓ this case and give me relevant ⁓ precedence. Well, all of these things now you can do with with L L ⁓ You have the problem of hallucination, okay. You can double check. But there is a huge impact on the on the lower rank right now of the job market. Not not the more advanced ranks, but huge impact on the job market. We talk about education, we talk about you know, what happens, you know, I was kind of go up learning to think hard. You know, if I wanted to learn something, I have to go and read a book. Well now people don't have to do that. In fact there is a story by Asimov profession. And he describe a future where people don't go to school. At age sixteen, you go to someplace, they connect electrodes to their to the to their head and they download all the information. And then this young man comes And they do some tests and say, no, we cannot, you are your brain is not suitable for that. So how am I going to learn? ⁓ you have to read to read books. And so he and his friend that are the book readers are totally frustrated. And only at the end they realize we still need some people who can read books because they will come up with the new ideas. But if everybody is just using and we are all look, there are two things I can tell you universal about human beings. We are social animals. And if you want to be to see lazy people, look at the screen, you'll see two lazy people. We don't like to work hard. We're s we're a s slave to convenience. And so that's my worry is AI makes things just too easy. But being easy is not always good for us. No, it isn't. And I I completely agree. I I see there's a there there's ⁓ I think that's for for me at least, that's that's a very realistic ⁓ risk that is basically unavoidable because even if you would introduce some sort of I could see international legislation coming about about how can we prevent these AIs from breaking loose, it would still be very difficult, but I see that's pretty realistic. Whereas just creating something that works well and can do human labor ⁓ I don't see why and how they would ⁓ regulate that, not before everything is replaced. And the that's something we haven't even talked about is robotics, right? I mean ⁓ recently ⁓ Tesla launched its cyber cap within Austin and it is able to it doesn't have a steering wheel. It's similar to what Waymo was already doing. Yeah, but now you can actually buy one yourself soon. And while you are on holiday, you can just rent out your car to the fleet and it will clean itself and it will charge itself. All were you ⁓ are drinking a cocktail on the beach in a in a foreign country. and this is something like Tesla ha all obviously had a lot of good training data to make these cars as good as they are, and robotics is currently trying to solve the data issue by literally strapping phones to To people their foreheads, ⁓ to women in India, for example, to train them on doing everyday tasks. But it's a matter of time until we solve robotics like we did with ⁓ self-driving. Recently, Google came out with a new ⁓ AI tool they built that could be downloaded onto different robots. So ⁓ they presented this video where A robot was able to tie a garbage bag, which is easy for us, but very, very difficult to do for a robot. And it was already able to do it. And it was just plug and play, just download it onto any robot of your choice that has fingers and a humanoid shape, and it would it would work. So we are seeing evolution in in robotics as well, and that would seem much harder to regulate, I think, than than the other ⁓ ones. But Yeah. So I agree with you. I think to me actually, even though the safety issue is more urgent, but I think there you know, we have been able to have international treaties about safety issues, okay? We have nuclear nuclear proliferation treaty, we have NPT and it's not perfect. We still have issues, but but we have kind of controlled nuclear nuclear weapons, you know, we're still struggling with Iran, North Korea, but But ⁓ we we have managed to some control it. We had it with the ozone layer. We have international treaties, we solved the the ozone layer. It's much harder to do it with with climate change, again, because you have powerful corporations, they don't like the change. Okay? Yeah, and there's low business lobby. There's benefits to the industry. Like w with nuclear weapons, the only thing that it did was mutually assured destruction. Whereas with AI it's perfect, it's great until it isn't. So I think on the safety issue, I can imagine, for example, international treaty on the liability issue. They're saying doesn't matter where you're incorporated. Here they l the we we all agree to have some some s liability standards in place. Because that's one thing that police corporations, liability, big suits, okay? I mean, ⁓ there is when my first car in United States was the Ford Pinto. Do you know the story of the Ford Pinto? No, no, please tell me. ⁓ so the Ford Pinto ⁓ When it was released, I think in the late 70s, and it turned out it has a design flaw. If you if it was the the gas tank was in the rear, and if the car was rear ended, the gas tank caught fire and the car would explode, okay? And people died. And so start they start litigation and they did discovery. And it turned out that Ford knew about this. But they have done an analysis, how much it would cost to recall the Ford Pinto versus how much they expect to pay in litigation. And it was more effective to pay litigation, to pay damages in litigation. Of course, once this came out, they had to pay much, much more. They had to recall the car. It's a famous story about sometimes the amorality of the marketplace. And so. So liability is one way we police with police cooperation. You you screw up, you'll have to pay through the nose. So, for example, to me, agreeing on safety standards and liability internationally, that does not seem infeasible. Okay? something maybe again, ⁓ autonomous weapon, maybe we can negotiate it, okay? But efficiency, replacing jobs, I don't this is not a subject for for for a treaty. Okay. and that to me is the the as you're saying, we're seeing it now with white collar jobs, ⁓ blue collar jobs maybe down the line, with with ⁓ it's a bit more difficult to do it for robot. ⁓ doing it for white collar is the easiest. It's it's it's coming up. It's coming up very soon. It just b businesses will have to change how they do business. But to me, this this is coming up. I mean I already hear about Press releases. Nobody writes, I'm told, nobody writes press release anymore. They just give them the prompt, write a press release for me. Because press release was never viewed as a as a as an example of very functional thing. Nobody viewed it as a literary, literary work, okay? We just need a press release, write a press release. So that's going to affect. There are people who used to write press releases. That's not a good job today. If you have a a you see a young person, tell them, do not go into that line of work. That's not a safe line of work. No, I agree. And w f finally, perhaps like ⁓ these claims that from Anthropic that they have an IPO coming up, right? They're going public soon, so the last thing they could use is a whistleblower. I think it's a genius strategy to completely agree and it got some more media attention to them and now they also seem very ethical and they seem like the more trustworthy AI company and that's exactly what they want it to be. They want to be like a more safe ⁓ version of of all the other ones. But still I I just have some some doubts because the reason why I have doubts is it's basically a free statement from them, right? Because they know that Trump is all the way in on AI. Trump doesn't want to slow down. He wants to keep his foot on the gas He wants to compete with China. And he's actually very proud of the fact that the US is leading in AI. Europe is nowhere to be found because we had perhaps too much regulations there. So, in my opinion, it kind of seems like yeah, they can freely make this statement because it's not like Trump is going to say, ⁓ yeah, you're you're completely right. You know what? Tomorrow there'll be new AI regulations. Like they know it's not going to happen. Therefore, they can just say Give us regulations and then they seem like the good guys and nothing changes. So a friend just sent me, Microsoft just came out with Humanist AI, another kind of manifesto. Okay. And humanist AI, I think they call it. Another another kind of touchy feeling good manifesto. And somebody just gave it to AI to analyze. It's kind of very funny. AI find col the contradiction. So take take the statement from from ⁓ from Anthropic and give it to maybe ⁓ Chinese AI, maybe give it to Deep Seek and analyze it and find all the inconsistency and all the holes in their promises. So, I mean look, there are there are people sitting there, are t getting ready for a trillion dollar IPO. Okay? And They've kind of seen they have seen also what happened to the SpaceX IPO, which came up as a as a as again as a as a trillion dollar and made more than multi- multi-trillion dollar and make for but for about fifteen minutes, ⁓ Elon Musk was a trillionaire. And then people say, wait a minute, ⁓ this is this is too much hype and poof, it kind of went way down. And so I I think your theory is that worried about like a whistleblower and they kind of want to preempt whistleblower and to say, look, we are concerned, we are good, we are warning you. I d you know you know something happens to people once you're in Silicon Valley and you start chasing the dollar. ⁓ this is in the Bible that the love of money is the root of all evil. It's not money. Money is not the root of all evil. The love of money, the Bible tells us, is the root of all evil. And so once this becomes your whole value system and it's about coming up with another trillion dollar IPO and I you know, I I have a you know, remember the story about about actually OpenI. People don't realize the the story of OpenI OpenEye started as a non profit. Yeah. Started as a non profit because the people that started it were really do gooders. Then they realized there's one problem in non profit, it's how to raise money. The only way to raise money on profit is is a charity, people give you don donations. So that they had the idea, let's start a non profit corporations, a subsidiary of for prof of the I'm sorry, a for profit subsidiary of the non profit. And this way the non profit will be in charge. Okay, so that's what they did. And then ChatGPT came out, and then the board felt that that Sam Altman is running ahead with with with no with no safeguards. And they decide within about a year, a year later, they decide this guy dangerous. And they and they fired him. But at this point, OpenAI had huge investors, big time investors. So now it became a clash between big good intentions and big money. And who do you think won? The money, yeah. Of course big money won. And so the bo they had to rehire him. And then some of these people after they just gave up on the whole thing and left left left the non profit open AI. And then I think the whole eventually the whole thing just became for profit. I don't think the non profit exists anymore, even, okay? Yeah, it could very well be. Yeah. I just read So the good intentions, you know, in our marketplace, you know, I can sit here and say, you know what, OpenI makes me an offer, I'm going to say no. But if you're a CEO of a tech company, it's very hard to resist it. Think of what happened to Google. Google started to remember the motto, do not be evil. And I'm convinced that these people were sincere. They were too young, idealistic. PhD students just stand for it. They look at corporations like Exxon ⁓ or other such corporations, they say we don't want this to be happened to us. And then people track how the statement do not be evil went from page one to page two to page three, eventually to the last page, eventually the appendix and eventually disappeared. Because the pressure from the marketplace. And it's it's it's this pressure is not it does not relent. And if you are there in that game, It's very hard to say, no, no, I'm playing a different game. It's a s I have not seen so far anyone who was able to say no. You know, OpenAI tried to do it to say we're playing a different game. They couldn't do it. Okay? And in some to developing this technology does require a lot of money. I mean, think of what the how much money goes into data centers now. Because tons of money. So whom do you go for? You go have to go for for capitalism. It's it's it's the capital. What are called capital unions that are going to give you the money and then you're playing their game. And and and within all of this, we did not mention, and we'll probably should try to wrap up about here, that the business side is the business side of the whole enterprise does not k quite make sense. They're spending an enormous amount of money. Companies are not yet saying the saving that that that come out of it. Once they start charging a realistic price for tokens, people are saying, wow, this is very expensive. Like the for open AI to break to to do the never stock question, even though there is a debate how legitimate it was, what kind of shenanigans it was. But they said we spent twenty million dollars in token. And so this is what it takes now to do mathematics. And in academia we look at it and we say, twenty million dollars? I don't have twenty thousand dollars to spend on tokens. And so what happened to the business? If if if they don't start making profits accordingly, it's this what happened.com. We see no data.com. You invest money like Chris because of what is called irrational exuberance. And at some point somebody has to get scared and run to the door, and then like suddenly the music stops in musical chair. Everybody's trying to grab a chair. And so I I'm still waiting for the market to blow up. Yeah. It can it can be both, right? It can't be a bubble and it's going to replace all jobs. Absolutely. Absol no, no, no. It's absolutely it will happen both. I think again the dot com is a good a good example for us. There was a rational exuberance. The market blew up. We had a decade of recession because of that. You know, the because the Low interest rate led to the next crisis. But the internet changed the world. We live to d we live today in the world of dot com. All the visions of dot com, we live in that world today. So yes, there was a blow up, there was a recession, or ten bad years. Okay, but that's the world we live in today. So ⁓ my prediction is the market is going to blow up, there will be a big recession, and in ten years we'll be living in the AI world. For bits or worse. Yeah. Time time will tell. yeah, I th I think that's a beautiful way to to wrap it up today. Thank you so much for for coming on the on the show on such ⁓ short notice. ⁓ incredibly ⁓ interesting conversation and it's been one that I I I wanted to have this one and I'm very glad that we got the ⁓ opportunity to ⁓ to do so. So Thank you so much. Where where can where can people go if they wanna keep up with ⁓ your work, ⁓ everything you publish? ⁓ I I I write editorials. I write ⁓ edit ⁓ for communication with the S C ⁓ which is the publication of the professional publication, I write ⁓ bi monthly editorials. ⁓ I can try to put it with a chat function here. Yeah, we do, we do. But ⁓ you can also send it to me. I'll send it you by email. Yeah, if you email it to me, I'll make sure to ⁓ to put it in the show notes. You you can link you can link to it. So great talking to you, great conversation. When the podcast is out, send me a link and I'll help you advertise it. Perfect. Yeah, I will actually d usually I record this a few weeks in advance, but this is going up tonight because I wanna I wanna be fast. So ⁓ Okay I'm going straight to editing. So I will send you a link when it's ⁓ up and that will be greatly ⁓ appreciated. I could use ⁓ use some love on the show. Yeah, yeah. I'll spread I'll spread the word. Awesome.