Latest / 80,000 Hours Podcast / #244 – Benjamin Todd on how we’re updating our career advice for the strangest time in history
Transcript
- Zershaaneh Qureshi: There are probably some people listening who really do want to do good for the world. They really do care about that, but they just don’t think that they’re going to take one of these AI-focused jobs. Maybe they don’t think that they’re well suited to any kind of AI-related roles. Alternatively, it might just be that they feel like they’re not in a good position to change their careers right now. What do you say to these people? Is there anything specific in the book for them? Benjamin Todd: In a way, it feels like my life’s work. It’s the last 15 years of thinking about this question at 80,000 Hours, and I’ve tried to distil all of our most central and important ideas into this one nicely honed package. Zershaaneh Qureshi: Hey listeners, Zershaaneh here. If you enjoy what we do at 80,000 Hours and you’re interested in joining our team, then you should check out the “Work with us” page on our website, because 80,000 Hours is growing! We’re hiring for lots of roles — I think there are nine listed at the moment, and they’re across several of our teams. Plus, if you like hearing our guest today, Ben Todd, who gives some advice on careers that do good, then you might be especially interested in the product manager role on our web team. In that role, you’d be helping our online advice about high-impact careers resonate with and reach more people. OK, that’s it from me. On with the show! Today I’m speaking with Ben Todd. Ben is actually one of the cofounders of 80,000 Hours, and he’s just published a book also called 80,000 Hours, which is all about finding a career that does good. It covers a bunch of advice, including: • What makes for a dream job • What skills are most likely to be automated soon • Which problems are most important for people to be working on • How to network And many other things. Today I want to talk to Ben about his wider views on AI that have shaped a lot of the book, as well as how people should actually be making career decisions in these quite uncertain times. Ben, thank you so much for coming on the show. Benjamin Todd: Thank you for having me. Zershaaneh Qureshi: So the title of the book, 80,000 Hours, is a reference to the number of hours in an average career. Some people have joked that, “Hey, maybe the book should be called ‘8,000 Hours,’ because the next few years could be so crucial.” Can you walk us through what you think might actually happen in the next few years? Benjamin Todd: The concept of the book, ultimately, is that your career is the most important decision you’ll make, especially for your impact on the world. And if we’re facing this crucial moment with AI, in a way, I actually think it’s even more important than ever to think about how we’re going to use these years in the best way we can. So in terms of what’s happening, I think a useful thing to try and forecast is: when will we have AI that can do AI research and development (R&D) itself? Zershaaneh Qureshi: OK, and just to be clear, the reason it matters how quickly we get AI R&D automated is that when that starts happening, that introduces some feedback loops where AIs are designing better AIs, right? What would the world look like if that happened? Why should we care about that particular moment in the future? Benjamin Todd: There are actually many powerful feedback loops in AI, but the one that could be the most sudden is the algorithmic feedback loop. If you get to the point where an AI agent can do AI research itself, given the amount of computer chips we have in the world now, each company could run the equivalent of maybe 10 million researchers, which would mean that effectively the AI research workforce would expand by like 1,000 times. And although they would still have a limited amount of computer chips, with that much more research capacity thrown at the problem, they should be able to speed up AI research a lot. And so Forethought, for example, has estimated that it’s quite plausible you could see something like five years of AI progress in one year. Right now, AI is actually still quite narrow in what it can do. It’s very good at multi-hour software engineering tasks, but it’s still a long way from being able to do most jobs, and it still can’t even play Pokemon better than most children. But if we suddenly had another five years like the last five years of progress in one year, you could get to a point where suddenly something much more general, much more like a true digital worker, you could say, started to work. And this would be an AI that you could literally ask to do almost any job that can be done virtually. Zershaaneh Qureshi: Yeah, and the idea then is that once that happens, society changes very rapidly, right? You’re deploying AIs in all sorts of different fields, progress in all sorts of fields goes crazy, things are moving faster than we can keep track of, and that throws up a bunch of risks? Benjamin Todd: Yeah, there’ll be these 100 million copies of Claude — which is almost like the whole working population of the US — it will be almost like a whole country’s worth of new people doing whatever those Claude things are doing at that point, and they’ll be able to think very fast and act very autonomously. So that’s where you also start to get these loss of control risks coming in. And more dangerous scenarios, like single companies might end up with a workforce that’s bigger than the entire human workforce now, which could give those companies huge amounts of power — and starts all kinds of novel risks and things to think about. Zershaaneh Qureshi: Yeah. OK, so you now think that AI R&D could be automated pretty soon, basically? That’s your feeling? Benjamin Todd: I don’t know, but when I was in the Bay Area in February talking to people about this, I was pretty shocked to find many of the people who seem to have the best track records of forecasting AI to me were saying it could be a 10% chance we reach this point this year. I wasn’t really thinking it literally could be this year that an intelligence explosion would start. And then the chance a couple of years later is even greater still. So Jack Clark recently published a piece arguing that there’s a 60% chance that AI R&D is automated by the end of 2028, which I think reflects the views of many people actually in the labs themselves doing this research. Zershaaneh Qureshi: He’s at Anthropic, right? Benjamin Todd: Yeah, he’s at Anthropic. But it’s not just people at Anthropic. I’ve heard this from people at DeepMind as well and OpenAI, that they just really feel like these tools are really helping them and doing more and more independently and autonomously. And they can just kind of see this process, they just draw the trendlines forward a few years and it seems like it could be really speeding up our research. Zershaaneh Qureshi: I kind of want to dig into that a little bit because I wonder how fair it is to be doing this kind of extrapolation. Benjamin Todd: Even if we just extrapolate revenue growth forward three years, it kind of seems to imply we get something like artificial general intelligence (AGI) in three or four years’ time. Anthropic’s been growing its revenue 10 times per year for three years, which is already completely insane. But in the first quarter of this year, it actually grew at an annualised rate of 80 times — even on a base of already $10 billion or so revenue — which is probably the fastest a company of that size has ever grown, just due to the huge demand for these more agentic tools. There are various memes about how dumb it is to just do trend extrapolation, like: “My baby has grown this much in the first year, so it’s going to be 10 feet tall one more year from now.” Which obviously isn’t how things work. But then I think just saying, “It is an S-curve and it will plateau at some point” is true, but it’s not very helpful because we don’t know where it will plateau. And it might plateau at vast superintelligence that’s spreading out over the galaxy, which is not very helpful. Zershaaneh Qureshi: Yeah, so my understanding is that a lot of this does rely on being able to increase the amount of compute that’s going into these AI systems. And it seems like there are some reasons to believe that it’s going to get much harder to throw more and more compute at these systems in the coming years. How does that affect your story here? Benjamin Todd: Yeah, you’re absolutely right that as we produce more and more computing power, it becomes harder and harder, which wouldn’t mean that progress would halt, but it would mean that it could slow down a bunch from here. Zershaaneh Qureshi: It seems like there are a lot of moving parts here. There are a lot of things that are uncertain. There’s a lot of debate over the trends that you’re describing: when things are going to begin slowing down, how capable our AI systems will be at the point when things start slowing down. And I think it strikes me, just moving this back to the book itself, that what people can do with their careers to make things go well really depends on whether they have three years or 30 years to act before things just start going crazy. So how do you write a book of career advice at a time like this? And how can you make the advice actually useful? Why do people think that it’s useful? Benjamin Todd: I find it useful to think in terms of three key scenarios for what might happen. The first being that basically the people at the frontier companies are right, and they are going to get an automated AI researcher within a couple of years. And then that could start an algorithmic feedback loop and you’re getting more general-purpose AGI perhaps in 2030, or even 2028, 2029. And then the next scenario is that this is basically happening, but those people are being a bit over optimistic and they’re not quite accounting for all of the bottlenecks that will happen. And then maybe compute growth will slow down a little bit and that could really push this automated AI researcher into the early 2030s. So that’s a slightly slower — but still very crazy — timeline. And then the third scenario is that the current paradigm runs out fairly soon and compute scaling becomes too expensive, and then you could have quite a long plateau — potentially several decades. That would be the slowest scenario. Which of those we’re in does have a big effect. The main ways it has an effect is: the longer you have, the better it is to build career capital and to explore lots of options. Some people say these short scenarios are the craziest, most dangerous scenarios; they’re when AGI happens when the world is the least woken up, so we’re the least prepared. And so those are much more impactful scenarios to work on, so you should just basically act as if we’re in the short scenario and just do whatever would be best in that. And if we’re not, you can figure out some other stuff to do in the medium scenario, you probably haven’t burned too many bridges, depending on exactly what you’re doing. Zershaaneh Qureshi: Don’t go too crazy! Benjamin Todd: And then there’s a bit of a counterposition — especially imagine you’re a college student right now and you’re like, “I’ve just started college, if there’s going to be AGI in 2029 I might not have even graduated yet, so it’s hard for me to do anything.” So for most people they actually have more ability to help in these slightly more medium-term scenarios. And that could be a big effect because of the difference between a fresh college graduate and someone who’s spent 10 years working their way up in the government and now is in a really influential position — they have much more ability to help with things, maybe like 10 times, 100 times more. So that could mean it’s better to gamble on the medium-term scenario because you’ll be in such a better position to help that you’ll actually have more impact by doing that, even though there’s a chance that you’re too late and you don’t actually help in time. And if you take the average of these two views, it implies that you should basically not act like the short scenario is happening, but instead one a little bit longer than that, because your ability to help is curving up but the leverage and neglectedness of AI is going down. And it’s where those two cross that’s the optimal year for your impact. Zershaaneh Qureshi: Yeah, it’s simple, you just got to do the maths. Benjamin Todd: But on a more practical level, I think most people shouldn’t just think it’s three years or nothing and act as if nothing beyond that matters. But I do think there is an argument for having quite a bit of urgency, especially compared to 10 years ago when we thought timelines were much longer. Zershaaneh Qureshi: How should people work out what worlds they should be trying to optimise their efforts for? How do you go about that thinking process? Benjamin Todd: Well, the biggest one is just: how likely do you think each of these three scenarios are? And then it would be, do you think these short scenarios are much more neglected, so therefore much higher leverage? And then, how much could you increase your ability to help by, for example, focusing more on career capital for a while? So in general, for younger people, there’s a good prior that they should focus on these slightly more medium-term scenarios. But if you’re already established and it’s harder to increase your career capital as much, then you can focus a lot on just the short scenarios. Zershaaneh Qureshi: So my understanding is that the book itself takes an ‘all things considered’ view, where you do the averaging out across the possibilities and then try to provide people with the most useful advice, with all those things taken into consideration. And what you come to is that, at this moment in history, the top options you recommend people to work on are various challenges in AI. In particular, I think you highlight the chance of loss of control of AI systems and protecting ourselves against losing control of AI systems as they become more autonomous. And the other one is protecting against AI extremely concentrating power into the hands of just a few people. But you do mention other problems that the world is facing that are also useful to be working on at this time, all things considered. So kind of stepping back, but there are probably some people listening who really do want to do good for the world, they really do care about that, but they just don’t think that they’re going to take one of these AI-focused jobs. And maybe they think that because they don’t think they’re well suited to any AI-related roles, or maybe they’re not that inherently interested by this AI stuff. And if you’re that person, I’m sorry that we’ve been talking so much about AI. I think alternatively it might just be that they feel like they’re not in a good position to change their careers right now — and if these things are going to happen very soon, it’s kind of pointless. What do you say to these people? Is there anything specific in the book for them? Benjamin Todd: To step back one step further: the aim of the book is to give you these general-purpose frameworks that anyone can use to figure out which career is best for them, no matter their view on causes. I also put my own best guesses in because I think it’s interesting to show the application of the framework, and people find it helpful. But that’s actually a relatively small part of the material, so I think it should be very useful for people who aren’t interested in AI. On the questions about how you can help, even if you don’t feel that interested in AI right now, I think there are a few things to say about that. One is, I think historically it’s easy to feel like AI means ‘the alignment challenge’ and that we just need technical researchers solving that challenge and there’s not that much else that other people can do. But I really want people to think: it’s not just about AI misalignment — what we’re facing is something more like the complete transformation of every aspect of society. It’s like the Industrial Revolution, but happening 10 times faster and going to some place that we don’t yet know. I do still think misalignment risks and AI loss of control risks are probably the risks I would say are the most important and neglected, but there’s now a lot of other things. So something like concentration of power, that’s a much more social science-y, geopolitical-elements-type cause which needs a lot of nontechnical people working on it. There are still pandemic risks, and engineered pandemic risks, and that actually mainly requires company-building and engineering-type skills. But there’s a lot we could do to make the world a lot safer from pandemics, which would just be good in general but also the risks could be accelerated if technological progress speeds up. And in the book I also briefly talk about this range of emerging challenges which are also super diverse, from the philosophical questions of what do we do about AI sentience and legal questions around that, to even things like space governance and gradual disempowerment, which is again more of a social structure type of challenge, rather than a technical challenge. I would also say even if you want to focus on global health, you should probably be thinking a lot about how you could use AI to help with that, or how AI might affect various aspects of it. So yeah, there’s a much broader range of issues, and then there’s also a very broad range of roles these days. More technical researchers would still be great, but we’ve done surveys asking organisations that we think are impactful what their biggest talent bottlenecks are — and a lot of them are saying operations stuff; so people to do management, HR, accounting, finding offices, hiring people, and all the things you need to do to run an organisation. So anyone who’s worked in business, or pretty much any other field, might have those skills. Another huge bottleneck is communications: spreading the word about these problems, doing PR and media for these different companies, or even being an individual writer talking about these things. And then there’s also still a huge need for people in policy and government. It’s very possible to get into these types of roles in just a couple of months, even if you don’t have an AI background already. One of the messages of the book is, people think, “I’m interested in these things and so I need to find a career involving them, otherwise I won’t be satisfied.” But we’ve seen so many examples of people just trying anyway and then eventually finding it really interesting. This is a very old school example, but in the book I tell Jess Whittlestone’s story: she was a philosophy student at Oxford and was really interested in philosophy of mind. And she was like, “The thing I feel most interested in is just continuing as a philosopher, but I don’t really see how I could have a big impact like that.” So she explored lots of other paths, like policy and nonprofits, and she went into evidence-based policy at the Behavioural Insights Team, and then from that she pivoted into AI ethics and AI policy. And then she eventually became head of the AI policy team at the Centre for Long-Term Resilience (CLTR), which is like the most important AI-risk think tank in the UK. And TIME named her one of the 100 most-influential people in AI. Zershaaneh Qureshi: Whoa! Yeah, what a trajectory. Benjamin Todd: But she finds it super interesting now because she’s learned the skills in this area and she sees that it’s meaningful and why it’s important and has interesting colleagues, and those are the things that make a career interesting, even if you might not start out already passionate about that topic. Zershaaneh Qureshi: Yeah, totally. There’s a really wide range of jobs that are helpful for making AI go well that don’t necessarily require technical skills. Basically any kind of dispositions or sets of skills that you currently have, there’s plausibly some really useful way you could apply that towards something AI related. And it also seems like there are lots of jobs where your day to day wouldn’t even necessarily need to be focusing that strongly on the specific details that may not interest you about AI, but you could still be furthering efforts kind of indirectly in that way. I’m also hearing that if you give it a shot, you might actually love it, which is true, I think, of lots of careers. I want to talk to you now about your personal trajectory here, because you’re an example of somebody who wasn’t previously that focused on AI. I know around 15 years ago, AI risks were already very much on your radar, but it wasn’t the thing that you were thinking about every day. And I’m interested in what changed for you. Were there specific moments or any research or analysis that you did that made things feel really visceral to you that changed your mind in some way? Benjamin Todd: I think it was either 2015 or 2016 where we had the combination of OpenAI being founded, Superintelligence was recently released, and then AlphaGo had just beaten Lee Sedol — and that was like, deep learning is working. And so at that point, if you extrapolate the trends, we had like a mini call to action: “Now is a good time to work on AI.” I almost wish I’d followed that advice more myself, but people who did follow that advice at the time, now they’re often really high up in the key nonprofits or safety research teams or governments working on these issues today. At that point, I still thought building effective altruism would be more effective for me: it’s a better match for my skills, and we were still very uncertain about AI. And a big advantage of building a community interested in generally doing good is the possibility that they can switch cause, depending on which one turns out to be most pressing in the future. As well, you’re getting a multiplier, you’re getting more people involved. But then, as AI timelines have shortened, I think the case for just working directly on AI gets stronger and stronger, rather than effective altruism, which is a one step more indirect approach. So yeah, that’s what I’ve been doing in my own career the last few years. Zershaaneh Qureshi: Were there any key moments or things you learned that really convinced you that AI risks were the big thing for you to be working on? Benjamin Todd: It’s just really been like, thinking about the mechanics of AI scaling and coming to appreciate that every step in the process works. The AI companies — Anthropic and OpenAI mostly — have increased their revenue about five times, and then that means that they can basically buy five times as many computer chips, or they can spend five times as much on computer chips. Historically, the efficiency of those chips has roughly doubled every two years. So roughly, they’re then able to get the next generation of chips, and that means they get 10 times more computing power compared to before. And then recently, having 10 times more compute has basically meant they’ve earned 10 times as much revenue. But then if you now have 10 times more revenue, you can invest that back and now you can get another 10 times more computer chips. So it’s actually not only working, but it’s potentially accelerating. And then I haven’t even talked about AI R&D. There are actually additional feedback loops here because if the AI model is also becoming better at doing AI research, then that is also feeding back and accelerating the rate of improvement of AI on top of that. And then also in addition, if you’ve made five times more revenue, you now have five times more money for salaries, so you get a bigger research workforce, which would also accelerate the progress more. And we now have empirical estimates of all these different factors: we can see that roughly the AI research workforce has been increasing 30% or 40% per year, but algorithmic efficiency has been [tripling] each year. So that suggests that each time the amount of research going into AI is doubled, the algorithms improve much more than a doubling, which is the thing that you need to start an algorithmic feedback loop. In the past it was a much more theoretical thing where it’s like, “If it could do AI research, maybe it could start a feedback loop,” but now you can actually have a model and estimate the parameters empirically and show that it probably does. Zershaaneh Qureshi: So for people who are listening to this and maybe having a similar wake-up experience where they’re getting increasingly worried about AI risks and the chance that these things will play out quite soon, what should they do? What’s your recommendation for what they should do now? Benjamin Todd: It really is possible to transition quite fast to working on these things. It’s not an easy thing to do, but we have seen lots of examples of people, in a matter of months, making quite dramatic career changes. And there’s a kind of standard playbook that these people use and that can work. There are a few steps. One is doing a crash course to understand the field: • What the big problems are • What the main views on timelines are • What some of the main interventions are • Who the main players are You want to be able to have a broad understanding of these things. And we have a reading list on the 80,000 Hours website; I think it’s called the 11 essential readings on AI. That’s a good starting point. And then the most-recommended thing is often one of the BlueDot courses, and that aims to give you that grounding. Then you can think about which kind of broad way of contributing might work for you. Are you more of an operations, organisation-building-type person, more of a comms policy person, technical researcher, or specialist? We need lots of lawyers; we even need historians, economists, and engineers doing biosecurity stuff. Identify a broad path and organisations that you might work at. And then just try and speak to as many people in the field as you can — especially at those organisations you’re interested in — and say, “With someone with my skills, how might I help?” Try and get more ideas like that. It’s also really useful to ask, “If in three months I wanted to be in the best possible position to get this job, what should I do next?” and get those customised answers. And then it’s just a question of applying to as many things as you can — either to jobs straight away, or there’s more and more fellowships which aim to take you in for three months, a year, or sometimes two years; accelerating you in this field as much as possible. So it’s really worth applying to all of those and seeing what you get. And then if you have any spare time, do a portfolio project of some kind — some type of real work that you can show people. What that is depends on the field: is it building something with AI, is it writing blog posts? It depends on what roles you’re targeting. But that really helps you stand out to employers and also learn a lot more about the field quickly. And then when you’ve done this, it’s a case of reassessing. If you get a bunch of offers, that’s amazing; you can try and choose the best one for you and do that. And then that’s the quickest way to get even more up to speed, and in one or two more years, hopefully you’ll have even better opportunities and you can reassess at that point. If you don’t get any offers — I don’t want to downplay it: many of these jobs are very competitive, there are a lot of applicants per place — then instead you can think, “What can I do in the next one or two years to learn a useful skill for helping with these issues?” And from there you could work in any field that will let you learn these skills: you can work in a startup, in government, in a different cause. There’s lots of places that are great for learning these skills. And then the third approach would be, “I’m going to contribute from within my existing role without changing jobs.” There’s also a lot of stuff that can be done there. I notice a lot of people think AI is the most pressing cause, but when it comes to donating, they donate to Against Malaria Foundation. Zershaaneh Qureshi: Interesting. Benjamin Todd: I think even our own host, Rob Wiblin, donated to the Shrimp Welfare Project. Zershaaneh Qureshi: How dare he! Benjamin Todd: Which is a very cool organisation, but I do think the returns to talent and funding can be different in different fields. But it needs to be to quite an extreme degree to think that it’s optimal to donate to one cause but work on a different one. If you think any of these AI-related issues are the most pressing issues, you should also donate to them. People get this idea that money can’t really do anything within AI, but that doesn’t seem true to me. As one example — and I don’t think this is the best donation opportunity by a long route — but an example would be METR [Model Evaluation & Threat Research]: the organisation that has done the most useful work in the world about tracking how close we are to AI-R&D automation, which is maybe one of the most important questions in the world. Their work is actually quite simple, there are so many ways they could make their time horizon project better and get us better measurements about these crucial things. And they were just saying, “We have like 30 amazing projects, but we only have the staff capacity to do one or two next quarter.” And this requires technical staff who could earn huge salaries working directly on capabilities, and they also need lots of compute to do these experiments. So additional money means they can hire better staff, have more compute, and do more of these really useful benchmark results. And that seems like it’s a perfectly impactful thing to fund. Especially in these few years, there’s a lot that can be done with money. Zershaaneh Qureshi: Yeah, yeah. I guess it’s much quicker to make a donation than to get a job. Benjamin Todd: Yeah. If you can change career, that will have an even bigger impact, but in a way it’s kind of amazing that it’s possible for anyone to help with this very technical and difficult question of AI. It’s the old argument that, via the invention of money, you can transform your labour into someone else’s labour. Zershaaneh Qureshi: It’s an incredible thing! Have people heard about this? Benjamin Todd: Yeah, so that’s the money side. Though I think maybe even more neglected is just thinking about spreading these ideas in society. It’s a very weird situation in a way because it does feel like everyone is talking about AI all the time, but I feel like very few people have actually internalised what’s happening. It’s kind of like there’s a lot of ‘frothy hype,’ but not that much actual, “This is what’s happening, I need to do something about this.” So I think there’s a lot that can be done by just helping people understand these ideas, like correcting common misconceptions on X/Twitter, even, I think is actually quite useful and something that anyone can help with. And then the [third] one would be politics, because I think we do eventually want at some point for there to be a strategic pause to AI. Like if we were facing an algorithmic feedback loop, I don’t think we’d just want to let that run as fast as it could possibly run — which is the default that would happen now. If we could pause, even for a year or two, that would be really helpful. And that’s going to be difficult unless there’s a lot of political will for such a thing to happen, because there’s a good chance the companies won’t just do it themselves. So we do also need to start building political support for this being a big problem, and again that’s something that anyone can help with. Zershaaneh Qureshi: Yeah, OK. So it seems like there are lots of ways that you can help, even very soon. One thing is that you could actually get directly working on AI risks fairly quickly by taking some of the courses and things that you mentioned. But there are also indirect ways that you can help very soon, including donating, trying to help build political will, and doing some communications work. So a really wide range of options, some of which can be done very very quickly, and some of which might take you a few months or something like that. Does that sound right? Benjamin Todd: Yes, absolutely. Zershaaneh Qureshi: We’ve talked a fair amount about what sort of jobs and areas people should be working on to have an impact on the world. I think a big reservation some of our listeners might have about taking on impactful jobs that will help make things go well, is that it might be concerning what the future of employment actually looks like. I know that you researched this when you were writing the book, so I’m curious to know what you make of predictions that AI is going to cause mass unemployment in the near term? So, for example, Dario Amodei, the CEO of Anthropic, has said that there’s a chance there could be 10–20% unemployment in the next five years. And then Goldman Sachs has predicted that 300 million jobs could be at risk globally. If these kinds of predictions are right, then it seems like maybe a lot of the jobs you mention in the book might stop existing pretty soon. Do you think that their predictions are right? Benjamin Todd: I do think something like 10% unemployment could be quite likely, partly just because the pace of change will be so fast. The length of tasks that AIs can do is increasing by maybe as much as eight times per year. But even on many other benchmarks, you have these rates of progress that are in the 2/4/8x per year range. That’s actually very counterintuitive because it means, suppose in one year, AI can do 0.5% of a job, so it’s a few minutes per week of help — so it’s hardly noticeable. But if that increases by eightfold in one year, then in year two that’s now 4%, which is kind of a noticeable help, but again not that big a deal. But then in year [three], it’s 32%, which is now becoming a big thing that people are using all the time. And then suddenly in year four, it can do pretty much everything. So it could be this very rapidly unfolding thing, where it seems quite slow — it’s hardly noticeable for a while — and then suddenly [it can do almost everything]. And this is what happens with a lot of AI benchmarks, where you go from like 1% to 2% to 4%, but then once you get further on it suddenly saturates. The speed of progress could cause unemployment. And I think it also means it’s difficult to know from existing data because there’s this pattern where AI might be having a small effect and then it suddenly gets much bigger. So the trends we see now in the employment figures, things might suddenly go the other way. But yeah, with those caveats, I do think people — especially technologists — are often a bit too quick to assume there will be lots of unemployment due to AI. The key way of seeing it is that partial automation of a job often increases wages and employment for that job. An empirical example is people have been saying radiologists are going to be made unemployed for ages because AI can do image recognition really well. But actually radiologist employment is up and I think average wages in the US are something like $500,000 a year, so it’s even still a super highly paid job. I think there’s actually a pretty straightforward explanation of this, which is that only about a third of their time is spent analysing images. That means all the other stuff they’re doing is stuff that AI can’t really do yet: coordinating with other employees at the hospital, talking to the patients about how to understand the results, and “the machine is broken,” and “we need to figure out whether to trust these scans,” and stuff like that. Even if the routine bit that AI can do becomes much more efficient, it’s only actually increasing their productivity by like 20% or 30%. And then if you increase the productivity of a job by 20%, that might even mean you hire more of them. It depends on the job. But say, like with a sales team, if each of your salespeople can sell 50% more — because they can use AI to find lots of leads and help draft emails and now they’re going a lot faster — why wouldn’t you just get 50% more revenue? Or even more, you might even be like, “Previously I had to pay this marketer $50,000, but they would only bring in $30,000 of revenue, so there was no point in doing it.” But if now they bring in $60,000 of revenue, then you might even hire more people. Zershaaneh Qureshi: Yeah, so here’s what I’m not quite understanding properly with this: if I’m an employer and my company has just become a lot more productive, I could either choose to spend the same amount of money on wages and hiring as I already was, and achieve a lot more with it. Or I could cut back, spend a lot less, do a bunch of layoffs or something like that, and still be achieving the same amount as I was before. It seems like in both situations I’m better off. So why wouldn’t I just do the second thing and do a bunch of layoffs instead? Benjamin Todd: Well, so companies, in general, would be trying to maximise total profits. So if you can get a lot more done, then you will make more revenue, which will mean you’ll make more profits. So that would generally be what people would try to do first. Though exactly how it shakes out depends on different industries and how much more of the thing people would want. With something like accounting, most people only need to do their accounts once a year. So if they can use AI to do that for a tenth of the cost, roughly speaking, they will just spend a lot less on accounting and get the thing done. But most things aren’t like that. Like if a luxury holiday became only 10% of the cost, I think many people would just take a lot more holidays. And so if you’re a company making software, and you can just produce more software and sell more of it and get more profits, that will often happen instead. Zershaaneh Qureshi: So it does seem like there are some reasons why — when you’ve got partial automation making people more productive in the near term — wages could be increasing or there could be more hiring, at least in some areas. But as AIs are able to automate more and more stuff, presumably at some point the balance starts shifting here, right? Like in previous waves of automation, at some point there has been some decline in wages once there’s a certain threshold met of automation, right? Benjamin Todd: That does seem to be a common pattern, where partial automation causes rising or steady employment and wages — and then when you get to really thoroughgoing automation, eventually employment does decrease. Zershaaneh Qureshi: Yeah, just to make this a bit more concrete, I think the classic example of this is ATMs, right? How did it look when it was ATMs? Benjamin Todd: When ATMs rolled out, there were lots of news stories about how bank tellers (the people who count money in bank branches) would be made unemployed. And actually it was true that the number of bank tellers needed to run a bank branch decreased by about half, which you might think would halve employment. But what that actually meant was that now a much smaller number of employees could run a bank branch, which meant it was much cheaper to run bank branches, which then meant they actually opened way more branches — and overall employment actually rose for another 20 years. And now the bank tellers wouldn’t spend their time counting money, they’d spend their time talking to customers about their mortgage or troubleshooting stuff. But then after that 20-year period, we had smartphones and people started to just do online banking and so they would totally bypass bank branches. Since that point, employment has fallen a lot. Zershaaneh Qureshi: Yeah, so your view is that — correct me if I’m wrong — at some point we’ll hit this point with AI as well, where we meet whatever threshold it is beyond which we begin to see employment going down or wages going down? Benjamin Todd: So for each job, it would hit it at a different point, depending on how easy it is for AI to do that job. People can also move into new jobs. What should generally be happening is, if AI is automating one job, that means as a whole the world is producing more stuff with fewer inputs. So that means we’re getting richer. And that means, while maybe the wages for the jobs that are being automated are going down, that extra wealth is then spent somewhere else. In particular, it’s spent on the things that AI can’t do yet, because those will become the remaining bottlenecks. And so wages for those other types of tasks should be going up. So that means people could switch from the automated things, and all the other jobs should be seeing rising wages. And so, as a software engineer, could you switch into some other area? Then we still won’t see mass unemployment unless people, for some reason, aren’t able to make that switch. Zershaaneh Qureshi: On a job-by-job basis, we’re seeing this trend of wages potentially increasing or hiring potentially increasing, and then at some point that sort of petering off and then there being a decline. And we’re also seeing this sort of complicated movement between jobs as well, while that’s happening, as people jump ship and move to the thing where the curve is happening slightly later. Zooming out, what does this pattern look like for society as a whole when you zoom away from the individual jobs? Benjamin Todd: Yeah, so there is this really interesting idea that maybe what happened with ATMs happens for humanity as a whole. Where initially AI is making us more and more productive and there’s this kind of massive employment boom and economic boom. Epoch AI has a macroeconomic model of AI automation, and in this model it imagines that an AI is created that can do 10% of tasks in 2026 — which I don’t think we have yet, but we’re maybe not that far off. And then they imagine that it turns into an AI that could do 100% of economically important tasks. I think in 2030, maybe 2032, is when that hits. So this is just a hypothetical scenario, but it finds that what happens is human wages increase tenfold and actually keep increasing even after full 100% AGI is created because it takes time to roll out. But then several years later than that they start to crash. This happens in many economic models where you can have full automation: you get a rising then falling pattern of human wages. But there’s another possibility, which is that maybe there’s 1% of tasks where we either don’t allow AIs to do them, or they can never quite do them, or it’s just inherently important that a human does them. And if you put that into this model, that only 99% automation is ever reached, rather than 100%, then in that model, human wages just keep increasing indefinitely and everyone is left working on that remaining 1% of tasks. So there’s a potentially huge difference between 100% automation and 99% automation. Zershaaneh Qureshi: Yeah, it’s baffling to me that 1 percentage point of automation can make all the difference between everyone getting richer indefinitely, and then the difference between that and mass unemployment, wages dropping to zero. I know you’ve just explained it, but do you have any way to make it more intuitive to listeners why that could happen? Benjamin Todd: I don’t know whether this will help, but it has kind of happened in history before. In the past, almost everyone worked in agriculture and now that’s only a couple of percent of people. So, in a sense, 99% of jobs have already been automated. But that remaining 1% of things just expanded to become the whole economy. And in places like South Korea, that transition happened in only one generation. So it can even happen very quickly. But yeah, in terms of what these things will be, Alex Imas had this interesting article recently: he called them relational jobs. So they’re jobs where a human doing them is part of the reason why we want them. For example, maybe a lot of luxury services or luxury crafts are like this. And he observes that typically, as we get wealthier, we actually buy more of these types of things rather than less. And you need to also picture: this is an economy where maybe there’s 100 or 1,000 AIs per person. So the idea that it could just be this tiny little sliver of the still-human things among this vast economy, I think that makes it seem a bit less counterintuitive to me. Another point is some of these things might essentially be oversight roles, where you’re giving the AI preferences and instructions because each person is kind of like a mini CEO, where they’ll have thousands of AIs and robots per person. And so that could almost be like a full-time job, just maybe instructing them or checking what they’re doing and seeing that it’s still aligned with what we want. And things like art and maybe nannies are examples people use where you’d really want a human to do it. To be clear, I actually think a lot of people won’t bother with these jobs because, in these scenarios, they would have so much basic income or their investments would be worth so much that they would just live off those. But it would mean, in principle, you could earn high wages if you wanted. Zershaaneh Qureshi: I think I’m struggling to see the vision. Benjamin Todd: It’s very controversial. I really don’t know which is more likely. There’s also a good chance there’s a finite amount of energy and, if you give it to the AI economy, it can just produce way more than if you have humans in the loop. And so basically all the important things become run by AI and robots, and human wages can drop, relatively speaking, an unlimited amount. There’s no real fundamental thing that would need to hold them up. Zershaaneh Qureshi: Yeah. I just feel like, even if we’re not in the 100% automation scenario — and we do stick around 99% automation or something like that — I think I see why wages overall could be going up because production is increasing; everyone’s more productive and stuff like that. In your vision, it sounds really lovely, everyone’s just enjoying their luxury goods. But I’m imagining a world where maybe there’s really vast inequality. I think you’ve described potentially loads and loads of people having their own little army of AI workers and being their own CEO. But the thing that I’m imagining is that, at least early on, you have a small class of managers — people who are already in managerial positions, already at a point in their career, or have a certain set of skills who can take over a little army of AI workers. They’re the ones who are capturing all the gains here, all the benefits, whereas everybody else is kind of screwed. Why wouldn’t that happen? Benjamin Todd: We need to be careful about different parts of the timeline here. But I think, in the short term, it does seem likely that many organisations become more top-heavy, where they have fewer people doing more routine white-collar work and maybe fewer junior workers — but there’s a maybe slightly larger class of managers overseeing many many AI agents. And so the managerial classes get higher wages, and also grow because there could be more organisations; so probably that’s an inequality-increasing force. And this has also happened a lot in automation before, so in finance — in investing — in the past some firms had hundreds of those guys with the jackets, shouting on the trading floor. But now mostly they have been replaced by like two guys with like 10 screens. Zershaaneh Qureshi: They have to shout even louder to make up for it. Benjamin Todd: Yeah, so that’s often one way that automation goes: a smaller number of higher-paid workers doing more. But then sometimes, if it makes a job easier, it can actually increase employment. One example is with ride sharing. At least in London, it used to be really hard to become a taxi driver because you had to memorise the entire roadmap of London and there were like a limited number of spots. Now it’s a lot easier to just log into Uber, get your own car, and you can become a taxi driver. So that’s decreased wages by about 20%, but three times more people work as drivers than in the past. And maybe you could have a thing where, for example, a job that would have needed a doctor before could now be done by a nurse with AI aid. So you could actually see nurse employment growing because now those workers can do things that would have required 10 years’ experience before, but the experience is contained within the AI instead. The third thing is the jobs that AI will be struggling to do the longest are things like service sector jobs, blue-collar jobs, and physical jobs. So those will see potentially rising wages, and that’s actually an inequality-decreasing effect because around 50% or 60% of people work in those sectors, and they tend to be below or around the median wage — and that could be going up. That’s the transitory effect. If we look out longer, I do think AI most likely would increase inequality. One reason for that is essentially it means labour is becoming less important relative to capital, because capital is what lets you buy all the robots and chips — and then whoever has the most robots and chips produces the most stuff, and so gets the most money. And wealth inequality is really extreme, much more extreme than income inequality. So if wealth is becoming relatively more important, then it will be increasing inequality. Unless of course, there’s taxation or something to counterbalance that. Zershaaneh Qureshi: Or some other mechanism, yeah. So I think I’m coming round to the idea that not necessarily everybody is unemployed, and also not necessarily, in the immediate term, there’s this like massive inequality. But I think something else that’s come up for me here is, when you’re talking about the jobs that do remain for people, I wonder if there is some tension now between this and your other advice about what kind of work people should be doing to have the most positive impact on the world. It seems like, from an impact perspective, you have recommended a variety of things, but some large categories of them are researchers and engineers who are trying to make AI safer for humanity. But then from the perspective of job security and making sure you still have a wage in the future, aren’t those sorts of research and engineering jobs and things like that exactly the kinds of things that get automated first? Instead — if they value themselves kind of selfishly — shouldn’t people be becoming the sort of blue-collar workers and nurses that you’ve described, and these kinds of things? Benjamin Todd: Yeah, so I think it comes down a lot to the time horizon that you’re talking about. And one of the big messages is that it’s hard to predict the effects because, if we have this pattern of increasing then decreasing importance, then you don’t want to leave too early because you miss the increasing phase where you’re getting even more. Even if a bunch of AI research is automated — let’s say we’re actually at a relatively low automation of AI R&D right now. Even if we get to a point where a lot more has been automated, that remaining bit is really increasing in value because that’s the key bottleneck for AI progress. We’re kind of seeing this with engineering roles now. You get your agents to do loads of stuff and then you review their work, but people are actually saying they feel busier than before because essentially every individual engineer has turned into a manager of 10 AI agents. People are saying what’s bottlenecking them is code review, because so much code is being produced and they can’t review and vet it quickly enough. But if you have that skill, then that skill has become even more valuable than before. All these things that are involved deeply in the AI economy probably increase in value a lot until you’re basically at full AGI. I think if you’re trying to maximise your impact or even just your wages in the immediate term, you probably want to go on those rising, really crucially important things for a while — and then later you could switch to one of these relational jobs. But there is a bit of difference in this between the social impact perspective and the personal perspective. Because from a social impact perspective, you mostly care about this transition and helping that go well. And then if we make it through AGI, then we can chill, hopefully, because we’ve done our part in history. To be honest, I’m not sure even from a personal point of view it’s that important to think, “What could I do that will still not be automated even in 40 years’ time?” Because if there’s been an AI x-risk then that won’t happen, so you don’t need to worry about it. Zershaaneh Qureshi: Don’t worry, you’ll be dead. It’s fine! Benjamin Todd: And if there hasn’t been an AI x-risk, we’ll probably be in some type of society that’s 100 or 1,000 times richer than today — and your material needs will be taken care of. Zershaaneh Qureshi: Assuming someone sorted out the inequality thing. Benjamin Todd: Even if there is increasing inequality, imagine if the world was 100 times wealthier, but the billionaires get 1,000 times wealthier so they’re now like trillionaires — an ordinary person is still 100 times richer. It would have to be pretty bad for people to be actually worse off than now, you’d have to have inequality increase hugely and there be like no redistribution at all. Zershaaneh Qureshi: Yeah, OK. We’ve talked quite a lot about what kinds of jobs might be automated and what the dynamics of that look like. But my understanding is that in the book that’s not the only component people should be interested in when it comes to what jobs or what skills are likely to be valuable in the future, right? Benjamin Todd: I like to focus on skills because jobs involve many different skills, so that adds an extra layer of complexity. But yeah, we’ve touched on many of the factors already. The skills that will most increase in value due to AI would be things that are hard to automate, so hard to ‘get the last bit done.’ Secondly are those that are complementary to AI. So I used the example of AI engineers: if you’re still doing the things that are the key remaining bottlenecks, those increase in value. And as AI gets more useful, the value of making AI 1% more efficient again actually increases, because more is being done with AI. The third ones are those where we could use far more of the outputs, which was the elasticity point we talked about before. So like with accountants, it’s less obviously safe because maybe there’s a more fixed amount of it that we need. Whereas things like healthcare, luxury travel, or software are things that society could use way more of. And then the fourth are those where it’s hard for other people to learn the skills. So in the book I talk about the example of how being a waiter at a high-end restaurant is actually pretty hard for AI to automate. It’s a thing where we might really want a person to do it for the human touch. But it’s relatively easy for other people to switch into that job from other things, which could mean wages would probably still rise in line with the rest of the economy, but might not rise faster — it might not be one of the most valuable skills. On the other hand, people like electricians who work on data centres in Virginia, they’ve already seen big increases in wages because that’s a skill set that takes longer for other people to retrain into, and there’s a shortage of them now. Those would be the four key things to look for in which skills will increase in value. And then I would say you don’t want to just look at what will increase in value; you also want to look at what’s valuable now. And so start with that as your base point, and then you can imagine some will go up and some will go down. This is one reason why I think it doesn’t make that much sense to tell a college student who could do consulting to become a plumber instead, because the difference between those white-collar jobs and plumbers is still quite large now. So, even if white-collar jobs come down in wages and plumbers go up a bit in wages, it might be quite a long time before they actually cross. Zershaaneh Qureshi: Right. Benjamin Todd: So instead that person should probably be looking at a white-collar job, but in one of these parts that’s harder for AI to do. So maybe it’s more social skills heavy, or it’s a more messy thing, or it uses management skills. That’s probably where that type of person should be focusing first. If you were very on the fence and not sure whether to go to college or learn a trade, and it’s a very borderline case for you, then I think — compared to the past — there’s a stronger argument for doing the complex physical skills, which will take a bit longer to automate. Though of course eventually there will be robotics, so all of these things are about at the time frame. I talk about riding the wave of the things that are most valuable at the time, rather than trying to come to some permanent solution. Zershaaneh Qureshi: Yeah, so a big part of this is going to have to be flexibility. Like, this is what you do for now, but I guess people need to kind of stay on the pulse of what’s happening and work out when to pivot. Benjamin Todd: Yes. Zershaaneh Qureshi: Which is kind of tough. Benjamin Todd: Yeah, and that’s a meta skill that, if the pace of change increases, then the skill of learning quickly, changing path, and the kind of psychological resilience to do that, those things become more valuable. Zershaaneh Qureshi: So I think one pessimistic response I might have as a listener at this point is: if I do think that there’s a good chance of very advanced AI systems coming very soon and causing the world to change really quickly and posing loads of really severe existential-scale threats, as well as coming with a chance that at some point loads of people might end up being unemployed… I know you’ve offered some reasons for hope here, but I think that if I was listening to this and thought, if all of this is about to happen very soon, and I don’t feel super confident that people are going to be able to put the right mitigations in place in time, I might just think maybe the best way for me to spend my time now is doing the things that I enjoy before my livelihood gets taken away from me — or before everyone dies, or some other really horrible thing happens. What do you say to those people? Benjamin Todd: Even in many of the good scenarios, the world could still end up kind of totally alien to us. And yeah, there’s something very sad about that. I think people whose reaction to AI is just pure excitement, that feels very off base to me. It is a scary thing we’re facing. It’s not that helpful to dwell in the fear, but that’s not to deny that it’s scary. This is an insane thing to be happening and we might not be able to handle it — but then moving from that into, “So what can I actually do?” Ultimately it’s about focusing on the things you can change and accepting things you can’t. Although in some ways it’s an overwhelming situation to be in — and I think it actually will get more overwhelming because the pace of change will increase — I can sometimes feel some gratitude for: “Isn’t it amazing that I get to play any part in this at all?” One of the most important things to happen in history, how is it that we’re here at this moment and might be able to do anything about it? It’s a feeling of gratitude or almost like amazement at this situation. Zershaaneh Qureshi: I think it’s very easy to fall into these pessimistic styles of thinking. But I think that one thing that really does help with that is remembering all the stuff that we’ve talked about so far about the things that people can actually do to help. Yes, this might be a really critical time in history, but it’s sort of like a critical opportunity that people can take. There is tonnes of work still to be done, and as you’ve said, there are ways that people can pretty quickly shift into roles or start indirectly helping in a way that could actually set the future on a more positive path. And I find that encouraging and motivating. Benjamin Todd: I think I find for myself, just even on a selfish level, I feel a bit less stressed if I feel like I’m doing my part, like I figured out what I can do and I’m doing my best. We can’t guarantee this will go well, but there is a lot that can be done at the margin to make it significantly more likely. We’ve touched on a lot of resources in this conversation, but one I want to highlight is that we just released an article by Matt Beard on the 80,000 Hours Substack on how to transition into working on these risks in just three months. And it’s a step-by-step process you can work through with a lot of links detailing each step. And then I’d strongly encourage you to consider applying — if you’re interested in transitioning — to 80,000 Hours’s one-on-one advising and they can see if they can help you. And then the third thing is you could try and build any of these skill sets we’ve been talking about that could help you work on these issues — whether that’s organisation building, some type of communications, government and policy, more relevant technical skills; any of those things that put you in a good position to help. Zershaaneh Qureshi: And more broadly, there is also your book that’s now come out. Do you want to say anything about what you hope it will do? Benjamin Todd: In a way, it feels like my life’s work. It’s the last 15 years of thinking about this question at 80,000 Hours, and I’ve tried to distil all of our most central and important ideas into this one nicely honed package. So I’d be really honoured if anyone wanted to buy it and check it out. What would be amazing for this book is if it becomes one of these key things that, if someone doesn’t know what to do with their life, it’s a standard thing people recommend, and it becomes like a standard careers book. Maybe that could let 80,000 Hours reach a much bigger audience and help a lot more people work on these issues. Also, if you know anyone who’s feeling confused about what to do with their life, then you could consider getting it for them. And it’s released this week, so any orders this week can help us make the bestseller lists, which would also help us reach a lot more people. So really grateful for any help! Zershaaneh Qureshi: Thank you so much, Ben. Thank you for joining us. Benjamin Todd: Thanks for having me, it’s been great.