Latest / 80,000 Hours Podcast / Village gossip, pesticide bans, and gene drives: 17 experts on the future of global health
Transcript
- Lucia Coulter: I think that from the perspective of a manufacturer, they’re only putting in a small amount of lead: 1% intuitively doesn’t feel like that much, right? Rob Wiblin: It’s just a splash of lead. Just a little bit. Lucia Coulter: Just a tad. So I think it’s not intuitive how bad that could be. One way of communicating it is that if you had a little sugar sachet — the type that you would get at a cafe to put in your coffee — if that was filled with lead dust, and you sprinkled it across an area the size of an American football field, that level of lead loading would be sufficient to cause lead poisoning if a child spent time in that environment. So a very, very small amount of lead can have these really toxic effects. And that’s not intuitive; that’s pretty surprising. So I think that’s probably how people think about it. But often manufacturers, in some of our experience, will move really quickly. Sometimes days after we show them the results, they’ve ordered their non-lead alternative ingredients. Luisa Rodriguez: Hi listeners, Luisa here. You were just listening to Lucia Coulter, cofounder of the amazingly cost-effective Lead Exposure Elimination Project. She joined the show in 2023 to share what she’d learned about rapidly scaling her intervention to multiple countries and contexts. I actually started my career at global health and development charities GiveWell and Innovations for Poverty Action, so I’ve been a fan of our episodes with pioneers in global health since long before I became a host on the show. In this episode, we share some of the most powerful and actionable insights from 17 past guests who’ve worked on a huge range of evidence-backed interventions to save lives and help people flourish — from improving agricultural productivity to using AI to eradicate malaria to harnessing the power of village gossip to get kids immunised. You’ll hear from: James Tibenderana on whether we should use gene drives to wipe out the species of mosquitoes that cause malaria Karen Levy on why pushing for “sustainable” programmes isn’t as good as it sounds Sarah Eustis-Guthrie on whether more global health charities should shut down (as hers did) Plus many more. All right, I hope you enjoy! Development consultant Karen Levy on why pushing for “sustainable” programmes isn’t as good as it sounds Rob Wiblin: You’ve been in the global health and development scene for a few decades now and have seen, I imagine, plenty of things in your time. If you’re game with me putting you on the spot, I would be excited to hear about any kind of common ideas or conventional wisdom in the field that you suspect might be misguided, or perhaps is barking up the wrong tree in some way. Is that all right? Karen Levy: Sure. I’m always happy to share my very unorthodox views on some of these things. One of these concepts that I think we should interrogate a little bit is that of sustainability. And it’s not that sustainability is bad, but I think that insisting on sustainability — making it a prerequisite of support for a particular programme or policy — can really distract us from what matters or what should matter. In some ways, I think that over-privileging sustainability in assessing the effectiveness or cost effectiveness or the extent to which we should prioritise a particular intervention can be almost like the moral equivalent of saying — to draw on Peter Singer’s famous analogy — “Well, a lot of kids drown in this pond, and since we don’t have a plan to save them indefinitely into the future, we shouldn’t save this one now.” And similarly, if I told you that for some reason, we were going to have to stop manufacturing a particular vaccine in five years, would we just stop giving that vaccine now? If that vaccine is truly effective, and cost effective, then we should grab all of the impact that we can while we can. So what people usually mean by sustainability is, “How is this programme going to continue after whatever initial funding is given has ceased?” But unfortunately, I think what people often really are asking is, “How are you going to create a perpetual motion machine?” And in high school physics, I was taught that there’s no such thing as perpetual motion, so I feel like we’re kind of collectively trying to defy the laws of nature here. A lot of the problems that we seek to solve are going to require long-term sustained investment. So if a donor doesn’t want to or is not able to provide that support for a long period of time, that’s fine. But then I think really the conversation we should have is, “How are we going to shift the coverage of this to some other donor or some other source of funds?” — and not, “How are we going to put in a little money, and then it’s going to create an engine that’s going to then fund itself?” I think we should be thinking about how long we think a particular programme or a particular problem is going to require resources. Is this a permanent need? Is this something like basic education, which needs to be funded essentially indefinitely? Or is this something like deworming? I don’t want the Kenyan government to have to sustain a deworming programme 20 years from now — I want them to not have a worm problem anymore, right? Is this something we think with sufficient investment now we could really move the needle on over the next five to 10 years? We may find that identifying a highly cost-effective solution — and funding it reliably for 10 or 15 years — is way more cost effective than constantly starting and stopping and reprioritising and dabbling in various pilot programmes that never get continued, because we have a little bit of magical thinking that they’re going to somehow just perpetuate themselves. But there are some examples where that can be done — there are some programmes that can ultimately rely on market mechanisms. That’s terrific, and that can be a highly leveraged investment for philanthropic funds. But a lot of the work that we do, a lot of the problems that we seek to solve, require the provision of public goods and services that will not have market-based solutions — that are just going to require ongoing investment. Economist Dean Spears on the social forces and gender inequality that contribute to neonatal mortality in Uttar Pradesh Dean Spears: So my work is about babies in India and health and survival at the start of life. I work in a place called Uttar Pradesh, which is a state in India with 240 million people. One in every 33 people in the whole world lives in Uttar Pradesh. It would be the fifth largest country if it were its own country. If it were its own country, you’d probably know about its human development challenges, because it would have the highest neonatal mortality rate of any country except for South Sudan and Pakistan. Forty percent of children there are stunted. Only two-thirds of women are literate. So Uttar Pradesh is a place where there are lots of health challenges. And then even within that, we’re working in a district called Bahraich, where about 4 million people live. So even that district of Uttar Pradesh is the size of a country, and if it were its own country, it would have a higher neonatal mortality rate than any other country. In other words, babies born in Bahraich district are more likely to die in their first month of life than babies born in any country around the world. So it’s a place where there’s a lot of good that could be done. Luisa Rodriguez: Can you talk about why neonatal mortality is so high in Uttar Pradesh? Dean Spears: So around the world, prematurity and low birthweight are the second largest killer of children. But in India, they’re the first killer of children, and it’s a particularly large challenge in Uttar Pradesh. Babies that are born too small don’t have the ability to do as well as they need to do. And there are a lot of babies that are born underweight here, in part because moms are so underweight. So in short, a huge part of the problem is a lot of underweight babies. Luisa Rodriguez: I think of India as being middle-income and doing reasonably well on lots of these things, and I’m really taken aback that it’s actually one of the worst in the world on neonatal mortality. Do we understand why that is? Dean Spears: Yeah, it is surprising that there’s so much neonatal mortality in India. And the thing to keep in mind is that it’s not just India, but there’s a lot of differences within India, and it’s in particular in this northern India area, for example where Uttar Pradesh and Bihar are, where there is a lot more neonatal mortality than you would expect. And in this district where we work, about a third of the moms are underweight, in the sense of having a body mass index below 18.5. So why are moms underweight? Well, some of the reasons that moms in Uttar Pradesh are underweight are the reasons that sort of everybody in Uttar Pradesh has worse net nutrition than they might, such as the disease environment and poor sanitation. So moms and babies and dads all live in a place where sanitation — and yes, there’s still open defecation here — means that people are exposed to germs that use up their nutrition, that sap their nutrition and their energy through diarrhoeal disease, or just fighting infections. But it’s not just the poor sanitation and disease environment that’s causing so many moms to be underweight. Very importantly, it’s also social forces like gender inequality and the moms being socially low-ranking people. Luisa Rodriguez: Can you say more about that? Dean Spears: Yeah. So mothers in India tend to have their babies at younger ages. Unlike in, for example, sub-Saharan Africa, where childbearing careers are more spread out in age, a lot of the babies in India are born to moms in their early 20s. And that’s a time where women in India tend to be particularly likely to be underweight. You might just be thinking it’s a poor country, and that’s why there’s undernutrition. But there’s more undernutrition in India amongst women of childbearing age than in, for example, sub-Saharan Africa or the rest of the developing world. And what we see in India is this distinctive pattern where, especially in a place like Uttar Pradesh, the youngest women in their early 20s are particularly likely to be underweight. And as they get older, gain more social status, have children, they become less likely to be underweight. So the likelihood that a woman is underweight falls in age in a way that we just don’t see in the same way in the rest of the developing world. So India has this double challenge, where mothers tend to have babies young and that’s when they tend to be underweight. Luisa Rodriguez: Right. And why is it that this is happening in India and not in countries in sub-Saharan Africa, for example? Dean Spears: I think a very big reason is social status and women’s status in the sort of hierarchy that you find, especially in traditional households. Not all households are like this, but let’s zoom in on a special case that it’s easy to learn from, which is joint households. So Diane Coffey and Reetika Khera and I wrote a statistical paper about learning from joint households: where you have two brothers, they grew up together into adults, they got married, and they all lived together. In a household like this, the wife of the older brother is socially higher ranking than the wife of the younger brother. And the wife of the younger brother, the lower-ranking daughter-in-law, is expected to do more work for the family, and is later in line for getting the food she needs to eat, even during pregnancy. So that’s a situation where we can see an effect of a difference in women’s social status, even comparing cousins — their kids who live in the same family, live in the same house, live in the same village, so a lot of things would be held constant. So what do we see? Well, we do see that the lower-ranking daughters-in-law are thinner, and that’s even though they have the same height. And so it’s not about their early life nutrition; it’s about what happens to them in adulthood. Luisa Rodriguez: Interesting. Dean Spears: Yeah. It’s like we have a little experiment here. We can look in these families as petri dishes to see, holding these other things constant, what’s the effect of mom being underweight for these social reasons? So what is the effect? Well, we see that the children of the lower-ranking daughter-in-law are more likely to die of neonatal deaths. And we see that they’re more likely to be stunted and small in other ways. So in a situation where we’re pretty sure that the difference in maternal nutrition is coming from these social forces, we’re seeing it all the way through into neonatal mortality. Luisa Rodriguez: That is fascinating and depressing. Dean Spears: I mean, I’m optimistic that in the long run, it won’t always be like this. It is the case, especially in other parts of India, that maternal undernutrition has gone down. These things are changing over time. A few years ago, 35% of women in Bahraich were underweight. In the most recent survey, only 30% of women in Bahraich are underweight. So it’s moving in that direction. And hopefully, in coming decades, this will not be such a challenge anymore. In the meanwhile, there’s going to be a lot of low birthweight babies in Uttar Pradesh who need this sort of professional nursing care. Charity founder Sarah Eustis-Guthrie on what we can learn from the massive failure of PlayPumps Sarah Eustis-Guthrie: For people who aren’t familiar, PlayPumps is this charity where they had this really exciting idea. You need to draw water from wells in many low- and middle-income countries, and often this involves a lot of laborious pumping that women especially have to do. So they designed this play structure, where the idea was kids play on the play structure, and that automatically pumps the water. And this was a really exciting idea. It just has this kind of intuitive appeal. Luisa Rodriguez: Super wholesome. Sarah Eustis-Guthrie: Yeah, yeah. It got huge amounts of money. And then it turned out, whoops, it actually doesn’t work very well. EAs love to hold this up as an example of, look, things in development often don’t work. But when I started digging into this example, I was like, wow, this is even worse than I thought it was — because it wasn’t just that this was something popular that was kind of like in magazine ads and people on the street started funding it; it got this big grant in 2000 and then grew in momentum over the next couple of years. And you got to this point where Laura Bush announced a $16 million USAID contribution to PlayPumps, and big celebrities were doing these benefit concerts. And it was actually only once they had built like hundreds, maybe even over 1,000 of these PlayPumps that some of these reports were commissioned or started seriously circulating, which actually went and figured out how well it worked. And it seemed from the reports that it was pretty easy to figure out that it didn’t work that well. I don’t know exactly what had happened in the process, but clearly before making these big donations, organisations — including orgs like USAID that require a ridiculous amount of reporting — were apparently not requiring reporting on the most important thing, which is like, that this thing actually works. So it turns out that there’s actually a bunch of different problems with PlayPumps. And just for context, a PlayPump costs about four times as much as a regular hand pump, and sometimes they replaced the regular pump with the PlayPump. And then the PlayPumps were really complex, they had a lot of parts, so when they broke down, it was tough to repair them. So not only are PlayPumps less effective, but I think you could argue that they were actually a net loss for communities. So that I think is a really damning part of this. And the guy who founded PlayPumps, as far as I know, previously I think had been an advertising salesman. I think this speaks to what we were talking about the incentives — where in order to make this programme happen and to make this big change happen to the lives of these people in low- and middle-income countries, in many communities, they weren’t at all consulted. This thing just happened that made their lives worse because some salesman was able to sell something — not only to people on the street, but to some of the people making big decisions at the US’s biggest development agency. So to me, that speaks to the problems with current philanthropy. I think what charities have to do to be more accountable is to take more seriously this monitoring and evaluation: to, for any intervention that they’re doing, actually do some sort of proof-of-concept test where they can go out in the field and consult with potential beneficiaries to see if it’s really helping them. But you can’t just ask them, “Is this helping you?” because they might feel pressured to say yes — but see, is it really connecting to aspects of their life where it’s meaningfully improving them? So that doesn’t mean giving textbooks written in a language that the kids can’t read. That doesn’t mean giving laptops to kids where there isn’t electricity, or where their reading isn’t very good either. It means trying to focus on the interventions that are meaningfully improving people’s lives, and taking a serious look at even some interventions that sound really good, even interventions that look good in initial randomised control trials but don’t always scale. You have to have this consistent, sceptical perspective, where you’re running an organisation and you’re saying, “We’re so enthused to be running this organisation, and we trust that we all have great intentions here. But we also are going to be dedicating a meaningful part of our resources to checking up on: Is this intervention a good idea? Is it still improving people’s lives meaningfully?” Economist Rachel Glennerster on how randomised controlled trials are just one way to better understand tricky development problems Nathan Labenz: Moving to RCTs in general and the state of debate around how much we should rely upon them: you mentioned that it’s kind of a 50/50 split right now, in today’s work. Do you think that’s an appropriate split? Do you think that it should be all RCTs? What do you think is the right balance as we try to figure out what is obviously a very complicated world? Rachel Glennerster: I think it’s really important to say that all of us who have worked on randomised trials have never suggested that this is the only methodology that you should use. People criticise us for saying it’s the only methodology, but nobody who’s done RCTs has ever thought that that’s the right approach. I think the right way to see things is you have a toolbox of ways to answer questions, and the right tool depends on the question that you’re asking. I think we need good descriptive work to understand what the problems are. A lot of development programmes just fail because they’re trying to solve a problem that doesn’t exist. They’re just solving the wrong problem. The first really important thing you’ve got to do is really understand what the issue is in any given area. If we’re worried about girls not going to school because of menstruation, well, let’s start by finding out whether they actually don’t go to school more when they’re menstruating. That’s a really basic, obvious thing. But we actually need more work on that. Understanding the context, understanding the problem is a really important first step. When I started doing agricultural work in Sierra Leone, the first thing we did was work with the government to do a really detailed analysis of what are the problems for smallholder farmers in Sierra Leone? Not RCT, just descriptive. It turned up all sorts of interesting facts that people weren’t aware of. I think that’s really important. I think then doing an RCT is useful for answering a really specific problem, a really specific question. But I think the best RCTs are the ones that test a theory. They test something that’s more generalisable than just, “Does this programme work?” — it’s asking a question about human beings. Here is an example: I did a project looking at how to improve immunisation rates in India, which was fantastically effective. It started with a first assessment of what are the health problems in this area? Only 3% of kids in this part of India were getting fully immunised. Given that immunisation is one of the most effective things that you could do, that rate is just appallingly low. There were a number of theories about why that could be, and a lot of people said people don’t trust the doctors. Well, not doctors — because you rarely get doctors in rural India or rural anywhere — but nurses and clinics. So they don’t trust the formal health system. There was also a question of, the clinics are often closed, so is that the problem? Is it that when you go and take your kid to the clinic, it’s often closed? Is it nurse absenteeism that’s the problem? Or is it just a behavioural economics thing — that you’re happy to get your kid immunised, but you’ll do it tomorrow? We read all this behavioural economics and we said maybe we should look at that. But we also wanted to test these other ideas. One arm made sure that, without fail, there was someone to immunise your child; and another arm did that, but also provided a small incentive. So yes, we were testing a programme — but we were also asking a more fundamental question: why don’t people get their kids immunised? What we saw in the data is a lot of people got their kid immunised with one immunisation, but they failed to persist to the end of the schedule. Already, that’s just descriptive data and it starts to tell you that it’s not that they distrust the system or that they think that immunisations are evil, because they’re getting their kid one immunisation. It’s more a question of persistence. Now, fixing the supply problem increased the number of people getting the first shot, and the second shot — but again, it failed to fix this persistence problem. Where the incentive effect worked was it helped people persist to the end. That tells you that one of the big problems was this persistence problem. It tells you a lot about why immunisation isn’t happening. Now, that project was completely impossible to scale. This was like economists designing logistics. It was a disaster. We learned a lot, but you would never want to actually do a programme like this. The logistics was a nightmare, but it tested a theory. Once you have that, you can think about what the implementation issues are, and how do we implement this at scale? Because you better understand the problem. You want to use an RCT when you can test a specific problem and get an answer to why something is an issue: it’s an important question, you can answer it well, and it has broader implications. But you also need to use other types of methodology when your question is of a different kind. Data scientist Hannah Ritchie on why improving agricultural productivity in sub-Saharan Africa is critical to solving global poverty Luisa Rodriguez: You’ve written an article that makes a claim that low agricultural productivity across sub-Saharan Africa is one of the most important problems this century. I’m interested in digging into that, and what concrete solutions might make achieving it possible. Hannah Ritchie: I should preface this by saying that one of the reasons I think it’s an important problem is I think it’s very overlooked and underrated as a problem. And I think that because it’s somewhat overlooked and slightly complex, I’m not going to pretend that I have all of the answers or solutions, because I think there’s open questions there that I wish people were paying more attention to. But my argument for why I think it’s one of the most important problems this century, I think there are two elements to it. There’s a very human element to it. I’m also aware that we’re talking about it regionally, when there’s obviously very large differences across the region, but if you look at sub-Saharan Africa as a whole, around 40% of the population still live on less than $1.90 a day, so below the international poverty line, which is very low. And there’ll be many more that are not that far above it. And then if you look at the employment across the region, more than half work in agriculture. And if you look at the poorest, it’s estimated that around three-quarters of those that are living in the deepest poverty are farmers, so they work in agriculture. So basically, what we’re saying is that most of the world’s poorest work in agriculture. And the problem they face is that they’re often really smallholder farmers; they don’t have the capital or the money to invest in fertilisers or machinery or to expand their land. So basically, they need labour. It’s often a family-run farm, where everyone in the family has to contribute. There’s no money to invest in education elsewhere, so they almost get trapped in this cycle where they don’t get a lot from crop production, but everyone in the family has to work there to just stay afloat. Basically, you get locked in. There’s almost no opportunities externally to go elsewhere. So one of my core arguments is that if you’re going to address global poverty, you have to increase agricultural productivity in sub-Saharan Africa. There’s almost no way of avoiding that. Luisa Rodriguez: Right. So the idea is that so many of these people that are earning the least in this region, which has many of the world’s poorest, are working in agriculture, but producing so little relative to the land they have that they’re not earning nearly as much as they could. And by increasing their crop yields — which at the moment, they can’t do because they just can’t afford to invest in it — you could lift a bunch of people out of poverty. And that just sounds like a huge win. Hannah Ritchie: There’s obviously human benefits to it, but there’s also the environmental benefits. One of the impacts of having low crop yields is that you just need much more land to grow your food. And that’s going to be a particularly pressing problem in sub-Saharan Africa, because that’s also where we’re going to see the largest population growth in the next 50 years or so. So sub-Saharan Africa is going to need to produce even more food, and if they don’t increase crop yields, then that’s just going to come from expanding land, often at the cost of forests. So there’s a very strong environmental case for, if you want to address deforestation and biodiversity loss, then you have to somehow increase crop yields. Luisa Rodriguez: Right. You’ve argued that the reason it’s so low in sub-Saharan Africa is because both labour productivity and land productivity are super low. So I wanted to talk about each of those in turn. Can you explain what labour productivity is? Hannah Ritchie: Labour productivity is basically just how much money — or in this case, crops that you then sell for money — you get out per unit of input. Here, we’re talking about per hour worked, or per worker. It’s basically how much human effort you have to put in to get like a dollar value in return. Luisa Rodriguez: OK, and that could be that if you’ve got great machinery, your labour productivity might be high, because you can use a tractor as one person, but do a bunch of productive work. Whereas if you don’t have a tractor and you’re using a hoe, your labour productivity might be much lower because you can get less done. Is that right? Hannah Ritchie: Exactly. I often think about it in terms of tending a garden, where often it’s really intense work from a human labour perspective. You really need to work really hard and see to it often — and often, the amount you get back is not that much. In that case, your labour productivity is really low, because you’re working really hard and not growing that much. Which, if you’re just thinking about your garden, it’s fine, but if that’s your livelihood, that’s not good. Luisa Rodriguez: That makes sense. And how does labour productivity in sub-Saharan Africa compare to other regions? Hannah Ritchie: So if we use a metric for it as the amount of value you’d get per worker — so the economic value per person working on the farm — the average for sub-Saharan Africa is half of the global average. Luisa Rodriguez: Oh, wow. That is much lower than I would have guessed you would have said. Hannah Ritchie: It’s like half of the global average, but it’s 50 times lower than you’d get in the UK or the US. Luisa Rodriguez: I spoke too soon. That’s much more shocking. Hannah Ritchie: No, it gets worse. If you look at some countries within sub-Saharan Africa, they’re like half of the sub-Saharan Africa average — so there you’re talking about 100 times less than you’d get in the UK or the US. Luisa Rodriguez: That’s hard to even fathom. Hannah Ritchie: So you put that in context: the value that an average farmer in the US might create in three to four days is the same as a Tanzanian farmer for the entire year. Luisa Rodriguez: That is really mind-blowing. And it makes it super visceral to me why this would be a huge problem, both economically and environmentally. Why is it so low in sub-Saharan Africa? Hannah Ritchie: I think there’s a couple of reasons. One is that the farms are really small, so often the amount of crop or value you get out is quite low. And maybe we’ll come onto crop yields: with low crop yields, you get not that much out — but also, as you said, you can’t afford machinery, or you can’t afford fertilisers or pesticides, or things that would basically substitute for human power inputs. So it just means you need lots of hands on deck to keep the farm going and keep it at that baseline level of productivity. So you don’t get much out, and you just need lots of people working on the farm. Luisa Rodriguez: OK, and then the other thing that seems to be really low here is land productivity, which feels a bit more intuitive to me. Is that basically how much crop yield you’d get from, for example, an acre of land? Hannah Ritchie: Yeah, exactly. It’s what most people would call a crop yield. So say we each have a hectare to grow wheat. If you got four tonnes, and I got two tonnes, your land productivity would be twice what I get. It’s just how much you get from a unit of land. Luisa Rodriguez: I imagine things that factor into that are some things that are intrinsic to the area — the quality of soil naturally, or whether it’s clayey or not — but also things that you could do to the land, like use fertilisers. Are there other things? Hannah Ritchie: Yeah, there’s a couple of factors that come into it. One is, as you say, the quality of land. So the texture of the soil, the natural nutrient density, carbon content of the soil, how well it drains: all of these affect how well a crop will grow. But there are also ways that we can change some of those aspects: we can use irrigation or drainage to determine how much water is in the soil, or we can apply our own nutrients for fertilisers. So there’s natural conditions, but there’s also inputs that we can use to change that. Luisa Rodriguez: Cool. And again, I’m guessing land productivity is much lower in sub-Saharan Africa. How much lower is it relative to other regions? Hannah Ritchie: Again, it’s very low. One way we can compare is we would use cereal yields, because most regions grow some cereals. If you compare the average in sub-Saharan Africa, it’s about half that of India, which is less than half of the global average. So it’s pretty poor. And then if you compare that to richer countries, it’s like four to five times lower. But then again, there are countries within sub-Saharan Africa that will get half again of that regional average — so there are some countries where you’re talking about getting 10 times less per unit of land than in rich countries. Which again makes the environmental point really clear. Imagine if we globally had to use 10 times the amount of land to produce our crops. Luisa Rodriguez: Right. And why is that? Is it something in particular about sub-Saharan Africa and the natural things about the soil and environment? Hannah Ritchie: I don’t think so. And the reason for that is that there are examples of some countries within sub-Saharan Africa, or even for particular crop types, where they can get good crop yields. I don’t think it’s just an issue that, in sub-Saharan Africa, you can’t grow food. I don’t think that’s the case. It’s quite hotly debated as to what the issue is there. There are very obvious inputs problems: can’t afford fertilisers, can’t afford irrigation. So there’s a range of, they just don’t have the inputs or seeds that they would need to do that. There’s also this interesting hypothesis — I’m not sure how convinced I am by it, and I’ll give the reason for that — that rather than it being a supply problem, there’s also a demand problem. Say you’re a farmer in sub-Saharan Africa, and you’re growing to subsistence level to feed your family. And you don’t have access to a market to sell any more than that — either you can’t get to the market; or at the market, people can’t afford to buy goods from you — then maybe you have no incentive to grow any more food than that and raise yields, because you would have to invest in fertilisers and irrigation stuff. Which, if you can’t sell the extra food that you’re going to grow, then why would you do that? So there’s this hypothesis that there’s a demand problem, where there’s not accessible markets to sell more. I get that, and I think there’s probably examples where that’s true. One reason I’m not completely convinced is that it’s not even apparent to me that many farmers are actually reaching subsistence, because so many within sub-Saharan Africa are undernourished; they don’t get enough food to eat. So it’s not obvious to me that they’re actually growing enough, even just to feed and meet their basic needs themselves. Luisa Rodriguez: Right. Yeah, that makes sense. One thing I learned while reading your work on this in preparation for the interview was that most of the world actually used to have much worse labour and land productivity, that was actually similar to sub-Saharan Africa’s. What were the key things they did to improve that sub-Saharan Africa didn’t end up doing? Hannah Ritchie: Yeah, that’s true. We think about these low-yielding countries as outliers now, but for most of agricultural history, that was just the norm, that was just the default. And the basic reason there is that our farming was just at the whims of nature. Like water came when it came — you couldn’t really control it; you couldn’t irrigate the soil. You had to just deal with the nutrients that were in the soil at the time because you couldn’t add any more. What’s really changed there is, one, we’ve been able to invest in irrigation and improved seed varieties and stuff. But a big change for many countries — and we see this inflection point in yields — was the beginning of the Haber-Bosch process in the early 20th century, where we basically figured out that we could make our own nitrogen fertiliser. For most crops, nitrogen was the limiting factor; it’s why it wasn’t growing. But we figured out that we could add it when we wanted to add it, and that’s been a massive driver of increased crop yields. Luisa Rodriguez: And why was it possible for the rest of the world to take advantage of that, but not sub-Saharan Africa? Hannah Ritchie: One obvious thing is that it just costs money, so if you have more wealth then you can invest in that. Some other countries have been really successful in subsidising fertilisers, because they realise that it’s so important to break this poverty trap, so they’ve subsidised it so it’s much cheaper for the farmer in the first place. There’s also the pairing of fertiliser inputs with having the right seed varieties. Many countries have gone through this kind of inflection point with basically genetic breeding of particular seed varieties that were really successful. We just haven’t really seen the same in sub-Saharan Africa. I’m not completely sure why. I think regionally, it’s just even from an agrochemical perspective — which has its critics, but if you’re looking for genetic breeding of particular seed varieties and fertiliser inputs and stuff, it plays a crucial role. And if you look at a lot of agrochemical companies today, sub-Saharan Africa is just not even on their radar. Some of the big companies, for example, would do regional reports and have regional divisions for some of them. It doesn’t even warrant its own region — it’s lumped in with Europe. Luisa Rodriguez: Oh, that’s really depressing. Hannah Ritchie: So basically what they’re saying there is, “Europe and North America and Asia — and to some extent, South America — make us loads of money. And Africa doesn’t. Therefore we’ll just lump it in with Europe and report it under the European numbers.” So I think part of my motivation for highlighting this as a big problem is to somewhat shift the focus. Luisa Rodriguez: Yeah. That makes tonnes of sense. If they were able to catch up to other regions on both labour and land productivity, how much of a dent could that make on the issues at stake? Poverty, hunger, wildlife destruction? Hannah Ritchie: It would be massive. If you do projections, just even out to 2030, on the number of people that will live in extreme poverty, most of them will be in sub-Saharan Africa. And as I said before, three-quarters of those that are living in the deepest poverty are farmers. My argument is that we’re not going to address global extreme poverty unless we fix this, and to me, that’s arguably our most pressing problem we face. It’s just not possible without doing that. There’s also the obvious argument that that’s also where population growth is going to be happening most over the coming decades. And already around 20% in sub-Saharan Africa don’t get enough calories every day; if we don’t improve crop yields there, that’s just going to get worse as the population grows. And finally, the environmental arguments. Some of my colleagues at Oxford University basically modelled what would happen to habitat loss for different species out to 2050 and what we could do about it. And one of their key findings is that in sub-Saharan Africa, if things don’t change, there’s going to be a lot of deforestation because of low crop yields. But what we can do — and actually this links back to the question of, is it just that maybe sub-Saharan Africa just doesn’t have the land and stuff to do this — researchers can also calculate what they call “attainable yields,” which is what yields these countries could achieve if they had the right technologies and fertilisers and stuff. And the estimates come out that they could around triple their current yields. Luisa Rodriguez: Just by changing how much fertiliser they’re able to use, the irrigation technologies they’re able to use, that kind of thing? Hannah Ritchie: Using existing technologies that we have in a good way, they could basically triple yields. So it’s not that this is not addressable. Luisa Rodriguez: Right. It’s not a pipe dream. Concretely, it sounds like the way to achieve that is doing whatever these scientists found was best to improve yields in terms of whatever these technologies are that would make the biggest dent. I’m both interested in which technologies those were, and also how we actually get those used, given that it’s not happened yet. Hannah Ritchie: I don’t think there’s one single thing. I think there’s a bunch of low-hanging fruits, and it’s very context dependent. I’ve seen very good evidence that one of the lowest-hanging fruits is irrigation. The other one is fertilisers. As I said, many other countries have heavily subsidised fertilisers, at least for a brief period of time until you can almost break that deadlock. That has come with other issues on the other end, so you often find countries that have really heavily subsidised fertilisers now overconsume the most because they’re so cheap, like farmers just put as much on as they can. But I think there’s really low-hanging fruit there. Again, I think investment from agrotech in this region is really important. To me, it just seems very obvious that if you could get that market going, it’s massive. I think it needs to be very early investments, either at a very high discount or subsidised in some way. I definitely think my suggestion here to some is controversial, because there are controversies around the way that many of these companies behave in these markets. There’s always the big backlash against Monsanto, for example, where they basically try to create a monopoly. And I think there would be concerns that some of those relationships could be quite exploitative in the end — where basically, agrotech companies create seed varieties that only work with a specific fertiliser that they then sell — and you could almost imagine farmers getting stuck in another trap. So I’m very aware of that, and don’t want to overstate that this is some easy solution that doesn’t have issues. But to me, it seems clear that we’re not going to massively increase agricultural productivity there in the region without some of these investments. Luisa Rodriguez: So what does it look like for this to go really well? Who has to do what? And then how does that trickle down to changing these outcomes? Hannah Ritchie: The big players are obviously the country governments themselves. And there, I get that it’s difficult when your finances are already constrained on how to allocate resources. But to me, it just seems like this is a really key fundamental problem, and these economies are not going to grow significantly until they address the problem. Luisa Rodriguez: And those governments are having to do things like subsidise some of these inputs? Or some other types of programmes that address the biggest issues? Hannah Ritchie: Yeah. One is subsidising or promoting the inputs. One interesting dimension is the market demand problem: how you create an environment, either within a country or internationally, where there is just a really strong market. I think some of that can come from domestic governments, but I think there’s also an international role to play there. You can, for example, change trade tariffs in some way — where you can give preferential treatment or even just equal the playing field a bit — such that there’s larger domestic markets and incentives, but also much larger international markets for farmers to sell into. Luisa Rodriguez: Right. So lowering the tariffs on crops produced in sub-Saharan Africa might make it so that there’s a bigger market for those smallholder farmers to sell to. And then when there’s a bigger market and they’re earning more, there’ll be this feedback loop where they’re then investing more in inputs. Hannah Ritchie: Exactly. There’s also just a large agricultural research dimension. As I said, I found the research on this quite murky in terms of really pinpointing how we have a big impact here. I think partly because it’s just so heterogeneous across the region, as you’d expect. But I think there is just a large area for research on how particular soils affect the growth of these crops, how different seed varieties fit in, what’s the ideal combination of fertiliser and irrigation, for example. So I think there’s still lots of room for good research. Charity founder Lucia Coulter on the huge, neglected upsides of reducing lead exposure Rob Wiblin: How many times more damage is lead doing in countries that LEEP operates in, as opposed to countries like the US or UK, where most listeners live? Lucia Coulter: So LEEP operates in low- and middle-income countries, and 95% of the global burden of lead poisoning roughly is concentrated in low- and middle-income countries. In low- and middle-income countries, the average blood lead level is around 5 micrograms per decilitre, which is classified as lead poisoning — so on average, most children have lead poisoning. And in high-income countries, the average childhood blood lead level is around 1 microgram per decilitre. It’s a bit lower than that in the US. Like you say, the data is not amazing anywhere on this, but it’s about five times lower in high-income countries. And the Flint, Michigan example is really stark. I think that was a huge crisis, having 5% of children in Flint, Michigan with lead poisoning. But the fact that just every single day, 50% of children in low- and middle-income countries have that level of lead poisoning is really concerning, really troubling. So we could think about it in terms of what would the impact be on the average child in a low- or middle-income country? The average child in a low- or middle-income country has a blood lead level of around 5 micrograms per decilitre, and that’s high enough to cause health, educational, and economic impacts. So a child with that blood lead level would have a reduction in IQ anywhere from around one to six IQ points, depending on which analysis you take. That, in turn, will affect their future earning potential. They’ll also have reduced educational attainment. There was a recent analysis by the Center for Global Development that pretty conservatively concluded that that would be equivalent to around one year of lost schooling. Then it also causes an increased likelihood of cardiovascular disease and mortality from cardiovascular disease. And that could be as high as a relative risk of around 1.5 at the average level of lead exposure that children have in low- and middle-income countries. That’s according to a recent analysis of US data. And then on top of all of that, it increases risk of kidney disease, anaemia, foetal health problems, behavioural disorders, ADHD, and possibly even mental health problems and dementia. Rob Wiblin: We’ve known for at least 100 years that lead is poisonous. I guess we can see that because France banned leaded paint in 1909. I think they were one of the first countries to do it. But nonetheless, they thought it was sufficiently dodgy that there was lead going in paint in houses that they banned it 110 years ago. And apparently even the ancient Romans suspected that lead was bad for you. I guess they probably didn’t have gold standard randomised controlled trials, but they probably noticed that people who worked in lead mines ended up with extreme health problems, and figured out that lead was probably bad for you. Is there a simple reason why this problem hasn’t already been solved? Why didn’t we know in the 19th century to stop adding lead to stuff that people were going to be eating? Lucia Coulter: It’s a good question. It’s probably not that simple of an answer. I think to start with, lead is just a really useful metal: it’s abundant, it’s malleable, it’s durable, and its compounds make loads of really helpful things — like strong glazes, bright pigments, antiknocking fuels. I think in the 1920s the industry in the US described lead as “a gift from God” because it’s just such a great thing. So I think people will just keep using it unless they aren’t able to, unless they’re strongly incentivised not to. I think another reason is that there is extremely low awareness of both the prevalence of lead poisoning and the harms of lead poisoning, and the sources of exposure. Low awareness generally, but also among important decision-makers, important institutions, and low- and middle-income country governments and funders. I guess that leads to the question: Why is the awareness so low? Rob Wiblin: Yeah, maybe this is something that a historian should be looking into understanding. Maybe they could find some mentions in parliamentary records or something from the 19th century of people raising the question of whether lead was safe. There must have been some stuff written about it, if it was banned in France in 1909. But why is it that that didn’t win the day? Why is it that industry, that wanted to add lead to things, won out the debate? Lucia Coulter: Yeah, I always wonder if one part of it is just the really invisible nature of lead as a poison. Of course impacts aren’t invisible: millions of deaths and trillions of dollars in lost income. But the fact that lead is the cause is not apparent. It’s not apparent when you’re being exposed to the lead. The paint just looks like any other paint; the cookware looks like any other cookware. And also, if you are suffering the effects of lead poisoning, if you have cognitive impairment and heart disease, you’re not going to think, “Oh, it was that lead exposure.” It’s just not going to be clear. Rob Wiblin: I guess it’s an issue with how we tend to treat specific acute diseases and be very aware of that, but then everyone suffering some relatively small chronic impact just doesn’t really rise to the level of anyone’s notice and cause a public outcry. Lucia Coulter: Yes, exactly. And the symptoms aren’t specifically characteristic of the cause. It’s not like malaria, where you get cyclical fevers and it’s like, obviously something going on here. I think that’s part of it as well. Malaria expert James Tibenderana on using gene drives to wipe out the species of mosquitoes that cause malaria From episode #129 – Dr James Tibenderana on the state of the art in malaria control and elimination Rob Wiblin: What do you think of the idea of using gene drives to eliminate the specific species of mosquitoes that carry the malaria parasite? I think there’s an organisation called Target Malaria that is working on research to figure out where this might be possible and whether it’s a good idea. Basically the idea here is you release a bunch of mosquitoes that have been genetically engineered such that whenever they mate with a partner, all of the children could either be infertile, or they could all be males or all be females, or something like that. That means that if that happens every generation, then after a series of replication cycles, they all die out because there’s no females or no males left, and so largely the species will go extinct locally. This does have the implication that if you release some of these mosquitoes and they don’t evolve to break this gene drive technology, that this could spread globally and eliminate that species of mosquito completely, which has proven slightly controversial with some people. Do you want to talk about gene drives and Target Malaria? James Tibenderana: Yeah. Gene drives are a novel intervention that certainly has a lot of potential. There’s still a long way to go. There’s quite a bit of research, especially epidemiological research, that needs to be done to understand the potential impact. But I think the technology itself is very powerful. And I think therein lies some of the risk. Rob Wiblin: It’s almost too powerful. James Tibenderana: Yeah. It’s really powerful. And of the gene drive systems, there’s sort of low-threshold gene drives and high-threshold gene drives. With high-threshold gene drives, you need a large number of mosquitoes released into the wild to have the impact. And with low-threshold gene drives, you need a few mosquitoes to be released into the wild and really propagate itself. And then within those, you have some gene drives that once introduced will suppress the population of mosquitoes or insects, and then you have some gene drives that will modify elements within the mosquito or the insect, to cause it to not be as efficient or not able to perpetuate a particular attribute. I think in terms of the gene drives for mosquitoes, Anopheles specifically, really it’s a low-threshold type of gene drive, and one that will cause suppression because of the doublesex gene that creates a generation of infertile offspring, and that continues to perpetuate itself until you actually sort of wipe out that particular species. I think the studies that have been done in cages so far suggest that that is possible, so it is certainly a very powerful tool. I guess the question that we will all have — and we see this in nets and we see it in some of the other tools — is what will be the acceptance of national governments or communities to a technology that can be complex to understand? The minute you get into genetically modified technologies, it raises eyebrows. And you have adopters, but you’ll also have the sceptics. I think that’s one thing Target Malaria is conscious of, and they’ve spent a huge amount of time working with communities, and working with members of parliament in some of these countries to understand what the legislative framework is, and what kind of information the decision-makers require for a policy to be adopted in terms of gene drive mosquitoes, or gene drive insects in general. And then communities: how will they perceive these genetically modified mosquitoes? I think they’ve really done some very good community engagement work, some studies, and they’re starting to show that there are ways that one can communicate, with both communities but also decision-makers, that can potentially make this adoptable. That’s one element. Now, with this type of technology, you can’t simply say, “I’m releasing it in one country and it’s not going to spread to the next country.” Rob Wiblin: It’s probably going to spread everywhere eventually. James Tibenderana: Exactly. So not only do you need a country adoption, you probably need regional adoption. So if you release it in West Africa, then what is ECOWAS going to think about it? If you release it in Southern Africa, what is SADC going to think about it? So you have all these regional bodies, the African Union, et cetera — what is going to be their perception of gene drive, genetically modified mosquitoes that are released? Because you really need the governance framework and the legislative framework that is regional, rather than just country specific. And that requires you to also be able to engage with all these communities. I’m not saying that this tool isn’t powerful. This is a really powerful tool, but I think in its power lies some of its risk. And you have then the question, how do you turn it off? Either when you don’t need it any longer, or — as we know with catastrophic risks, in the future something goes wrong and we don’t expect it to — if it does cross over into other insects unintentionally: how do you turn it off? So there’s the technology, but I think we also need the cross-country collaboration and governance. And we do need a good understanding as to what will happen if we have unintended consequences and we need to turn this technology off. Rob Wiblin: Yeah. I think there is a way of turning it off as I understand. Unfortunately I didn’t look into this, but I think basically you would release new mosquitoes with a gene drive that cuts out and deactivates the other gene drive, basically. So there is an option, but it’s a challenge. Or it’s something we haven’t done before. James Tibenderana: Exactly. So you need the two happening almost concurrently. I mean, let’s see what happens when it goes into larger-scale trials in different locations. But it certainly has potential, and it could be an important game changer maybe in the next 10 or 20 years. But I think you do need the investment in understanding how to mitigate some of the risks of the technology, so that we are able to describe both the technology as well as the mitigation of those risks to communities and governments who need to be on board for this to be introduced and have the impact that it could achieve. Rob Wiblin: Yeah. A lot of people worry about unintended consequences here, or they’re nervous about this technology. I think that’s understandable. I’m a bit more of a risk taker maybe. I just want to say, to be frank, we’re talking about driving very specific species of mosquitoes extinct globally. We’re trying to get rid of them because, unfortunately, they’re the ones that carry this parasite. However, we would in the process save 600,000 children’s lives every year, and prevent 200 million cases of this very unpleasant illness. It could be something that really does end malaria, or at least dramatically reduce malaria. So the benefits we’re talking about here are very large. And I think some people hear this and they’re like, “You’re getting rid of all mosquitoes.” To be honest, I’m not sure that I would be that against getting rid of mosquitoes. Maybe there’d be other insects that could fill that niche that aren’t as annoying as mosquitoes. However, that’s not actually what’s being suggested, because there’s tonnes of other species of mosquitoes that don’t carry the malaria parasite and so don’t have this problem. And likely, given that they are an extremely similar insect, they would probably just colonise the same niche in the environment that the Anopheles mosquitoes and so on are currently filling. So to me, it does just seem like the benefits greatly outweigh the costs on their face. So I would like to see maybe a bit more hustle about figuring out how we can do this. To some extent, it surprises me that there hasn’t been one country that’s been like, “We want to get rid of malaria and we’re just going to do this.” And then that does have effects on other countries, and maybe they don’t like it. But it’s something where a single actor can potentially do this for the whole world if they’re willing to be unilateralist about it. It’s kind of interesting that that hasn’t happened, or that there’s no proposal, or that it doesn’t seem like that’s likely to happen anytime soon. James Tibenderana: Yeah. I mean, Rob, there’s like 3,500 species of mosquitoes. So getting rid of 40 specific species, as you know with gene drives, you have to go species by species, right? There’s 4 billion people at risk of malaria. I think if we ask those 4 billion people about how they would feel getting rid of these vectors, I suspect they will be keen not to be at risk of malaria. So there are benefits certainly — in the short term and probably in the long term — because of the value of a malaria-free world. Remember it’s not just the disease, but it’s also the economic benefits that will be had if we are able to achieve a malaria-free world. So I think there’s huge benefits — and like you, I would certainly be a proponent of the risks. But that’s you and me. We’ve still got to recognise that national governments — and I suspect they will see the urgency, they will see the need — but at the same time, we have to recognise that they have to go through their own legislative process. It’s not just going to be a ministry of health policy. This is a big deal. We have to recognise that countries will have to understand both the benefits and understand the risks. And providing the right evidence, and at least a sense of what the potential mitigations are for the risks, would really go a long way in the fast adoption of the technology. So that we don’t have a technology that we’ve shown we’ve proven, and then we spend another five years trying to get it adopted. I think what we all want is that by the time this technology is available, everyone is on board, engaged communities, they’re ready, and the potential can be achieved. I mean, the history of malaria seems to be that you take 15 years from a WHO recommendation to a potential scaleup of a tool. So there’s this long period between a recommendation and scaleup. We don’t want that to happen with the next generation of tools that are so powerful, especially one like gene drive. Charity founder Varsha Venugopal on using village gossip to get kids their critical immunisations Rob Wiblin: It seems like parents don’t want their kids to die, and the benefits of vaccinations are really big, and the parents don’t object to vaccinations. What’s going on that’s causing the parents not to really prioritise, as a really high priority, making sure that their kids get the later vaccinations? Maybe they don’t appreciate that one in every 20 or something of the children who don’t get vaccinated end up getting these horrible illnesses and suffering severe consequences? Varsha Venugopal: So, to put this in context for India, we have about 26 million children born every year and we have 16 million children that are completing the vaccination schedule, so we are talking about the remaining 10 million. So one of the big reasons we think that they drop off, and there has been some research, is to do with time inconsistency argument. I think it’s something referred to by Banerjee and Duflo in their book Poor Economics as well. This whole idea that I value my present very differently from the way I value my future, right? So even when I’m making decisions on exercising or gym, it’s all something I’d rather postpone for all these other myriad intrusions on my time in the present. And I think some of that is what’s at play for these parents as well. So caregivers may miss their appointments for various reasons. They could just be forgetting about them. They may not have the right information to accurately understand the benefits. We do know sometimes they don’t know how many appointments they need to come for, or they just don’t want to take a day off work. They don’t want to deal with a crying child. They don’t want to take the bus to go somewhere for vaccinations. So all reasons which possibly by small nudges in the margins could be addressed. Rob Wiblin: So their best guess is, it’s just the basic thing that we can all relate to of, you put something off and you put something off because it’s kind of a pain in the ass to do, and no particular day feels like the day that you want to go and do the vaccination when it requires crossing town, or dealing with your baby not wanting to get an injection. Varsha Venugopal: Yes, and this is exactly where it’s really important to reinforce that it’s very different from COVID vaccine hesitancy or other kinds of vaccine hesitancy that may exist somewhere, which is possibly far more complicated and involves issues of trust. I mean, here we know, because they’ve come in for the birth dose and at least one other dose, that they broadly trust the vaccines and believe it’s a public good — but for a myriad of reasons are then dropping off because of just being overwhelmed with their daily lives. So the gossip intervention is, in the original J-PAL study, the surveyors were sent to a random set of 17 households. And they asked several questions, but the main question is, “If there was a fair in town, who is most likely to tell you about it?” We then run an algorithm to identify the top gossips, or community influencers, and the surveyors then go back to these top influencers and recruit them as immunisation ambassadors. What we did as a result of COVID was pivot to a remote model, which already brings down the costs significantly, in that we call up a random set of households to identify these influencers and then call these influencers to recruit them as ambassadors. So compared to the SMS model, this model at the moment is still not as mature, and we’re still iterating to find the most cost-effective way of doing this. Rob Wiblin: So you identify a village. It seems like a significant fraction of India’s population is living in villages of 500 to 1,000 people or so. Is that kind of where you’re targeting? Varsha Venugopal: That’s the standard ratio we are using. Yes. Rob Wiblin: Yeah, OK. And so you call up a random bunch of households and you say, “Who would tell you about an event?” And then you pick maybe the person who is most often mentioned and get their phone number and then call them up. What do you say on the phone? It seems like an odd call to get. Varsha Venugopal: We have some standard language we use. And in fact, we are iterating with some of this, which also seems to increase the success rates. But broadly, it is saying that, “You have been identified as a community influencer, and we would like you to be an immunisation ambassador. There is no monetary cost involved in this. It is a voluntary exercise. We would be sending you regular reminders on immunisation camps. Are you willing to be an ambassador?” In more than 90% of the cases, when people were contacted randomly, they were able to give us not only the name of an influencer, but also their phone number. And then in more than 95% of the cases, the influencers agreed to be ambassadors. The challenge we had was reaching these influencers. Possibly because they are influencers, they’re not around most of the time. It’s unclear. That’s just a supposition. But that’s the bit we are now iterating, to try and call at different times of the day or different days of the week to get to them. The other bit was, are these ambassadors then sharing the information? So one thing we did early on was give them a phone number that they could then share with parents and ask the parents to give a missed call to enroll into the SMS reminders programme. And I think we had more than 30 people sign up in the first week, which again, gives us some confidence that the ambassadors are sharing this information and people are receiving it and somehow responding to it. Rob Wiblin: Yeah. So just trying to build a better model of exactly what this looks like in my head. Do you know, are these like cool people, or very extroverted people, or high-status people in the village, like maybe someone who’s involved in local politics? Do you know what kind of person we’re talking about? Varsha Venugopal: That’s a great question. So in the J-PAL study, they looked at both trusted people as well as community influencers and see the interaction effects, and it turns out it is the community influencer bit which seems to be causing the biggest impact. Other than the fact that we know about 80% of our influencers are women, which is kind of similar to the J-PAL study, we are very curious to better understand the motivations of the ambassadors and why they take it on. And that’s the kind of thing you don’t get in an RCT, so that’s something we want to go there and interrogate further. Rob Wiblin: Yeah, makes sense. Just to have a picture in the head of the magnitude of the benefit, you’re saying it increases vaccination rates by 10 percentage points — and that’s about 27%, so huge magnitude. How large is the SMS reminder effect again, separately? Varsha Venugopal: Somewhere between 3% and 7% Rob Wiblin: OK. So I’m guessing the SMS reminders are much cheaper because you’re just sending texts, and I guess they’re probably easier to scale as well, because you just have to get a bunch of phone numbers and then stick them in some piece of software that sends out text messages. On the other hand, it seems like the ambassadors have a larger effect size, maybe a more robust result in the RCT. How do you kind of trade off this easiness of scalability and low cost? Varsha Venugopal: Yeah, so we know SMS reminders, you’re absolutely right, are easier to scale up. In fact, in one of the two states we work in, Maharashtra, we already have a memorandum of understanding with the state government which allows us to rapidly scale up across the state. Having said that, we are quite aware that it’s the ambassador programme which seems to be having some of these large effect sizes, and we’re really excited to overlay it on top of the SMS reminders programme. Rachel Glennerster on solving tough global problems by creating the right incentives for innovation From episode #189 – Rachel Glennerster on why we still don’t have vaccines that could save millions Rachel Glennerster: Market shaping is the idea of leveraging the power of markets to incentivise innovators to generate the innovations we need to solve some of the world’s biggest problems. Now, markets have a lot of benefits: they help incentivise innovation, they develop products that people actually want to use, they aggregate information. But they also get things wrong. They’re not very good at solving things where there are externalities, like climate change. So the idea is to get the best of both worlds, and incentivise markets to respond to the things where we really need them to respond. Luisa Rodriguez: Learning about this actually made me think of this blog post that a friend shared with me recently. The blog post was by Jacob Trefethen, and it’s called “10 technologies that won’t exist in five years.” Jacob’s the science policy grantmaker at Open Philanthropy, so he has kind of a sense of where science funding goes, and how hard it is to make progress in some areas. And I found the blog post really moving and really tragic, and it feels relevant, so I wanted to share a couple of the technologies that he thinks won’t exist. One relates to tuberculosis, which kills 1.5 million people a year, which is twice as many people as die of malaria each year. And according to Jacob, a tuberculosis vaccine that worked in adults is totally achievable. But he thinks, for basically no good reasons, we won’t have one in five years — nothing related to the science will prevent us from getting there. Similarly, strep A kills 500,000 people a year, and again, a vaccine is achievable. Hepatitis C kills 300,000 people a year — again, a vaccine is achievable. It’s really horrible. I definitely had the intuition that diseases like this, that are this deadly, we’re not creating the vaccines or treatments we need because the science is too hard. But when I learn that’s not the case, it just feels really unacceptable. Are these the kinds of market failures you’re talking about, and if so, can you talk me through why they happen? Rachel Glennerster: Yeah, those are exactly the kinds of targets we need to do market shaping for. Actually, tuberculosis was one of the diseases that brought me into thinking about this many, many years ago, when I first started working on trying to accelerate vaccines. And it took way, way, way too long for us to get a malaria vaccine compared to the huge death toll. There’s a number of market failures going on. There’s a number of reasons people don’t invest in doing innovations for these kinds of problems. One of them is that there’s a lot of pressure to keep prices down once you invent something, especially if it’s really important for poor people. So it’s a time-inconsistency problem, we call it: you would be willing to pay for something in advance, but once it comes, there’s a big pressure to keep prices down. And the patent system that we have to incentivise people for innovation is really, really inefficient. It works by allowing companies to keep prices high, to get a reward and a return on their investment. But if you’re designing something for really poor people, nobody’s going to buy it when the price is high. So you have a small quantity at a high price, and you make your money back, and then later you reduce your price and get it to lots of people. Or a vaccine: people are willing to pay a lot to get a drug that you’re about to die of a disease. But a vaccine, you need a low price to get to lots of people. That’s not how our patent system is designed to work. It works by charging a high price to a small number of people. And that’s just really inefficient. Luisa Rodriguez: OK, let’s talk through a few specific examples of pull funding mechanisms. One example is an advance market commitment, or an AMC. How do AMCs work? Rachel Glennerster: So an advance market commitment is something done in advance. You commit in advance to either buy or subsidise a set quantity of a product at a set price. So you’re defining the market, because the market is price times quantity; you’re committing in advance; and you’re defining what it is that you want to buy. You’re setting out, “I will buy it if you meet these criteria.” And it’s legally binding, which is really important, because there are all sorts of statements about, “I will do this green thing or that green thing,” but this is a legally binding commitment. Luisa Rodriguez: Yeah, that does seem really important. I guess, trying to be sceptical, it seems really hard for an institution to make funding plans when making advance market commitments. Maybe you know how much you’ll end up having to pay, but you don’t really know when. And I guess you don’t know, in the sense that it’s possible no one produces the output — so maybe you’ll pay zero, or maybe you’ll pay some capped amount. Is that not a dealbreaker for some funders? If I imagine government bureaucrats sitting and thinking about their budget, I wouldn’t expect them to be able to say, “Maybe $2 billion in two to five years.” Can they do that? Rachel Glennerster: It certainly makes life harder. That is absolutely clear. And it’s funny, because one of the benefits is that you don’t have to pay if it doesn’t get invented. But actually that turns out to be quite complicated for governments to cope with. So one thing is, philanthropists often have a bunch of assets of stock in the company that they founded or whatever, and then they’re paying out money in their foundation every year. But they can actually use that stock of assets and put it to work now by stimulating innovation, by saying, “I’m willing to sell some of that stock and spend more in any given year if somebody comes up with this thing.” So I think philanthropists, actually, it’s not hard to do it because they have a stock of assets that they’re going to run down slowly. But if there’s a brilliant opportunity, they should be willing to pay more that year, right? And now let’s turn to governments, because governments are important, and I think will probably do more of this. Let’s start with the logical economic argument, which is: if this brilliant, cost-effective opportunity comes along suddenly, you should be willing to borrow for it, right? You should just say, “Wow, we suddenly have a way to reduce climate change that is incredibly cost effective and way more cost effective than other things we’ve been doing. You know what? We should suddenly just do it, and we should be willing to borrow for it” — because you set the price so that you will only be buying it if it is cost effective. So there’s no risk. You know it’s going to be cost effective. So you should just borrow for something that has a high benefit whenever it hits, and you can just borrow whenever it hits. That’s a logical economic argument. Is that actually how budgets work? Well, if it’s a big enough thing then maybe, yes. And with pneumococcal, the Ministry of Finance in the UK was really behind this, and they kind of understood it and they realised they can just do this. But you’re right, that normally the way bureaucracies work is, “I’m the group that has got $X million to spend this year on this particular thing, and my budget is capped over many years, and so I don’t want to put money aside for something that might not happen.” So part of it is, yes, it makes things complicated. I think the solution is you sort of have to get people at a higher level who understand that this is worth it and kind of say, we’ll just increase your budget if this hits, because it’s worth it. Part of it is working through some of these details, like when we got so close to doing an AMC for malaria — way back, way back — and actually Larry Summers figured out a way to put it in the tax code, and it was in the Clinton budget, which then never passed. If the budget passed, I can’t remember exactly which year it was, it would have happened. So, you put sufficiently clever people… Luisa Rodriguez: They figure it out. Rachel Glennerster: They’ll figure out a way to make it work in the government structure. So, yes, I have spent many hours talking to bureaucrats about their challenges of how do they make this work. And some of the solutions are, “I’m just going to put some of my budget every year into a fund, and it’ll just accumulate.” I don’t think that’s the most efficient thing; that’s sometimes the only way that they’re going to do it. But it’s part of the challenge of what we’re doing is to work through these mechanisms. Again, we did it with pneumococcal: it’s possible to do it, governments did it. But we want to work through it again, because these things do come up. But there’s billions of gain on the table. It’s worth figuring out the financing mechanisms. Luisa Rodriguez: Right, right. I’m actually curious to go back to the case of the malaria vaccine. What is the basic story? It sounds like something like the AMC was designed, and then people got on board, and it was literally just the failure of not getting the budget through? Rachel Glennerster: So the basic story is, Michael Kremer, my husband, was working on different ways to promote innovation, and then got into the neglected tropical diseases as an important case study. And we worked together on designing something for doing the work of what the benefits were of accelerating — again, it wasn’t that you come up with a malaria vaccine; it was that you would accelerate getting a malaria vaccine. We did malaria, HIV, and tuberculosis. We actually wrote a book on how you would design this, called Strong Medicine, and it went through a lot of the details of how you would design it. There was an expert working group that was set up with the Center for Global Development, and that was the one that ended up saying let’s start with pneumococcal. We’d been hoping that it would be a far-off challenge rather than a kind of near-case challenge. But the idea is, well, let’s try it out with pneumococcal. If it works, we’ll then go on to malaria or something. And it worked, but kind of things had moved on. We never got to the malaria vaccine. I mean, we were arguing for it to be malaria or HIV from the beginning, but it was kind of like, we haven’t done one of these before, let’s do something and get a win. And of course, there were companies who thought that they were close on the pneumococcal, so they were lobbying that it should be pneumococcal. Interestingly, and annoyingly, the pharmaceutical companies were all saying, “We’re not putting much resources into a malaria vaccine. Not because we couldn’t make money from it. No, god forbid we would be influenced by money or returns or anything — we’re not evil people. We would work on it if it was scientifically possible, but it’s too hard a science problem.” Luisa Rodriguez: Wow. Rachel Glennerster: And I can see why they do that for PR purposes, but no, you are meant to take into account your shareholder value. Of course you shouldn’t be working on something, of course you wouldn’t be working on something that wouldn’t make you money. And of course, if there was a malaria vaccine, there would be huge pressure on you to keep the price down. This was in an era when there was huge pressure on people to keep the price of HIV drugs down — which saved a lot of lives, but also taught pharmaceutical companies, “For god’s sake, don’t invent anything that’s useful for poor people, because you will have your price driven down.” So in a sense, it was the right thing to do for HIV, but it had these negative consequences. And our response to this, and remember, we were doing this in the midst of all of this HIV battle, and we were saying, “Great, I’m glad that HIV drugs are getting out to people, but you are sending a terrible signal to pharmaceutical companies about how to do innovation for diseases of relevance to low-income countries. So can we please do this other thing that combines an incentive and gets it out at low price to people?” Like, you don’t have to have this zero or one argument about patents or not patents — which, again, we saw in COVID. That’s just the wrong argument to be having. We can have both. We can have the incentive for innovation, we can reward the innovation — and charge a low price and get it out to people. Sorry to be so passionate, but if we had had a malaria vaccine earlier, so many people’s lives would be so different. Luisa Rodriguez: I think passion is the appropriate reaction. Karen Levy on when governments should pay for programmes instead of NGOs From episode #124 – Karen Levy on fads and misaligned incentives in global development, and scaling deworming to reach hundreds of millions Rob Wiblin: To some degree we’re already talked about this, but a lot of people might have the reaction that the Kenyan government should be funding this, that it’s weird that a nonprofit is involved in coordinating this kind of thing. It seems like there was a test where that disappeared for a year and the programme stopped — it wasn’t as if the government stepped up and paid for it. I guess there’s probably quite complicated, interesting, decision-making institutional reasons why it is the case that it’s not easy to grab a few million dollars and a bunch of staff to take over the role that you were doing. If you walk away, then the programme freezes. Karen Levy: Kenya is a resource-constrained place. I mean, this may sound glib, but it’s kind of like saying, “Well, if poor people just spent more money on stuff, they would be less poor.” They’re poor because they’re poor. And so there are enormous pressures: there are lots of needs that resource allocators within the Kenyan government need to fulfill. Countries like Kenya borrow a lot of money. So I think it’s an overly simplistic mental model to say, “Well, there’s this big pile of money. The government, they’re so rich, why don’t they just spend money on this?” If you think about those funds as taxpayer funds: we’re convinced that the people receiving the deworming drugs shouldn’t pay for them, but that’s their government that we’re talking about. Rob Wiblin: They would be paying for them through the taxes, effectively. Karen Levy: Exactly. So look, this can be taken to extremes. Of course there are things that governments can and should pay for. But in some ways, I almost feel like the amount of effort that it would take to ensure that these funds were allocated and dispersed and delivered and used well, year after year, is frankly — Rob Wiblin: It’s easier just to pay for it. Karen Levy: — we should do that for Kenya. We should just do that for Kenya. And there are other countries, like South Africa, that pay for it themselves. India is paying for most of it themselves. It’s leveraged by a much smaller amount of money that pays for the technical assistance and support around those basic costs. It’s a similar thing, like about blaming poor people for being poor. Kenya is resource constrained. There’s a lot of things they need to spread their resources over, and if this is something that can be easily taken care of by philanthropic funds, I would much rather see that happen. And hopefully there will be a time, not that far from now, where Kenya won’t need to have a deworming programme. Open Philanthropy lead Alexander Berger on declining returns in global health, and finding and funding the most cost-effective interventions From episode #105 – Alexander Berger on improving global health and wellbeing in clear and direct ways Rob Wiblin: It seems like if distributing bed nets is something like 10 times as good as just giving people the equivalent amount of cash, shouldn’t you then be able to get leverage on top of that by lobbying governments to allocate more aid funding to malaria prevention, including distributing bed nets, or doing scientific research into a malaria vaccine, which it seems like there’s a pretty good candidate that’s come out recently that might really help us get rid of malaria completely? Why don’t those, in addition, help you get further leverage and have even more impact? Alexander Berger: You see the issue with infinite regress right? It’s like, “Why can’t you go one layer more meta than that, and advocate for people to…” I think the answer is that in a weird way, the problems of the world actually will just not support giving at that scale in a super cost-effective way. I think this is an interesting point that I wish effective altruists would pay a little bit more attention to. I haven’t done a good job articulating it, so it’s not something that people just necessarily understand, but I think the GiveWell top charities actually set a very, very, very high bar in terms of spending at large, large scale. One way to put it would be like, there’s the Institute for Health Metrics and Evaluation at the University of Washington. They compiled the Global Burden of Disease report to try to say how many life years are lost to every cause of death around the world every year. And they estimate that there’s something like 2.5 billion DALYs lost to all causes every year. This is off the top of my head, so I could be wrong, but I think GiveWell thinks they can save a disability-adjusted life year for something like $50. If you were trying to spend just a billion dollars a year — which is 3% of the NIH budget, less than 0.3% of US philanthropic dollars every year — on stuff that’s as cost effective as that, then you would need to be reducing total global life years lost from all causes everywhere by just under 1%. I think that if you sit with that number, that’s just really, really high. Amongst other things, it just shows that if you were trying to do that at a scale 100 times bigger, you literally couldn’t because you would have already solved all health problems. I don’t know where the curve is of declining marginal returns, but I would guess it sets in pretty steeply before even 10 times bigger than that. I think people sometimes underestimate the size of the opportunities when they think we can make a leveraged play that could be 10 times better. Maybe an individual donor could, but Open Phil will need to eventually be giving away a billion dollars a year, maybe more. That is actually not the relevant benchmark for us. We’re giving at a scale where it has to be able to absorb more resources. South Asian air quality I think is a really interesting example. You know this, but your listeners might not: we have these three criteria that we use for picking causes: importance, tractability, and neglectedness. And on importance, I think this is a crazy case. I mentioned earlier IHME, who produces the Global Burden of Disease report. They estimate that almost 3% of all life years lost to all causes globally are lost due to air pollution in India. And that’s a mix of indoor smoke from cooking, and outdoor air pollution from burning coal, from cars, from burning fuel crops. And in some ways, I think it’s appropriate when you hear numbers like that to be sceptical, and to say, “Should I really believe these?” And you have to rely on some social science to get figures like that; you can’t really run randomised controlled trials where you expose people to a lifetime of air pollution, thankfully. So, as with all social science literature, I think there’s some reasonable concern or a question of, is the magnitude that we’re getting right? But I don’t think it’s going to make you want to downweight that by a lot — maybe it’s a factor of two or something, and you’re starting from such a high base that the importance just ends up continuing to be huge. And then on the neglectedness criterion, it gets a really small amount of philanthropy right now. The best report we’ve seen on this I think estimated something like $7 million per year of funding for air quality work in India. For something that’s causing so much of all of the health problems in the world, that’s a trivial, trivial fraction. And a lot of those funders are actually motivated by climate. Climate will get you some of the benefits that you care about in air pollution, but they can come apart, so I think there’s a lot more to be done there. The last criterion honestly is the weak point on this one: tractability is a challenge. Funding in India as a foreign foundation is hard, and frankly getting harder. And air pollution has a bunch of different causes, and there’s no one silver bullet policy that’s like, if you could just get the legislature to pass this, then you would be OK. There are a bunch of things, from trying to encourage modern stove usage, to getting coal power plants to adopt these units that remove small particulate matter from the air, to changing the emissions standard for new vehicles: these all seem like they would have a reasonable shot at this. If we did something like quadruple the funding in the field, we would only need to reduce air pollution in India by something like 1% relative to the counterfactual in order for that to be more cost effective than the GiveWell top charities. And I really don’t think that’s trivial — I think that’s actually a hard, high bar — but I think it’s probably doable. GiveWell researcher James Snowden on making funding decisions with tricky moral weights From episode #37 – Finding the best charity requires estimating the unknowable. Here’s how GiveWell tries to do that, according to researcher James Snowden James Snowden: It turns out which charities you think are most cost effective, it’s just extremely sensitive to what your moral weights are on different kinds of good. Rob Wiblin: Yeah, let’s talk about some of these moral weights, as opposed to empirical weights. How do you go about estimating the value of these different benefits and harms? James Snowden: I think there’s a really high-level question about whose values should we be using to make our best guesses. One, you might think the beneficiaries are the values we should care about. Secondly, maybe our staff are the best people to be making these decisions. Thirdly, our donors. Fourthly, maybe some idealised consensus of all people — what do other major prioritisation organisations do, or how did the World Health Organization make this tradeoff? That kind of thing. At the moment we do it for our own staff, but trying to take into account these other various data points. We have asked our donors; we’ve surveyed a few of our donors and generally got the impression that most people just don’t have numbers in mind — which is just consistent with my understanding of how people think; not many people are walking around with this figure in their mind of how they would value preventing a death compared to increasing somebody’s consumption. This is not true for some of our donors. Some of our really engaged donors, I really encourage them to go to our spreadsheet and just input their own values and see how that changes the answer. I think it’s quite unlikely that we’ll end up investing a lot in trying to elicit our donors’ weights, because we just don’t think most people have a good answer there. We have invested a lot in trying to understand how other organisations make these decisions. There’s a few different ways you can think about this. The World Health Organization has these thresholds for what they consider a highly cost-effective intervention, in terms of dollars per DALY — so anything which is more than three times GDP per capita, per disability-adjusted life year, is not a cost-effective use of money in that country; and anything which is between one and two would be considered pretty cost-effective. There’s another question about what does that actually mean: pretty cost-effective versus not cost-effective? I’m not entirely sure whether those thresholds are really used in practice, because it just depends on your budget, I guess. That’s one input. You can also look at stated preference surveys or revealed preference surveys. There’s a very famous study, I forget the name now, but it looked at different occupations and tried to come to an estimate of how much people were willing to pay to avoid a micromort, a very small chance of death. You could use that to think about how to trade off consumption. Rob Wiblin: Against survival. James Snowden: Against survival, yeah. One big problem we’ve seen is a lot of this literature is based on developed countries. That’s why we’re doing this work with IDInsight to go into an area of Kenya where we think the people there are relatively similar to our typical beneficiaries, although that’s obviously still a huge generalisation. IDInsight are going in there and asking them questions about how they would make these tradeoffs, and we hope that that might be useful in thinking about what the people we’re trying to help actually value. I think one problem with this is it’s pretty hard to ask most people these questions, and particularly to try and get people thinking in a pretty consequentialist or utilitarian mindset. I think that’s something that’s likely to be quite challenging. We’ll see how that goes. Rob Wiblin: Just to make it more concrete, you’re weighing up things like increasing someone’s income versus the risks of them dying, versus the risk of maybe their child dying, versus perhaps direct suffering from disease? Those are the kind of things that you’re weighing up and trying to say, 10 of this is worth two of that? James Snowden: Yeah, that’s right. One of the most controversial ones would be, what’s the value of preventing a death at a particular age relative to preventing the death at another age? This is quite interesting, because I think the global health community generally has a way of thinking about this, which is a disability-adjusted life year or the quality-adjusted life year. We actually don’t use that anymore as a baseline for our own moral weights, and the reasoning for that is that it turned out that a lot of our staff actually value preventing the death of an adult more than they value preventing the death of a very young child. So we stopped using that, and now we all have our own ethical systems that we use to prioritise lives at different ages. I can speak to my own one, very briefly, which is based on the time-relative interest account, which is an ethical theory by Jeff McMahan. Andreas Mogensen has a really nice paper on his website where he formalises this. The basic idea is that there are two things that are bad about dying — there are lots of things that are bad about dying — but one is how many years of life you lose and the other is how much interest you have in those future years of life. This helps account for my intuition that I don’t feel particularly strongly that preventing the death of a very, very newborn child might actually not be so much more valuable than preventing the death of an adult. You take the number of expected years of life left and you multiply it by this pretty subjective factor, which accounts for: Does this person have cognitive function? Can they make plans? Are they a functioning agent in the world? You multiply those two things together and you end up with a roughly log normal distribution over age. The death of a very young child is something I would value relatively less. I think my peak value is the death of an eight-year-old, is where I end up. Other people have really different [values]. Rob Wiblin: That’s when it’s the worst for them to die. James Snowden: That’s when it’s the worst on my values, yes. Rob Wiblin: Another controversial moral issue that you guys encounter is suicide versus involuntary death. Have you discussed that a whole lot because you were supporting the anti-pesticide suicide group? James Snowden: This is probably the most controversial of any of our moral weights. Rob Wiblin: I know this issue is controversial with my friends whenever it comes up. James Snowden: It’s really something that sparks people — and I think fairly; it’s a very difficult thing to talk about neutrally. Maybe I’ll give a bit of background about the organisation that we ended up making a grant to, and that’ll maybe give a bit of perspective on which we can think about this moral question. We made a grant of about $1.3 million to the Centre for Pesticide Suicide Prevention. I won’t go through the full evidence base, but basically the idea is they’re going to go into India and Nepal and work out which pesticides people are using to attempt suicide, and what the case fatality of those different pesticides are. For context, we think that about 800,000 people a year die from suicide, and somewhere between 100,000 and 300,000 of them are through pesticide. The data’s that bad: it’s quite hard to narrow down closer than that. This is a very common method of suicide, particularly in agricultural communities in developing countries, and particularly in South Asia and Southeast Asia. There’s two possible mechanisms here. One is that you’re making a common and lethal form of suicide more difficult to access, and that might reduce the amount that people have this very easily available method of suicide in front of them. And then the other possible mechanism — we’re not really sure which of these is more likely — is people are still attempting suicide but they’re doing it with less toxic pesticides, or pesticides which are less likely to kill them. They generally have a higher chance of getting through that period and hopefully go on to make a full recovery. I think there’s a big thorny argument here about how valuable it is to prevent a suicide. You’re not treating these people — you’re not treating mental health disorders; you’re essentially just preventing them dying. I think this is an interesting one, because two of our staff place zero weight on preventing a suicide through means restriction. I think there is a decent argument there. A lot of it depends on what your definition is of a life worth living, and I’m not sure anyone has a really good answer for that. There is also a lot of empirical information which is relevant, so I think this highlights the nature of these conversations. A lot of times it’s going to be dominated by your ethical view, but other times it really does seem like there’s actually evidence we can go out into the world and find out which should change our minds. Lucia Coulter on the “hits-based” approach to funding global health and development projects From episode #175 – Lucia Coulter on preventing lead poisoning for $1.66 per child Rob Wiblin: I asked for audience questions for this interview, and we got a very cheeky one from a previous guest of the show, James Snowden, who used to work at GiveWell and now works at Open Philanthropy. He actually declined to fund you guys back in 2020 or 2021 when he was working at GiveWell, but I think he’s given you a grant more recently, working at Open Philanthropy. He asked: “GiveWell declined to fund you when you were first starting out, but more recently you’ve gotten this Open Philanthropy grant. Why was that? And what do you think they missed?” Lucia Coulter: Thanks, James. So when GiveWell were first looking into lead as an area, they were prioritising their time and using quite broad heuristics — like how confident can they be in paint as an important source of exposure, or where can absorb a lot of funding soon? And at the time, they weren’t convinced, and they didn’t have a [cost-effectiveness analysis] that they felt confident in, but they planned to come back and look at it more. And then James moved to Open Phil, and lead kind of moved with him as an area. Now Open Philanthropy is thinking about the lead space more holistically, and is excited about LEEP. I did ask him what had changed. He mentioned that he’d updated on the health and mortality effects with the new evidence, and also that he’d previously underestimated how tractable the work would be, and also how many countries we would be able to get to. I think he said he was partly sceptical about our relative inexperience in the field and also our track record, but that he’d now endorse a less conservative attitude towards that. Rob Wiblin: A concern that I’ve had about the GiveWell mindset — which is looking for opportunities to do good where you can really demonstrate that they’re having impact, and you can kind of repeat the same thing again and again that’s been shown to work in the past — a worry that I’ve had from the beginning is just that it could push people against systematic solutions, very high-level solutions, where the effect isn’t that predictable, but the impact when you succeed might just be really enormous. And this is a common critique or a common concern of the GiveWell mindset that they’re very well aware of — and I think, actually, that they’ve been trying to resolve by funding more things like LEEP over the years. But if you think about it, the UK didn’t solve waterborne diseases by putting chlorine dispensers in each house: it solved it by having government build enormous sewage systems and enormous piping systems that brought clean water to everyone simultaneously. And there might just be no real alternative to having governments at a massive level do the things that only governments can do, or that only city governments can do, at least. And likewise here: we didn’t solve the problem of people getting exposed to lead in the air through leaded gasoline by giving people face masks or telling them to change their behaviour. We just said, “No, we’re getting rid of it. It’s gone; we’re banning it.” I guess you’re doing something that’s a bit of a hybrid model, where you’ve got this replicable model where you can demonstrate that this worked in Malawi, and it’s probably going to work in these future countries the same way. But you’re leveraging the power of the state to just fix problems somewhat by force, saying, “We’re now just not doing leaded paint anymore, and if you do this we’re going to send you to prison. Ultimately that’s where this will end. So no more leaded paint, please.” But I do worry that the fact that GiveWell didn’t fund this I think maybe does show a weakness in the research methodology, or it shows that it’s not going to be able to identify — or at least the mindset most strictly applied is not always going to be able to identify — really amazing, high-expected-value interventions. Because things that are extremely high expected value will often have too much uncertainty, too much that’s unmeasurable about what they’re going to do. Do you have any reaction to that? Lucia Coulter: Yeah, I think that sounds right. I think it’s just much harder to identify interventions like that, like health policy, regulatory interventions. It’s much harder to identify with that level of certainty the expected impact. And maybe GiveWell is not best placed to be doing that with their methodology, or maybe it’s something that they could expand their scope into, but there’s been very little of that type of thing that they’ve looked into or recommended — especially now that James has left; I think that was something that James is very interested in. Rob Wiblin: Yeah. So I might get some of this wrong, but I think GiveWell has paid attention to this critique, and has been open to funding more of these things. We talked about that with Elie Hassenfeld earlier in the year. So James Snowden has gone to Open Philanthropy, which has a bit more of this high-risk, high-return mindset — I think they call it “hits-based giving” mindset — where they’re going to make 100 grants thinking that one of them is going to hit it out of the park and pay for the entire portfolio. And he’s maybe taking more of that approach on the global health and wellbeing side at Open Philanthropy. So this is maybe exactly in his wheelhouse now. If there’s any entrepreneurs out there who are thinking of starting charities to focus on issues in the developing world, it would not surprise me if the highest impact opportunities are exactly the kind of thing that you’re doing: it’s improving policy in neglected areas, where you can just have an enormous impact by getting the policy settings right, by getting government to take responsibility for things that government ought to be taking responsibility for. You’re nodding your head. Same intuition? Lucia Coulter: That sounds right, yeah. Rachel Glennerster on whether it’s better to fix problems in education with small-scale interventions or systemic reforms From episode #189 – Rachel Glennerster on why we still don’t have vaccines that could save millions Luisa Rodriguez: So it seems plausible that focusing on specific interventions is meaningfully worse than encouraging governments to make systemic reforms to their education systems. Is that something that you worry about? Rachel Glennerster: It’s certainly the case that there are big-picture systemic issues that cause problems in education. It’s a big deal. So we were talking about the benefits of teaching at the right level, and the fact that curricula is sometimes very overly ambitious. That’s a problem, and it’d be great if people fixed that problem. There’s a lot of RCT evidence that basically individual programmes that help you get around that fact, like Teaching at the Right Level, are really beneficial. And you might think the right thing to do is, let’s go fix the underlying problem. I’m all for that, but it’s hard, and there’s all sorts of political economy reasons why things are designed for the top of the class. So if you can do that, great. If you can get an education system to respond to the needs of low-income children — that’s more flexible, that’s evidence-based — great on you. But I’m not going to wait for that to do the other things. Because let’s be clear: every single thing that we’re recommending here has been tested in isolation and worked in isolation. So it’s empirically not the case that you need systems reform to do these things. They have been tested and they have proved to work when they were the only thing that was done. Because a lot of these come from RCTs. Not all of them, but a lot of them do. And that’s what RCTs do: they just move one thing, they don’t move everything else. So systems change is important. Having systems that actually care about low-income kids is really important. But you can’t just magic that, you can’t put in that as a recommendation: “Care about low-income kids.” And there’s a political economy of that. It is true that there’s probably some benefits of doing a bunch of things. There may be benefits of doing some things together. Like the structured pedagogy is an example of that: they actually tie together a whole bunch of things that move together. But don’t wait until you have it perfect. Get on with doing the things that you can do. GiveDirectly cofounder Paul Niehaus on why it’s so important to give aid recipients a choice in how they spend their money *From episode #169 – Paul Niehaus on whether cash transfers cause economic growth, and keeping theft to acceptable levels Luisa Rodriguez: I guess the key theme here seems to be GiveDirectly just values recipient choice super highly. My sense is that one of the core values of GiveDirectly is giving the recipients of charitable efforts a choice around how they use donated resources. Is that right? And if so, why does that seem so important? Paul Niehaus: That’s right, and there are two reasons. One is instrumental, which is that we often think that actually people who are there on the ground living their own life are going to have more insight and more perspective on how to use money than we as outsiders would. Not always, but often. So the housing, the metal roofs, I think are a good example of that for me personally. I have a PhD in development economics, so I sort of feel comfortable saying that as far as expertise goes in what to do about poverty, I’m as well trained as anybody. I never would have guessed that so many people wanted to replace their thatch roof with a metal roof. And when we saw so many people doing that, and looked into it to try to understand why they were doing that, you learn interesting things. Like if you have a thatch roof, you have to replace it or repair it every so often and that costs money; if you have a metal roof, it lasts longer, so you save that money — so it ends up looking like a pretty good long-term investment to build a metal roof. Or you can use a metal roof to collect clean drinking water from the rainwater, and you don’t have to travel a long way to a lake or a river, and you’re less likely to get sick from things that are in the groundwater. Things like that. That’s all stuff that was complete news to me as an outsider, but completely obvious to the people living on the ground. So I think it’s partly in order to be able to tap into that local information. But I do also think this may vary a little bit, depending on the donor. I personally put a lot of value in people’s ability to make choices per se. I would say that in my description of an ethically good world would be one in which a lot of people have more autonomy and more self-determination than they do now — even if they do sometimes make mistakes, or use it in ways that I disagree with to some extent. I put a lot of intrinsic value in that. Luisa Rodriguez: Let’s talk about the empirical evidence a bit more. Unconditional cash transfers have been studied empirically many times in a range of contexts, as you’ve noted. Can you summarise what we know about the return on investment recipients get? Paul Niehaus: There are certainly cases you can pick out where a large share of the money got invested in some sort of asset and business got better, and the return on capital in that business was maybe 20% per year, or 30%, or even up to 50%. So there are certainly cases like that, where in a very narrow financial sense, we can say that we’ve learned from this that people have access to high-return investments, and it’s great that we’re able to finance them. But I would actually push back a little bit about that instinct of trying to kind of put everything into one number — because I think once you get into the reality of how diverse life is, it’s too complicated for that. Luisa Rodriguez: Yeah, it must be frustrating. It seems like there are all these randomised control trials on a bunch of interventions like this, including unconditional cash transfers. And many of them, in some ways, have it easy. They’re tracking the effect of bed nets on malaria, and it’s pretty easy to measure malaria — at least relative to how difficult it seems to be to measure how people spend money, when there are dozens, hundreds, in some sense an infinite number of potential options for them. And how do you measure the benefit they get from that? Paul Niehaus: And there are all of these knock-on things, like you see impacts on mental health, or recently there have been papers that found reductions in rates of suicide or rates of all-cause mortality. So do you also think about that? Is that a separate thing that I need to value separately, or is that the result of all these other things that I was just talking about? So I think it’s really hard. And actually, I think that the way economists have traditionally thought about it, which to me makes more sense, is to say we’re actually going to think of this as like the numéraire, right? The value to giving someone a dollar is a dollar. And then we’re going to use that as a reference point in a comparison to other things — and say, relative to that, how great is a bed net, or deworming, or any of these other things we want to think about? Luisa Rodriguez: I see. And at least part of the thinking behind GiveDirectly is that, in surprisingly many cases, the value of giving someone something that you’ve decided in advance might be best for them, that costs a dollar, might actually be less than a dollar — because people have such different needs, and it’s hard for us living in other countries to predict them. Paul Niehaus: That’s the thing we want to watch out for. And the issue there is that in the aid or philanthropic system, there isn’t any built-in feedback loop that prevents us from doing that, right? So think about it: by comparison to a commercial business, if I’m trying to sell something for a dollar, and people value it at less than a dollar, then nobody buys it — and I learn quickly that this isn’t working; I don’t have product-market fit. In the philanthropic world, if it costs you a dollar to produce something, and people value it at less than a dollar, they’re going to say, “Oh, thank you. This is better than nothing.” You don’t get that feedback loop of people telling you that there’s something better that you could have done with your money. So we have to be very intentional about building that in. Sarah Eustis-Guthrie on whether more charities should scale back or shut down, and aligning incentives with beneficiaries From episode #207 – Sarah Eustis-Guthrie on why she shut down her charity, and why more founders should follow her lead Luisa Rodriguez: It strikes me that charities seem to scale back or shut down at potentially a much lower rate than businesses — and that seems bad. It seems like businesses have some incentives that don’t always lead to incredible outcomes, but they are probably tracking something like whether they provide value. And if charities aren’t shutting down nearly as often, that might suggest something about too many existing that aren’t providing much value. Does that seem true to you? Sarah Eustis-Guthrie: I think you’re pointing to the most important factor here, which is that structurally charities are built in such a way that your expectation on priors would be that a lot of them would just be doing stuff that’s not very useful. Because the difference in the way a business works is, at least in an ideal case, a business provides a product or a service to their consumers. If that product or service isn’t very good, then unless there’s a monopoly or something wonky going on, consumers stop purchasing that product and that business goes out of business. But what happens is that instead of it being dual when it comes to charities, it’s actually this triangle: on one point you have the charity, on one point you have its beneficiaries, but then on another point you have the donor. And in some ways, the donor ends up having most of the power — because if the donor is the one that’s giving the money to make this programme happen, and you’re the charity, and you’re looking at your beneficiaries and you’re looking at your donor, you’re saying, “If the donor doesn’t like what we’re doing, the programme can’t happen. But if the beneficiaries don’t like what’s going on, as long as the donor keeps liking this, this can keep happening.” And to be clear, I’m not saying that charity founders are sitting there saying, “Bwahahaha, I’m going to do bad things for my beneficiaries.” I think nearly all charity founders are really well intentioned and are trying to make the world better. But you end up in this space where you are structurally incentivised to make your donors as happy as possible — and then you’re only really incentivised to make sure you’re helping the beneficiaries insofar as the donor cares about it. So maybe the donor wants to see photos of happy-seeming beneficiaries; maybe they want to see studies of this programme is really effective; maybe they want to see ongoing monitoring and evaluation data — but depending on what they demand, things could look very different on the ground. And of course, organisations can also demand these things, but in general, I think funders are often the ones who have the most leverage. Luisa Rodriguez: It seems like one solution is focusing more on outcomes for beneficiaries, and really trying to figure out some incentive structure that means that the actual thing that you’re measuring and holding yourself accountable to as an organisation is beneficiaries getting the thing that you think is valuable for them, and that they hopefully say is valuable for them. But there’s been this huge backlash against the randomista movement, which tries to forefront outcomes by conducting randomised control trials to better measure the actual impacts of global health and development programmes. And you pointed out in the article that you cowrote with Ben — which is in Asterisk and which we’ll link to — that there’s this kind of new move toward what’s called “trust-based philanthropy,” which is a term I actually hadn’t heard before. Can you talk about what trust-based philanthropy is, and maybe what the best case for it is? Sarah Eustis-Guthrie: Absolutely. So trust-based philanthropy is one of the biggest forces in philanthropy right now. If you go to a conference on philanthropy, people will use trust-based philanthropy language. It’s gotten a lot of acclaim, and I think it’s very much in the philanthropy water in a way that I did not fully understand until I both started talking to some donors and also doing some research on this. Trust-based philanthropy burst into wider awareness in the US around some of the racial justice protests a couple years ago. And it’s very much rooted in this idea of how we need to equalise the balance of power between donors and nonprofits and the communities they serve. And it points to this really important thing, which is that it seems wrong for donors to have a disproportionate amount of power, and if they’re trying to improve things in the communities that they serve, that’s only going to happen when there’s this partnership of equals among donors and nonprofits and their beneficiaries. So what does that actually mean in practice? Well, that’s a trillion-dollar question, and I will say I get the sense that in practice sometimes this just means using different words to describe the same thing happening as it was before. But a lot of the shift that you see in practice is about reducing these burdensome reporting requirements that charities have to do for their donors. The fact of the matter is that a lot of donors — especially if it’s money from government development agencies rather than from private donors — will require this ridiculously long list of reporting requirements. So they want you to report on, “This is exactly how we used the budget, and this is exactly what we did for this programme.” And maybe your listeners and you are thinking that that makes sense. But somehow, many organisations managed to do this in the most aggravating and time-consuming possible way. I remember talking to another charity about a big grant that they’d gotten, and they said, “Maybe you should apply for this grant. But just heads up, we had to hire a part-time operational person for a full year solely to do the reporting requirements for this one grant.” Because they would make them fill out all these forms, and if you didn’t do it in exactly the right way, they would take away your money. So reporting requirements are a huge headache for a lot of nonprofits. And I think this is actually less known in many effective altruist charities because many EA donors actually operate in a way that’s very much like the trust-based philanthropy approach — which I suppose I haven’t even gotten to; it’s just saying that maybe we should make these reporting requirements less onerous. And I fully endorse many aspects of that, because I think there’s a lot of ridiculous time that’s spent on reporting that ultimately just makes the funders feel better. Like, “We’re doing our due diligence to get you to do it in this detailed way.” But really, for years nonprofits have been saying, “Oh my gosh, can you just make this easier for me?” — but maybe not even saying that to donors, because they’re afraid to say that to donors, because then maybe the donors will take away their money. So lessening reporting requirements is a big part of this. And some donors have gone very far in that direction. MacKenzie Scott has become one of the biggest philanthropists in the world. I don’t even think many people are aware of the full extent of her grantmaking. But over the last four years, she has given I think more than $17 billion to nonprofits. Probably that number will be outdated by the time that you release this podcast, because she’s just dispersing ginormous amounts of money. And she doesn’t have a huge staff. I don’t know actually how many staff she has, but if you would do that under the normal reporting requirements, you would require the world’s most ginormous foundation. But what she’s done is she has been giving this money out with absolutely zero strings attached, at least to my knowledge. And often she doesn’t even talk to the nonprofits ahead of time. I don’t know how this actually works, but I think it’s something along the lines of like the head of the Boys and Girls Club in Boston gets an email that says, “Hey, MacKenzie Scott wants to give you a million dollars, no strings attached. It’ll be in your bank account next Tuesday.” So in some ways, this is a big win, because charities have to spend less time on these reporting requirements; they have to spend less time trying to figure out exactly what does this donor want from me, cultivating these relationships. But I also have worries about this, because I think in some ways this approach conflates the nonprofits with the beneficiaries themselves. Because trust-based philanthropy in some ways boils down to trusting nonprofits to do what’s best for the beneficiaries. And you can think that people who run nonprofits are really well intentioned and great, and you should be friendly and make their lives easier, but also think that maybe it’s better for everyone involved if we have some ways of making sure that that money is actually helping the beneficiaries lead better lives. Luisa Rodriguez: Yeah, it does sound really nice in theory. What is your best guess at how to solve this? How to give power to beneficiaries in particular? Sarah Eustis-Guthrie: I don’t have a five-point plan to fix philanthropy. Luisa Rodriguez: Yeah, it’s a tall order. Sarah Eustis-Guthrie: But I do think there’s a couple ways of going about this. First of all, if you’re really focused on, “I just want to empower beneficiaries as much as possible,” if that’s your biggest priority, I think the best thing you can do is direct cash transfers. And I think that organisations like GiveDirectly have set it up in a way that mostly gets rid of concerns about corruption, that makes sure it’s going to folks who are the neediest around the world. I am eagerly waiting for the moment in which people who are really focused on empowering beneficiaries start adopting cash as one really promising approach, because I think it’s such an exciting approach. When I think about ways to empower beneficiaries, one approach is maybe they’re on a committee that helps decide where some of these charities’ funds go, and they have these long discussions. That seems maybe vaguely useful, but it also seems really easy for that to end up in a failure mode where it doesn’t actually make that much of a difference. On the other hand, if you just send people money, they can do whatever they want with that money. And a lot of studies have shown that people in extreme poverty tend to really reliably use that money on stuff that meaningfully improves their lives. But I think that if you care about how we can empower people to do what they want to improve their own lives, then just giving it to them directly and bypassing the nonprofits entirely — not to put myself out of a job — but that is maybe just the simplest, best possible way. Give them the money directly. Luisa Rodriguez: Yeah. I mean, I’m a huge fan of GiveDirectly and of that intervention. I guess you’ve pointed at one of the potential ramifications, which is putting a bunch of charities out of a job. Do you have an idea for how this happens? If you actually follow this to the logical conclusion, you’d probably see a bunch of charities shut down. And what does that actually look like that doesn’t feel horrible to a bunch of people actually at charities working and hoping to do good work? Sarah Eustis-Guthrie: I think in my ideal world, there would be a lot more money that’s directly going to cash. And what that means is not that everyone who works at a charity is out of a job, but that instead they can be working on the interventions that are most impactful to help improve those people’s lives. So I think in my world, what that looks like is not that there are zero charities — I mean, unless we’re living in our ideal world — but that there’s charities that have different focus areas, and that maybe have a different approach to monitoring and evaluation. And I think that that’s a really exciting future for both beneficiaries and the people working at charities — because 99.9% of people who are working at charities are there because they want to help people. So to me, this idea that we can help more people better, that is in fact a win-win. I think that reframing it from shutting down to shifting programme focus areas, or shifting people from one programme to another programme, is a really helpful way of thinking about this. One interesting example for this is New Incentives, which provides conditional cash transfers to help incentivise immunisations. You may have heard of New Incentives because right now it’s doing really well: it’s one of GiveWell’s top charities; it’s widely recommended. But what you might not know is that when New Incentives originally started, they were doing a very different programme. Their founder was really excited about conditional cash transfers to help reduce poverty. She was originally focused on, I think, cash transfers to prevent mother-to-child HIV transmission. So she was working in Nigeria, and they were working in a bunch of clinics. And it was going decently, but they realised things looked different than they originally thought, and they were really not going to be able to scale up with this programme. So they’d been operating for a couple of years, it had been going kind of well. I wasn’t involved at all, but my sense is they faced this fork in the road of: should we keep going with this programme that we think is decent, or should we try to pivot to something else? I can imagine them sitting in the room trying to figure it out, like, “Wow, this is a really tough decision. We have a lot of staff who might be affected. What’s going to happen?” What they ended up saying is, “We are here to make a cost-effective, impactful charity and we want to focus on the most useful thing.” So they ended up pivoting to these cash transfers for immunisation. They ran an RCT, it turned out really well, and then they massively scaled. So they went from a small number of employees to I think they now have more than 3,000 employees in Nigeria. Luisa Rodriguez: Holy crap! I did not know that. Sarah Eustis-Guthrie: So pivoting is what allowed them to unlock their potential as an organisation. So I think that this is a lot about how we frame this conversation. Are we framing this as, “More people should do the painful and unpleasant thing of shutting down”? Or do we frame this as, “More people should look at the exciting opportunities of pivoting their programmes to things that can help more people”? Luisa Rodriguez: If you want to learn more about Sarah’s lessons learned, I highly recommend the Asterisk article she coauthored with Ben, “Why we shut down.” It’s so, so good. And there are also some juicy quotes from people in the field who refused to even be named because they were so controversial. It’s just a really good read. James Tibenderana on why we need loads better data to harness the power of AI to eradicate malaria From episode #129 – Dr James Tibenderana on the state of the art in malaria control and elimination James Tibenderana: Malaria is heterogeneous. The distribution — especially now, as we’ve seen the last two decades of success — we’ve sort of controlled the malaria that is not embedded within the context. And now we are having to deal with malaria that is context specific, localised, and has variables that are making it more difficult for you to achieve the continued decline. And then we have hard-to-reach areas, or even have behavioural elements. I mentioned treatment seeking in the private sector. You’ve got to be able to reach the private sector. You’re not going to suddenly change and say, “People should all go into the public sector,” because they made a choice, they want to go into the private sector, right? So what are you going to do about the private sector? And then we’ve talked about hard-to-reach areas, and malaria is a disease of poverty. You may say that there are health facilities, but there are households that are not able to access services, because of either geographical distance or the fact that the health facility may be open at a time when they should be looking after their livelihoods, or looking after their gardens. And they won’t have that time. So there’s some of these barriers that are preventing access, and having the capacity to understand those nuances within the context requires data. It’s not insurmountable; it’s just that you have to have the right data; you need to have the data in terms of the people. You need to have data in terms of the mosquitoes — something called entomological surveillance. And you need to understand whether the parasite is continuing to be susceptible to the drugs, or that you are likely to be identifying the parasite itself when you test for it using a malaria rapid diagnostic test. Because even now we have what is called HPR2 deletions, where the malaria parasite is now deleting a gene that the rapid diagnostic tests are supposed to pick up. Rob Wiblin: Oh wow. That’s savvy. That’s because, I suppose, there’s selective pressure on not being detected, because then you can spread better? James Tibenderana: Yes, selective pressure. So just having that information that allows you to then make the right choices, and really deploy the tools in the right location, the right intensity. We’ve scaled up everywhere, but we still have gaps. You might find a situation where there is something going on and you need to either react with a better net (for example, a pyrethroid PBO net), or you may have to react with more intense community case management (for example, integrated community case management), because you may have seen an upsurge starting and you really want to make sure it’s kept down. Or you have a situation where there is genuinely an upsurge taking place and you really want to stop that happening — because the sooner you stomp it down, the less likely you are for mosquitoes to be able to transmit. Rob Wiblin: I guess we got into this topic because you were saying that this work — where you are tracking where is malaria taking off, and using that information in order to shape your strategy — could be incredibly useful, but it’s maybe harder to convince donors to support, or at least donors who are focused on proven interventions to support it. Because it’s harder to say ahead of time exactly what is the cost effectiveness for this. It’s not just a matter of delivering the same treatment to other people in areas with the same malaria prevalence. It could be that it’s incredibly useful some year, or it could be that some year it doesn’t really help your strategy all that much. It’s a bit more of a speculative spend. James Tibenderana: Yes it is. But from a personal viewpoint, and I think from an organisational viewpoint, surveillance and response is a critical intervention if we are to achieve malaria elimination. We already see that in Asia, where surveillance and response is playing a very important role in continuing to identify the last case of malaria where it is and to make sure it’s dealt with. But we’ve not seen that kind of investment in surveillance and response in the control setting where you have higher transmission intensities — because I think because the numbers are so large that in some ways, the trends or the spikes get lost out by the noise. But if one was to say, “If you had additional funding, where would you spend it?”, I think surveillance and response for me is one area that can have potential, if done properly: having the right data and using it. Because you need to have data and then you need to be able to use it at the subnational level — the districts, the provinces — and then at the national level. Looking ahead, it will be difficult to use machine learning, it will be difficult to use AI, without having data. And if in the next decade, one is to envisage machine learning as contributing to some of the decisions, some of the predictions that allow us to be really more savvy at our choices, we are going to need this data. And if we don’t start collecting that data now, when we have tools to use the data, what will we be doing? You need this long-term data and you need the consistent quality that can allow you to then get into decision support tools, and to really optimise your decision making at the moment we are making decisions. We are bringing on board modeling to help with some of our decision making — so all the cost-effectiveness modeling, some of the modeling work that allows you to identify where to put particular interventions and combinations of interventions. I think in the next 10 years, we should be looking to things like machine learning to be able to support that decision making, so that we’re probably more precise and we are more targeted. And that’s going to require data. Rob Wiblin: It seems like the decisions would be things like, “Malaria is more prevalent here or malaria is increasing in prevalence here, so we need to invest more money in that particular location.” Do we need something as sophisticated as machine learning to make these decisions? Is that going to add a lot of value beyond what a human decision-maker can do? James Tibenderana: It will add value, especially as the data elements increase. Because as I’ve stressed, you have the parasite, the vector, and human behaviour. So looking at the mosquito data: the breeding sites, the breeding habits, the human-biting elements. There are situations where the malaria mosquito is changing its habits, so rather than biting at night, some mosquitoes are starting to bite earlier in the evening, or later in the morning. Bringing that data together. The parasite data: what’s happening in terms of some of the genomics, some of the changes that are taking place. And then really the trend data: bringing those data elements together more comprehensively, to allow us not only to anticipate what’s happening now, but also to anticipate what’s coming forward. Climate change, climate variability is going to affect malaria. And the variability is going to increase and become more complex. For seasonal malaria chemoprevention, for example, we are constantly monitoring the start of the rainy season, the duration of the rainy season. In some of our locations, we’ve had to change from four treatments one month apart to five treatments, because the season of malaria transmission with rain is getting a bit longer. And there will be some locations where it is shorter, because of climate variability. And so you’re bringing in climate data as well. So these data elements I think will give us the opportunity to be a bit more precise in some of the choices that we make, but also some of the transitions. Can you switch from indoor residual spraying to nets? Can you switch from seasonal malaria chemoprevention to integrated community case management? What are the switches that we need to be making as it’s coming down? So we continue to maintain a cost-effective approach, and we continue to sustain the downward trend of malaria until we have those game changers — whether it’s gene drive, whether it’s the transmission-blocking vaccine — in which case, the decision making will probably be simplified. Rob Wiblin: Yeah, that makes a tonne of sense. Lucia Coulter on rapidly scaling a light-touch intervention to more countries From episode #175 – Lucia Coulter on preventing lead poisoning for $1.66 per child Rob Wiblin: Let’s talk about the Lead Exposure Elimination Project. We’ll get to more recent events later, but can you start by telling us this amazing story of some of the very first things that you did? What did LEEP set out to attempt in Malawi? Lucia Coulter: We were helping the Malawi Bureau of Standards — supporting them with testing capacity, updating their regulation to make it more enforceable, and making sure that the right samples were being collected. It needs to be coloured paints, because they’re much more likely to be leaded than white paints. And the default globally for anyone doing monitoring of a paint industry would be to just collect white paints, so you’d naturally miss lead in that. So that’s a bit of a process. And then industry typically is unlikely to reformulate until they feel that there will be consequences. Rob Wiblin: Did you call them up and say… ? Lucia Coulter: We spoke to them before we did the study, and we said, “You’re aware of lead as an issue in paint?” and that sort of thing. Broadly, they said that they were aware of it, but that maybe some other manufacturers were doing it, but not them. So then we come back with the results and we say, “There is a lot of lead in your paint.” And they’re like, “Ah, OK” — and at that point we offer our support. We can help in quite a few ways. We have a paint technologist who’s absolutely amazing, who can give really detailed technical support with the switch. We can help them find suppliers of the non-lead raw materials, which can sometimes be a barrier because their usual suppliers might not necessarily supply these non-lead alternatives. We also offer to retest their paints for them so they can be confident that they’re lead-free and that sort of thing. Some of them engaged, some of them didn’t really. Some of them said to us, like, “I don’t really know. Does the Malawi Bureau of Standards really have testing capacity? Do we really have to do this?” Rob Wiblin: Wow, that’s ballsy to be saying, “OK sure, you busted us, but we don’t think that you have the testing capacity to check again, officially.” Lucia Coulter: Yeah, exactly. Rob Wiblin: Interesting. Were any of them mortified? Sounds like no. Lucia Coulter: No, not that was apparent to me. I don’t know how they were feeling internally. In a professional context, you might not really let on your real feelings about it. I think that from the perspective of a manufacturer, they’re only putting in a small amount of lead: 1% intuitively doesn’t feel like that much, right? Rob Wiblin: It’s just a splash of lead. Just a little bit. Lucia Coulter: Just a tad. So I think it’s not intuitive how bad that could be. One way of communicating it is that if you had a little sugar sachet — the type that you would get at a cafe to put in your coffee — if that was filled with lead dust, and you sprinkled it across an area the size of an American football field, that level of lead loading would be sufficient to cause lead poisoning if a child spent time in that environment. So a very, very small amount of lead can have these really toxic effects. And that’s not intuitive; that’s pretty surprising. So I think that’s probably how people think about it. But often manufacturers, in some of our experience, will move really quickly. Sometimes days after we show them the results, they’ve ordered their non-lead alternative ingredients. Rob Wiblin: What’s happened since then? What countries have you expanded to, and what’s your experience been? Lucia Coulter: So since then we have expanded to 17 countries. Most of them are in Africa — Zimbabwe, Madagascar, Sierra Leone, Niger, a number of others — and then a few outside of Africa as well: Pakistan, Bolivia, Uzbekistan. Basically we prioritise countries based on the expected burden of lead poisoning from paint. Obviously, it’s pretty difficult to estimate. Also on neglectedness: are there any other actors doing anything there? And also on tractability — the classic INT. Rob Wiblin: I’ve heard of it. What goes into the calculation trying to figure out the burden of leaded paint? Lucia Coulter: A big part is population size. If it’s a big country, it’s more likely there’s more lead poisoning from paint. Anything we know about the size of the paint market, use of paint in the country; anything we know about whether there’s likely to be a lot of lead paint on the market. Rob Wiblin: Do some countries use more paint than others? Are some countries big on paint and others are not? Lucia Coulter: Yeah, in some countries a lot of the population live in rural areas where there’s just not much painting going on — usually less economically developed countries. But in those countries, there might be very rapid paint market growth, so they might actually be quite important. Rob Wiblin: Yeah. I guess you want to ideally get a country right before an explosion of urbanisation or construction. Lucia Coulter: Yeah, exactly. Rob Wiblin: So where are you at in the different countries? You’ve gone from you did one country three years ago, now you’re operating in 17. It’s a big increase. Have you been hiring hand over fist? Lucia Coulter: Our team is nine, so it’s a small team. I think it highlights the fact that it’s a relatively light-touch intervention: it’s really the civil servants in government that are doing most of the work, and we’re there to support — we’re there to help them overcome whatever barriers come up. So where are we at in the various countries? In all of them we’ve now established good collaborative relationships with the relevant government authority. In 11 of them we’ve completed paint studies, and in eight of them we have commitments from the relevant government authority to either introduce regulation or to start implementing regulation if it already exists. And in four of the countries, we already have reports that paint manufacturers representing over half of the market share have started switching to lead-free. Rob Wiblin: Has the responsiveness of the government been at the same level as in Malawi? Has anywhere dragged their feet? Lucia Coulter: Not really. I think when we first started LEEP, we were expecting a big part of our role to be advocacy, to be convincing governments that this is an important issue. But that hasn’t really been the case. It’s more that there’s sometimes a lack of awareness to begin with, and then we can communicate about the problem and do the paint study to bring that awareness. Then they’re convinced; they’re on board. Obviously no one wants lead paint in their country. So the barriers are more about the relevant part of government having limited capacity, limited time. Rob Wiblin: Yeah, I would have thought that would be an issue: that you’d contact someone and they say, “Sorry, I’m busy and my hands are full. I agree with you, but I can’t act on this.” Does anyone say, “I don’t have a budget to deal with it; I don’t have budget to do the testing,” or, “I don’t feel like I have the authority to act”? Lucia Coulter: Yeah, the budget thing comes up a lot, and that’s where we can help. So we’ll offer support with funding the paint study, and actually we often do the study in collaboration with a government partner. And then we offer support with funding for multistakeholder meetings. That’s often a big step to getting progress on an issue: the lead government authority will need to get other parts of government involved, and they’ll need to engage industry and bring them along. So they need to have all these meetings, and sometimes it’s to agree on draft laws and that sort of thing. So we can help provide funding for the meetings. And then we can also help with the testing capacity to some extent: we can help with accessing international testing or improving their internal capacity. And then we can also do a lot of the industry outreach side of it as well. So if we can offer all of that, then the budgetary constraints are usually not a problem. Rob Wiblin: I see. Is it an issue that in many of these countries, they presumably don’t have great testing for lead? They maybe don’t have the best scientific facilities to handle this? Lucia Coulter: Yeah, I think probably most low- and middle-income countries don’t have existing ability to test for lead in things. The approved method is using a lab-based technique. The machine costs at least $30,000 to $140,000. And then there’s also consumables, there’s also training that’s needed. So that can be something that limits their ability to enforce a regulation. Rob Wiblin: But I think you were testing samples for $14 or $15 or something overseas. Why don’t they just mail them out to a lab overseas? It sounds like it costs peanuts. Lucia Coulter: Yeah, that’s a good option, and it’s something that we help get set up. And we provide training on how to prepare the samples so that they can be sent abroad. We’ll even cover the costs of the international analysis for a period of time while they’re getting set up. It’s a very cost-effective option, but I think a lot of regulatory authorities would ideally like to have their own capacity. Rob Wiblin: Just as a matter of principle, that they don’t like stuff that comes from outside? They feel a bit embarrassed, maybe, that they’re having to ship this to another country? Lucia Coulter: I think it’s been mentioned before that sometimes they don’t really like the idea of sending money abroad. Like, “Why can’t we do this in our own country or locally?” I think also, if you’re a regulatory authority whose job is to run a lab and test for things, it’s quite nice to just be able to do that and not have to rely on externals. Rob Wiblin: I see. It could be more convenient to just be able to send it to the people in the basement rather than mail it overseas, from a logistical point of view. Lucia Coulter: Yeah. And mailing paint can be a bit tricky sometimes. You can’t actually ship wet paint because it’s flammable, so you have to prepare dry samples. But then sometimes mailing companies think that you can’t ship dry samples. So it’s a little bit inconvenient, but it’s pretty easy overall. Rob Wiblin: I guess maybe one reason that leaded paint is still an issue is that it’s usually produced locally, right? It’s this kind of commodity business, where it’s an extremely heavy item that’s sold at relatively low cost and it’s also relatively low tech. So most countries are producing their own paint, or even most cities have their own paint supplier. And the fact that it’s at that medium scale means that it hasn’t been possible to just identify a handful of places in the world where paint is being produced and tell them to stop. You have to go kind of city by city, country by country. Lucia Coulter: Yeah, exactly. In the vast majority of the countries that we work in, the vast majority of the market is locally produced paint. Rob Wiblin: How big a deal is it that you offer this paint technician or paint formula specialist? I have to say that is one of my favourite examples of a super-high-impact career: I don’t think anyone would ever have guessed that you should go into paint formulation and then this is going to allow you to save the lives of tens of thousands of children. But that’s how it’s turned out. What’s the guy’s name again? Lucia Coulter: He’s called Phil. Phil Green. He’s amazing. Rob Wiblin: How important is that to the paint companies? Is it often an impediment to them that they just don’t know how to make yellow paint or red paint without his help? Lucia Coulter: It depends on the paint company. Some have internal chemists that are really experienced with formulation, but others don’t. And it’s not actually a simple thing. It’s not like a one-for-one replacement; it’s more like baking a cake — where if you substitute an ingredient, you have to adjust other things to make sure it has all the right properties. So having someone who has a lot of experience with both different types of formulations, but also the local context, and what suppliers are available, and what raw materials you can actually get where, and how that could work, and how reliable all the different raw material suppliers are, is just absolutely amazing. So I think that’s a very high-impact career. Rob Wiblin: I guess inasmuch as the companies are reluctant to push forward, after they speak to Phil they can kind of be persuaded that maybe this isn’t going to be as much of a pain in the ass as they were expecting. Lucia Coulter: Yeah, exactly. Rob Wiblin: It sounds like this has all been suspiciously smooth. You go in, you find that there’s lead in the paint, you call out the government. The government mostly follows up, you hassle them a little bit, you pay for them to have some meetings, pay for them to run some more tests. What part of the whole process here takes the most person time, or the most money? Is there any stage that is kind of a hassle for you? Lucia Coulter: I think the bit that takes the most time is the part between getting the government on board, establishing that relationship, doing the paint study, identifying that there’s a lot of lead in paint — but then actually getting regulation in place. And that’s just because it’s a complex process: it varies by country, it involves a lot of different steps, complicating factors can come up. And also the civil servants that we’re working with are juggling a lot of plates, so that can take a long period of time. Our approach is just to try and be in really close contact — lots of calls, emails, WhatsApp messages, visits — to just make sure that we are as responsive as possible to whatever part of that process that we can help with. And that varies a lot by country, but that period takes up a lot of time, basically. Karen Levy on why pre-policy plans are so great at aligning perspectives From episode #124 – Karen Levy on fads and misaligned incentives in global development, and scaling deworming to reach hundreds of millions Rob Wiblin: I know one methodological innovation that you’re excited about at the moment are ‘pre-policy plans,’ which I actually hadn’t heard of until you mentioned them. Could you explain for the audience what they are? Karen Levy: The idea of a pre-policy plan is inspired by a pre-analysis plan — which we know is common, but becoming even more and more typical in the social sciences. And this idea came up when I was doing some work with an organisation in Botswana called Young 1ove. Young 1ove is led by Noam Angrist and Moitshepi Matsheng, an incredible team of people that they lead in Botswana. Several years ago, they were replicating a study that had been done in Kenya many years earlier by Pascaline Dupas that provided relative risk information to adolescents in order to help them make better decisions about their sexual partners and reduce their vulnerability to HIV infection. And the idea being that, traditionally the options given to adolescent girls are: have sex and die, or be abstinent. I mean, these are not realistic choices for people. And I think the real brilliance behind Pascaline’s study was empowering girls with information that allowed them to manage risk: explaining to them that older men are riskier and more likely to be HIV positive, and so by choosing to have relationships with younger partners, they could reduce their risk of HIV. And so that was the intervention. Rob Wiblin: And they decided to test it again in this new context and this new era. And it’s just as well that they did. Do you want to explain why? Karen Levy: Exactly. Now of course, while the test was happening, we had no idea if it was going to work or not. And the Young 1ove team had done a really extraordinary job in developing relationships with government partners. They were doing this programme in schools. They had gotten the Ministry of Education all excited about it. And ironically, you often worry that if you find evidence that something works, no one’s going to take it up and run with it. But there’s also the converse problem: if you find something doesn’t work, you don’t want people to scale it up, right? So in discussing this dynamic, before the results of the study came out, we decided to work with the government partners to put together what we called a ‘pre-policy plan.’ The idea was to game out in advance, “What will we do if the results are X? What will we do if the results are Y?” And the reason that we did that is, when you’re thinking about a hypothetical future, it’s very easy to assume that you’re going to be rational — that you’re going to follow the evidence. But the reality is, when you’re standing there later and you’ve got some data [that it didn’t work], there is this enormous temptation to say, “Oh, but you know, it would have worked if it had just been different,” or vice versa. So it was creating a space in which there was nothing on the line, and everyone could really think about in theory what they would do. When you create an environment where everyone cares about evidence and we want to be evidence driven, it’s very easy to get people to say, “Of course we won’t scale this up if it doesn’t work, and of course we will scale it up if it does.” And then sure enough, when the results came in and they were, very sadly, disappointing, it was much easier to connect people to that mindset that they had been in beforehand. You know, people really liked the programme. They wanted it to work. It was an incredibly charismatic group of people doing this. And so everyone’s first reaction was, “Oh, but let’s do it anyway.” But when we looked back at that pre-policy plan, it really helped. Rob Wiblin: “But we said we wouldn’t.” Karen Levy: Exactly. I think the other thing that’s really useful and important about a pre-policy planning process is that it’s a wonderful tool for aligning the research and policymaker perspectives. Very often there’s not enough communication upfront. And what this can lead to is evidence not being useful because it’s not available on time, for example, or the results are not applicable to a population that policymakers are actually needing to make policy for. With a pre-policy plan, the idea is to get people to sit down in advance and really line up: “What are the questions we’re asking? What are the outcomes we’re measuring? When are we going to have this evidence?” alongside, “What decisions are being made as policymakers? What choices do they actually have? When and how are resources getting allocated?” So researchers can make their work relevant and responsive to the actual choices policymakers are making and vice versa. Rob Wiblin: Pre-policy plans are figuring out under what conditions would you want to scale a programme up, under what conditions would you hold steady, under what conditions would you cancel it — but ahead of time, before you get biased by what the results you’re getting actually are. It seems like in this case, the results were very clear cut and quite devastating. It’s interesting to me that even in this case, people kind of wanted to push on. It’s so hard to let go of a programme, even when the most likely outcome seems to be that you’re causing harm. It’s possible that having this conversation ahead of time might have made the difference between continuing the programme and not. Karen Levy: Yeah. I think the truth is that most people want to do something. And I can relate to that: I am action-biased, so all else equal, I’d rather do something than nothing. So I understand where it comes from. And in this case, as you say, it was very clear cut. But often it’s not, right? So the other thing that a pre-policy plan can do is force some tough conversations about “How much impact is enough to make this worth doing?” or “What if it’s effective for some subpopulation, but not for another?” These are all questions that are much easier to really think through and interrogate when you don’t know the answer. I think there’s a lot more that we can do in the global health and development space to be thinking about how we would use new information before we get it. Rachel Glennerster on the value we get from doing the right RCTs well From episode #49 – Rachel Glennerster on a year’s worth of education for under a dollar and other ‘best buys’ in development Rob Wiblin: Earlier this year, I spoke with Eva Vivalt about the generalisability of RCTs. And she had this archetypal stylised fact that, at least in her sample, the typical result differs from the average effect found in similar studies so far by about 100%. Which is to say that if all existing studies of an education programme find that it improves test scores by 0.5 standard deviations, the next result in her sample was as likely to be negative or greater than one standard deviation as it is to be between zero and one standard deviations. Do you think that Eva is potentially overstating the implications that this has, and how dire the situation is for external validity of RCTs? What’s your general perspective on this? Rachel Glennerster: I think it’s really important to distinguish the different causes of why a different study might have a different result. Because we take away different conclusions, we act differently depending on the reason for why a different study might have a different result. One reason why a second study might have a different result is the problem didn’t exist there. So then I’m not at all surprised that you have a different finding in another study; it doesn’t worry me at all. A second reason why you might have a different effect in the second study is it’s implemented less well. In the first study, you had 80% takeup, and the second study, you had 20% takeup. You don’t want to just compare the results of the two studies; you would then want to adjust for takeup in those contexts. And the third reason is that people behave really differently in different contexts. And that, in a sense, is the assumption behind saying this is a problem. My reading of the evidence is that actually, most of the variation between studies that we see is either they’re actually implementing a completely different kind of programme — and we don’t have enough studies, so we bundle a whole bunch of things that are completely different together, and it’s no wonder we get different results — or the implementation was really different and bad or really good. And that just tells us we really need to work on implementation. It might tell us really important things about how this is a really hard thing to implement, and that’s a really useful lesson. It’s not really about generalisability. Generalisability for me means that people act differently when they’ve faced the same problem, and they’re given the same incentives, but they respond to those incentives really differently. My reading of the RCT evidence is that actually we get surprisingly similar results, if anything, across different studies. So I think the way we should think about this is: we have very few studies that test exactly the same thing in lots of different contexts; we have quite a lot of studies testing some fundamental underlying principle about human behaviour. The trick is then to take that fundamental principle about human behaviour — which we’ve tested many times and now know to be true — and think about how to implement that in the local context. And that could be about offering lentils in one context and ice cream in another, whatever it is. But those things don’t need to be tested by RCTs. Some of that is about, it is knowing your context, but it’s just basic logistics — and some of the basic logistics we do need to test every single time, but we don’t test with an RCT. You test with good monitoring processes. So it’s like saying, there have been enough studies that I know if I hang a bed net it will work. I’m not going to test that again. I’m going to test: did they hang them up? Rob Wiblin: Which is probably a lot easier to test. Rachel Glennerster: Much, much easier to test. So it’s doing that causal pathway — making sure that you’ve analysed the problem right, that they actually have malaria. If the people have tested, the next step is: if you give malaria bed nets for free, normally you get higher takeup and people are most likely to hang them. But the third thing is, how do we make sure that we have a monitoring system that the malaria bed nets actually get out to people? And also maybe test occasionally that they’re hanging them up — but that doesn’t need an RCT; that’s good monitoring. Rob Wiblin: If you need a lot of trials to establish a stylised fact about how humans behave, how many things have we learned of that kind? It seems like it might be a fairly short list if you needed dozens of studies to figure out that something just recurs in most cultures, and most countries, and most situations. Is there kind of a list of these things that we’ve learned, these underlying principles that we use whenever we develop any programme at all? Rachel Glennerster: Maybe not every programme, but we know that people are very sensitive to pricing convenience in the takeup of healthcare — so never charge for preventative healthcare. Like that’s a pretty big policy conclusion; we’ve got a lot of different studies that point towards that. So in the health field there are a number around this price and convenience type of things. And then in education I think we’ve learned this more general lesson from many, many studies: that it’s not about the inputs; it’s about the teaching, how people teach. And the biggest problem in most developing countries is that the teaching is way above the heads of most of the kids. So they’re in grade two, you’re teaching them at grade six. That’s a pretty fundamental thing. That’s going to come into a lot of different education programmes to use that principle. People respond to incentives… Rob Wiblin: …the price goes up, people buy less… Rachel Glennerster: Yeah, so there’s quite a lot. I’m not going to remember all of them off the top of my head. To do a plug for my old organisation, J-PAL: the last thing I did before I left was bang heads together of the various academics working in an area, and of course they love to put caveats on everything, but to say, what are the main cross-cutting principles that have come out of RCTs in your area? So those are called policy insight notices, and you can go to J-PAL and under each one they will at least list three common cross-cutting principles that they found — so in agriculture, and in health, in education, in governance. So that will give you a list of some of those. For most of them it’s not more than about three, but they are quite general. Rob Wiblin: This potentially makes RCTs look substantially better value if you can come up with these principles that are going to apply to basically every other programme in the area in future. Rachel Glennerster: Well, not every other programme, but — Rob Wiblin: But at least something you might want to consider in most cases. Because I guess sometimes people worry that each RCT is quite expensive, and then to test all of the programmes that we might consider ever running, it’s just prohibitively expensive. But if you can find these underlying principles, it’s of much broader value. Rachel Glennerster: So I think that’s a point that people really, really get confused about: the more academic and abstract the RCT, in some ways the more policy relevant it is. Which sounds really odd and counterintuitive, but an RCT that tries to test one of these fundamental principles of human behaviour is actually much more useful. Because if you’re just testing a package of things, and you can’t really tell an underlying principle from it — you’re testing whether these six things together have an effect — you can’t take that to another context. You can’t learn as much from it as if you go in deliberately saying, “My question is not does this work, but are people price sensitive in this area? Do people use something more if they pay for it?” I’m testing this more general principle: that’s more generalisable and it’s better value for money. Rob Wiblin: So there’s a lot of different potential sources of heterogeneity — like differences in culture or differences in the quality of implementation or the fact that a programme was different in some ways that aren’t really getting picked up in this meta-analysis, and also just random noise of course. Do you think that any of these is underrated as how significant it is or how much variability it creates? Rachel Glennerster: I think implementation quality is something that people don’t take into account enough, but that there is by the kind of programme. If someone is taking a deworming pill, there’s not that much difference in the quality of the pill usually, right? The quality is very easy to measure in, did someone take it? But if you’re talking about a training or graduation programme, then it really matters. So I think we should be looking at, when we do these meta-analyses using our theory to say, is this something where the quality of implementation is going to matter, or is it not? Rob Wiblin: Do you think we potentially have delivered too many programmes where the quality of implementation matters? I guess that can potentially reduce the expected value quite a lot just because there’s a high chance you’re going to screw it up. Rachel Glennerster: So I’m a big fan in general, unless you’ve got really good evidence against this, of doing less complicated programmes, fewer components, and just doing one thing well massively. I think that’s a huge problem, that we try lots and lots and lots of different things. And I don’t mean test lots of things, because we want to test lots of things to find the one really good cost-effective thing — and then we should scale that up massively. And we don’t do that enough. Economist Mushtaq Khan on really drilling down into why “context matters” for development work From episode #111 – Mushtaq Khan on using institutional economics to predict effective government reforms Mushtaq Khan: You find that actually a policy that worked very well in one country is disastrous in another country, and different countries which have successfully developed have done so with quite different policies. And that’s a puzzle, but the puzzle becomes much more explicable when you look at this interaction: that actually the policies that work, the institutions that work, the rules that work depend on the capabilities, interests, and power of the players and how they implement and follow those policies. Because no policy can be enforced on people who don’t want to follow it. The idea that a policy is a black box that the government just announces and everyone starts following it is a total mistake. A government is just one organisation amongst many organisations in society. And there is an interplay going on between governments, political parties, opposition parties, trade unions, churches, mosques, the people, different kinds of agencies and forums: all of them are trying to influence this policy outcome, but also — and this is critical — implementing it on a daily basis. And if the vast majority of your organisations in society are happily violating the rules and not checking each other, then it’s not going to be an implementable policy. And we always make the mistake of thinking that enforcement always comes from above, that there is some kind of police or some kind of enforcement agency sitting and watching everyone. And if you violate rules, they come down on you. What we often forget is that 99% — or at least 90% — of enforcement is happening without that, without what we call “vertical enforcement.” It is happening through horizontal enforcement, by peers at different levels of society. When we say “powerful,” we mean the relevant people at that level of activity who are as powerful as the people violating. Who is powerful depends on the policy being discussed. For some policies, the powerful are people within a village. For other policies, the powerful are big firms. But whoever is violating the rule needs to be checked by other people who are just as powerful as they are. So what we are looking for always are these incremental processes where whatever policy you’re looking at, whichever level of society, you’re looking for rifts between the players, between the powerful at that level, and seeing if some of them might in their own interest support developmental outcomes in a rule-following way. And we promote that. And if you look at the history of how development happened in advanced countries and how a rule of law emerged, it was always through these incremental processes where people at different levels of society in their own interests started to follow rules and impose rules on their peers. And that horizontal process is for some strange reason not adequately examined by economists and institutional economists. We are often much more concerned about these vertical enforcement strategies, which then typically fail. So I think a lot of what I’m saying could be read by people in different ways. Let me boil it down to some very basic things which might focus people’s minds. I think now it’s commonplace in development policy that people will say, “Context matters. One size doesn’t fit all.” Everyone will say these things — from the World Bank, down to the Foreign, Commonwealth & Development Office, down to Oxfam. Everyone will say this. The real question is, you have to ask yourself, “How does context matter? What am I supposed to do about this? What is it about it that matters?” Here is where I think all that I’m talking about has a direct relevance. Context matters in this way: you have a set of plans and policies, or you’re supporting some activities which you think will support human development. To me, that inevitably involves looking at capabilities, organisations, et cetera. Do you think these policies will be implemented? Do you think they can be enforced? This is where context matters. Context matters primarily in the sense that enforcement and implementation vary hugely depending on the interests and capabilities of the players themselves, and much less on the enforcement capability of some supra-agency, which is like the prime minister’s office, or the cabinet, or the police, or the anti-corruption commission, or anything like that. The basic thing you should do, always, is look at the people you are working with, their capabilities, and then ask, “Here is a policy to help them do something better. Will they themselves actually support it?” Not are they saying they will — because everyone will say they will do whatever you ask them to do, because they want the money — but actually, from their past behaviour and their activities, do I think this is something that they will actually be able to implement, enforce, monitor each other as peers, report back, and do the basic capabilities of policy implementation at a higher level exist to monitor and enforce this? If not, you have several options. You can redesign the policy so you have greater confidence it works. You can say, “Actually, policies won’t work here, because there’s some networked problem which is so integrated that I can’t break it.” Then you have to think out of the box about some exit strategy which meets the requirements of your anti-poverty or whatever that you are trying to achieve. Or you have to go back one step before, and ask, “Can I build those capabilities directly? Can I work with people to actually build those capabilities that I’m assuming exist with my policy? I shouldn’t assume they already exist. Maybe I need a policy that is one or two steps behind in terms of its sequencing.” So I think there are ways in which all of this makes a lot of practical sense as a sense-check when you are asking yourself, “Will this work here?” Then of course you can refer to the work that I do and that others do. It’s all available. It’s free to access online. You can read some of the stuff on the SOAS website, or the work of Dani Rodrik; or North, Wallis, and Weingast; or Acemoglu, Johnson, and Robinson; and all the people we have discussed. Not all of it, but those bits of it that are relevant to the particular policy question that you’re addressing. I think that’s the starting point. That’s the starting point for you, as a development practitioner, beginning to refine your questions and getting down to better questions and then answers which you think might make sense there. Then, the final step is, don’t put all your eggs in one basket: experiment with a trial, see how that goes, and only scale up once it’s working. GiveWell cofounder Elie Hassenfeld on contrasting GiveWell’s approach with the subjective wellbeing approach of Happier Lives Institute From episode #153 – Elie Hassenfeld on two big-picture critiques of GiveWell’s approach, and six lessons from their recent work Rob Wiblin: One alternative worldview or general take on how to think about effectively helping people comes from this outfit called the Happier Lives Institute. The distinctive part of their take is that they want to cash out all of the impact of these projects in terms of subjective wellbeing itself — self-assessed subjective wellbeing. So to compare across charities, they use this metric called the WELLBY — and one WELLBY is the value of raising someone’s self-assessed subjective wellbeing by one point on a 10-point scale for one year. If an intervention made me go from rating my life a 5 out of 10 to a 6 out of 10 — and that impact lasted for 12 months — then that would be one WELLBY that had been generated. Their hope is to evaluate charities on that basis, trying in every case to say how many WELLBYs are generated per dollar spent. First off, what do you think of the pros and cons of trying to use improvements in subjective wellbeing per dollar as a measure of cost effectiveness? Elie Hassenfeld: First I think it would be helpful for me to just explain what GiveWell is doing today, which is we cash everything out either in terms of increased ability to consume (i.e. people have more money) or reductions in disability-adjusted life years — some of which are health-related and some are mortality-related. But I very much take the point that subjective wellbeing is an important consideration. We don’t view the two outcomes we use today as the only outcomes that make sense. They’re just the two outcomes that we’ve been able to use to date. I do think over time, as we continue to grow and increase the size of our team, we’ll be in a position to include more factors explicitly in that analysis. I think the pro of subjective wellbeing measures is that it’s one more angle to use to look at the effectiveness of a programme. It seems to me it’s an important one, and I would like us to take it into consideration. I think the downside, or the reasons not to, might be that on one level, I think it can just be harder to measure. A death is very straightforward: we know what has happened. And the measures of subjective wellbeing are squishier in ways that it makes it harder to really know what it is. Also, I think some people might say, “I really value reducing suffering and therefore I choose subjective wellbeing.” I think other people might say, “I think these measures are telling me something that is not part of my ‘view of the good,’ and I don’t want to support that.” That would cause someone to want to leave it out of their calculus and the donations they’re making. In some ideal world, I would love for GiveWell to be able to offer options for donors who have different philosophical perspectives about what they want to achieve. Obviously, GiveWell institutionally also needs to have a view, because there’s funds that come to us directly. But ideally, in the future vision of GiveWell, for people who have subjective wellbeing as their core focus, other moral values, or maybe even a very different tradeoff between increasing income and reducing disability-adjusted life years (or increasing DALYs, maybe, depending on how you think about it), those are programmes we’d like to be able to bring to donors and let them choose. Because we’re not trying to add value by being particularly good philosophically. That’s not part of GiveWell’s comparative advantage. It would be better if we could, where donors want it, allow them to use their own judgements to make decisions. Rob Wiblin: I think to most people, it’s intuitive that it’s more valuable to save the life of someone who feels that they’re really flourishing and is super glad to be alive than it is to save the life of someone who thinks their life is barely worth living, who maybe doesn’t even care that much whether they live or die. It could be useful to use some numbers to make it a bit clearer how this might end up affecting your relative priorities here. If you imagine people scoring their quality of life out of 10, that’s kind of the standard subjective wellbeing scale. Let’s say that we use the number 3 as the number at which someone is rating their existence as neutral, with the good and bad things in their life cancelling out: that’s kind of a typical answer for what people say would be the neutral point for them if they were scoring themselves. If someone is going to report a quality of life of 4 out of 10 for the rest of their lives, then from a wellbeing-adjusted life year, a WELLBY, point of view, then it’s equally valuable to them to prevent them from dying as it is to increase their wellbeing permanently by one point out of 10. That would be from 4 to 5 in this case. On the other hand, if someone reports a quality of life of 5 out of 10, then from a WELLBY point of view, it’s twice as valuable to save their life as to increase their wellbeing permanently by one point — in this case from 5 to 6 — because the difference from 3 to 5 is twice as great as from 5 to 6. The Happier Lives Institute notes that many people in very poor countries — who otherwise might die of malaria in the absence of additional antimalarial bednets — have unsurprisingly pretty challenging lives with plenty of hardship in them. That, as I understand it, suggests that to them it’s more likely to be cost effective to make people’s lives better than to make them longer, all else equal. What do you and GiveWell make of that line of argument? Elie Hassenfeld: I think this is a case where I feel most strongly that I would want to hear from the people themselves in low-income countries about this topic. Because if you kind of draw out this line of reasoning, it leads you to the conclusion that there is a very high proportion of people living in low-income countries who would choose death over continued living, based on their self-reported life satisfaction. That’s a very uncomfortable conclusion, but maybe more importantly, one that is so counterintuitive that I feel the need to follow up on it before accepting it at face value. That may be a somewhat minor point about where you draw the line on the scale, but still, in this case, I think the maybe purely emotional urge I have is to say that doesn’t quite seem like it could be right. Intellectually, I know it could be right — therefore I need to follow up on it, because it’s so inconsistent with my starting point for what people would say. Rob Wiblin: Yeah, it definitely can get uncomfortable or weird. Or maybe if you were surveying people on their subjective wellbeing, and you really said, “If you score yourself a 2, we’re going to take it that you actually mean that you would rather not be alive right now,” then maybe people would reassess. Because an interesting thing is that when you survey people, almost everywhere in the world, even people in serious poverty almost always say that they think their life is better than not existing, and they want to continue surviving and so on. I’ve heard some philosophers say that that kind of intuition that we all have about how great it is to continue existing might be suspicious, because we might have evolved to have that attitude. We necessarily almost have to have evolved to have that attitude, even if our lives are very unpleasant. That kind of bias might affect all of us. But I’m not really too keen to go there, and I feel extremely uncomfortable. If someone says that saving their life is really valuable, I’m inclined to take that at face value and to trust that over some subjective wellbeing survey. Elie Hassenfeld: Right. I think that discomfort is a good starting point, though not an ending point. Certainly something that we are very committed to internally — one of our company values or whatever you want to call it — is truth-seeking. What we mean by that is we’re going to have the hard conversations, and keep digging to try to get the answer that is correct, as far as we can see it. Therefore, in this case, I would say I am very suspicious of philosophising and reaching a conclusion that seems extremely counterintuitive and then running with it. But we’re a place that wants to go deeper and embrace strange conclusions. Or maybe I should say it differently, like: conclusions that seem strange to us today that will not seem strange to us in the future once we’ve spent more time with them and done more research on them. James Tibenderana on whether people actually use antimalarial bed nets for fishing — and why that’s the wrong thing to focus on From episode #129 – Dr James Tibenderana on the state of the art in malaria control and elimination Rob Wiblin: Quite a few people in the audience were curious about the gap between provision of chemoprevention drugs and provision of nets, and actual use. There’s this kind of mainstream concern that you provide nets, or you provide drugs for free, and then people just don’t use them. I guess there’s this really popular meme that somehow got started that you provide people with nets to protect them from mosquitoes and they use them for fishing — which I think has happened in at least some places, but I suspect is probably substantially overstated because it’s a fun story to tell. Do you want to comment on that? How do you measure whether things are actually being used, and to what degree are they being used? James Tibenderana: There is an access and use gap for nets and for some of the other interventions. It’s something we pay particular attention to because we would like to close that gap between access and use, because then at least you’re maximising the impact that you get. It’s important to appreciate that wherever nets are distributed, the majority of the nets are being used. I think WHO tracks that information, we track that information — so all the campaigns will have a component of results measurements that allow us to be able to have a sense in terms of those who have been given the net and those who use the net. And those results measurement systems have improved over time. In the issue of misuse of nets, whether that’s fishing, we’ve got to keep in mind that malaria is a consequence of poverty, and malaria can contribute to inequities. Let’s appreciate that some of these households are really poor, and even the notion that they are actually using these nets for fishing or over their gardens is a sign of the poverty that some of these households are experiencing. I think it is overstated, because even in those households, what we know from all the evidence is that the access to nets is an important driver of use — and so by ensuring this access, you’re actually facilitating the use of them. So yes, from my perspective, I do hear people talking about fishing, et cetera. But when you go into those communities, the majority of people are using those nets, and I think data suggests that the majority of those nets are being used. So sometimes the question is, is it an old net that has been replaced? Is it a new net? I think the fraction of nets that are being misused in that way is probably negligible compared to the nets that are having a positive impact in those communities. And so a question will be, what else is being done in those households that is addressing some of the livelihood issues that they’re experiencing? We know malaria is a consequence, but also the level of poverty itself can contribute to malaria. Rob Wiblin: I think from memory, Against Malaria Foundation has some auditing process, where they go and check whether the nets are actually being used. And again, from memory, I think the rates of use were over 90%. So yes, some people don’t end up using them for one reason or another, possibly they use it for something else. But in terms of measuring the cost effectiveness of what they do, they only do that relative to the nets that are being used — they only estimate the impact they’re going to have based on nets that are actually hung up when they go and check, not based on the number of nets that are distributed. One thing that amuses me about the meme about fishing is, what fraction of all households that are receiving nets are even near a place where they would go fishing or are interested in fishing as a source of food? I think there was one particular case where there was a village that was specifically a fishing village. They all do fishing, that’s how they make their livelihood and they were using the nets for this purpose. But I imagine for the great majority of people receiving nets, fishing is not a key source of income. It’s not the way that they’re getting food in the first place. So it’s hard to believe that this is the first thing that they think of to do, especially if it would involve traveling to somewhere where they can plausibly fish with any meaningful success. It’s a meme that frustrates me, because I suspect that people are into it in part because it’s a clever story. It’s like, “Oh, you thought that the nets would help them, but actually it didn’t.” So it allows you to one-up other people with your level of sophistication and knowledge, even if it’s inaccurate. And also I think it gives people an excuse for not donating, if they can always say, “Well, it’s always the possibility that they’ll use it for something else and then it won’t actually help” — even though 90% of the time it does. I’m not sure whether you share my cynicism. James Tibenderana: No, I do share it. I do share that cynicism, because really it takes away the message from the positive impact that nets have, and also the majority of people who actually use those nets the way they’re intended. That for me, in itself, is also a signal, that is a reminder of the inequities that thrive when malaria is present, and that we should be doing more to ensure that there is access. Karen Levy on working with governments to get big results From episode #124 – Karen Levy on fads and misaligned incentives in global development, and scaling deworming to reach hundreds of millions Rob Wiblin: A common complaint with people who work with governments, in this context and in all contexts, is a really rapid staff turnover — where a programme could be turned upside down because the minister changes. Is that a problem that you ran into? Karen Levy: Yes. And one of the things that I have learned — and that I work with partners on now — is: how do you codify and institutionalise relationships, such that they survive turnover? We all know how important it is to have champions in government, but champions leave. And so you need to find new champions. So relationship building is essential and never ends. But you also have to be thinking from the beginning about how to codify these relationships in a way that is not just an individual thing. So that can be [memoranda of understanding]. It can be membership on existing committees or technical working groups, getting written into annual operating plans, seconding people to ministries so that you have people actually sitting there in the office. These are all ways that relationships can be deeper and longer term and more enduring than you would get by just getting one individual person excited — which is a great way to start, but that needs to then lead to some of these other longer-lasting approaches. Rob Wiblin: Did you ever have to coordinate with people who were in some sense unqualified or maybe out of their depth? And so perhaps weren’t up to what you were hoping they would do? Karen Levy: Well, there were certainly times where I was out of my depth, let me tell you. I think that there is a perception of civil servants or working with governments that it’s slow, nobody cares. I actually have not found that to be the case. I find that by and large, the people that I have met and have worked with in the Kenyan government and in other governments are public servants: they are incredibly dedicated and they are working within really tough constraints. Sure, there’s gaps in capacity that we all have to face and work on together. But I really think that very often those perceptions come from misunderstandings about how decisions are made — who has authority to make certain decisions and why. So we can very easily say, “This person’s not being helpful” or “They’re being obstructionist” — but they’re working within a set of rules that is not of their making. So I think that there are way more opportunities to work constructively and collaboratively with government partners than we allow ourselves, perhaps because of some of those assumptions. I think it’s really important for people to learn more about how governments spend money, how they make decisions, and how they deliver goods and services. I often like to use the university as an example when I talk to people about how you should be thinking about working with the government. So let’s say you wanted to do a project with a university, and you said, “Well, I’m going to go and I’m going to talk to people. I want to talk to a broad selection of people at the university to find out about this project. And I’m going to do that in July.” Well, anyone who’s been to a university, certainly in the US, would say, “But you can’t; nobody’s there in July.” Similarly in governments, there are fiscal years. Talking to somebody right at the end of a fiscal year is not the right time to be thinking about how they’re spending money. The plans for the next year have already been made. The money from the previous year has already been spent. So being cognizant of the cadence of the way that a government works is really important. Another analogy that I think works well with universities is, let’s say you wanted to say, “What does the university think about this?” People ask me that all the time: “What does the Kenyan government think about this?” Well, who do you mean? Shall we ask the president? Shall we convene the cabinet? Governments are these incredibly complex institutions, within which there’s a lot of disagreement. And at the same time there is policy that flows from the top. There are structures, there are annual operating plans, et cetera, that you need to understand how to work with. So I think that understanding the periodicity and cadence of the way that governments work is really important: budgets, election cycles, the time horizon for policy change. And meanwhile, in the development sector, everything works to the timeframe of the donor project cycle: you need to put in your reports to the donor; they’re marching to their cadence. And so working with the government, it’s a big ship: it does not turn on a dime, and you wouldn’t want it to. So really recognising that and embracing it as a pathway to massive scale that you have to work within and alongside is absolutely essential in terms of getting things done. I also think one of the most important functions of a government is a coordinating function — and it’s all too rare to find development partners that are willing to be coordinated. I remember sitting in a meeting of a bunch of partners in the school health space. It was representatives from a bunch of different NGOs, all of whom had grants to coordinate the school health sector. Nobody was there to be coordinated. That’s what we should be letting governments do: let them coordinate us. And that’s hard. Charity founder Leah Utyasheva on how a simple intervention reduced suicide in Sri Lanka by 70% *From episode #22 – Leah Utyasheva on how to massively cut suicide rates in Sri Lanka, and her nonprofit’s plan to do the same around the world Leah Utyasheva: We’re planning to do this work following the very successful intervention in Sri Lanka. So, this is where my colleague, Professor Mike Eddleston has worked for many years. First he worked as a physician treating patients who self-poisoned, trying to save their lives; however, what he found was that this is not a very effective means of saving lives. Because in quite many cases, it could be too late, and people die a day or two after reaching hospital. However, what he found is that if you reduce the access to this means, people do not go on to commit suicide. So, in Sri Lanka the situation was as such that the suicide rate has increased dramatically after the introduction of highly hazardous pesticides into the agriculture — as the result of the green revolution in 1960s. The suicide rate increased from five per 100,000 people to 24 per 100,000 people in 1976, and then peaked at 57 incidences per 100,000 people in 1995. This is a staggering increase in suicide rates. And you can see a direct correlation with the increase in pesticide use. So, when my colleague Professor Michael Eddleston and the pesticide registrar at that time noticed this trend, they thought what kind of intervention could help? So, from 1984 to 2011, there were a lot of pesticides that were banned in Sri Lanka. The suicide rate in Sri Lanka has dropped significantly: from 57 instances to a 100,000 population in 1995, it has dropped now to 17. This is a 70% reduction in suicide rate. So this is a very significant success, and this is the greatest decrease in suicide rate ever seen. Of course, there was a team working on this, but these people saved close to 93,000 lives in several years. Rob Wiblin: Wow, there was 70% reduction in suicide across Sri Lanka as a whole? Leah Utyasheva: Yes, not only pesticide suicides, but 70% reduction in suicide rate at all. Yes. Rob Wiblin: That’s extraordinary, and it lines up just with exactly the dates where these pesticides were withdrawn? Leah Utyasheva: Exactly, yes. Exactly. And these incredibly successful interventions are mirrored in Bangladesh and South Korea where the highly hazardous pesticides were banned and suicide rate has decreased significantly. Rob Wiblin: Has anyone written a paper where they’ve tried to estimate exactly how many lives you’d expect to save, and how much it costs? If indeed it really costs anything, other than the advocacy required by some activists. Leah Utyasheva: Well, of course, introducing this public policy feature will cost quite a lot of money, because first you need to go ahead and estimate which pesticides are used for suicides in this particular country. So, here I must say, there could not be just one list of pesticides that you just decide to ban, and this will work for every country. Each country uses their own brand names. Maybe it’s generic names, maybe it’s different compounds, different mixtures and solutions so that different active ingredients of pesticides are produced. So each country needs to do its own research in terms of what substances are used for suicide. And then, of course there will be some cost of implementing those bans. I must say that it’s important to also monitor what substances are sold in shops, because quite often there is illegal trafficking of pesticides, and some substances that are not legally registered within the country may find their way to farmers’ markets and shops in the country. So all this needs to be counted as a cost of the intervention. However, we estimate that the successful intervention could cost as much as $400 per life saved, and this is what happened in Sri Lanka. And it could be an extremely, extremely effective intervention in terms of cost effectiveness. Rob Wiblin: Is it expensive for you? Because perhaps you have to lobby against the manufacturers of these pesticides? Leah Utyasheva: Right now we’re starting work in two countries, in India and Nepal, and we exactly selected those countries because we think that we could implement a highly cost-effective intervention in those countries. Due to my colleague Professor Michael Eddleston’s long work in the region, we have a good network of pesticide registrars that understand the situation, and that are on board with our goals. And we also have a good network of United Nations experts, who as I mentioned previously, have designed the cost-effective and the best policy interventions to prevent suicide, and ban highly hazardous pesticides. So, due to that, we think that we could be effective in doing this in those two countries. But what we’re planning to do is to first collect information from 20 hospitals in India and 10 hospitals in Nepal, and based on that information we present clear policy choices to decision-makers in these countries. And of course, we’ll work with civil society, and we’ll work with other stakeholders to engage them as well. Rob Wiblin: How do you estimate the probability of successfully getting law reform in a particular country? Leah Utyasheva: It depends on the country, it depends on how ready the civil society is to support our ideas. It depends on how accepting the decision-makers are to the ideas. Rob Wiblin: Have you generally found politicians to be very sympathetic to the cause, and open to banning these pesticides? Leah Utyasheva: Usually I would say yes, because the countries we’re planning to work with, and we work with now, have low capacity, and they have low budgets for this particular intervention. Usually there’s one or two people working in the pesticide registrar office, and they’re overworked, they have little support, they have little knowledge how to use those highly sophisticated tools that the UN develops for them. And they are very eagerly engaged in conversation and in planning on how they could do their work more effectively, and how they can bring more good to the country. Of course, a lot more attention is currently being paid to environmental harms, to occupational exposure, to incidental exposure of children, residues of pesticides in food. This is why I think this issue is so neglected. Because not when so much talk is devoted to pesticide harms, people don’t talk and don’t know about suicide by self-poisoning with pesticides, and people don’t know that this issue could be quite easily resolved with just taking some highly hazardous pesticides away from the market. This is why I think this is, to some extent, is a social justice and equality issue. Because people who are mostly affected by it are poor farmers in low-income communities, who quite often don’t have a voice, and cannot express their own sorrows and their problems. So in many respects it is a call from help, which has so far been unanswered. Rob Wiblin: So your cofounder was initially just trying to help people directly as a doctor in Sri Lanka? Leah Utyasheva: Yes, exactly. Rob Wiblin: And it sounds like they’ve had vastly more impact by working on advocacy and policy reform, like 10,000 times more impact. Leah Utyasheva: Definitely, yes. Rob Wiblin: Interesting. So this is potentially a good example of how you can have a lot more impact by trying to have more systemic change, rather than just going person to person. Leah Utyasheva: Definitely, definitely. Law reform and policy reform are extremely efficient ways of preventing lots of inequality and human rights violations in the world. I worked with decision-makers and policy people all over the world, on many issues. And what I found is that a lot of bad policies, such as for example preventing women with children staying in the shelters, or preventing people who use drugs from buying clean syringes. These are bad policies, but they developed not because of ill will, they are developed because people don’t know better, decision-makers don’t know better. They are negligent about what problems real people face. Usually they come from the same gender, from the same social group — mostly male, mostly well educated, mostly in their middle ages — and they don’t care about vulnerable groups; they don’t care about marginalised groups. However, if they’re told about the effects this particular policy, let’s say, is having on vulnerable and marginalised groups, they could listen. And it would be more effective if they’re told that it will improve the cost effectiveness of their work if they’re shown that their public image will be better viewed by the constituency, or by their boss, and so on. So, in the policy work, it is important to understand that influencing the decision-makers goes a long way if you show them the way to do it, which could be beneficial for them as well. Rob Wiblin: How important was it to have strong evidence from an experiment in order to convince policymakers to change the law, and other people to get on board with this whole project? Leah Utyasheva: That was exactly important, and I think this is why our approach to solving this problem is also unique, because we work with hospitals, with academics, on providing highly precise information, on providing results of randomised control trials that point in this direction. So, I think it is extremely important to monitor, and to provide the best academic studies behind this. Rob Wiblin: Do you think that it was important to have an academic as the founder and leader? Does that give you a lot more credibility? Leah Utyasheva: I think so. We have Professor Eddleston’s entire academic knowledge as a background to our work. He has worked on this for many years, he has researched the best ways, and he knows everything about this issue. So, it gives us leverage to present our views. It gives him the ability to speak the same language with pesticide registrars, with WHO, and with the FAO, with the UN agencies. So I think it’s extremely important to have an academic as a head, definitely. Especially in our case. Karen Levy on working with academics to get the best results on the ground From episode #124 – Karen Levy on fads and misaligned incentives in global development, and scaling deworming to reach hundreds of millions Karen Levy: In academia, you’re trying to find the optimal answer or the optimal solution. But in a policy setting, you’re trying to optimise within constraints — you don’t have an unlimited set of options from which to choose. So the way you would construct a study or the way you would ask a certain set of questions is really very different. The timeframes are really different. What success looks like and how one is rewarded for a study happening well or not — they’re just very, very different. But even though these differences exist, many of the examples that we have of programmes that have scaled successfully have been ones that have been driven by academics who really care about the policy influence of their work. I think the reason that matters is because evidence isn’t just a set of answers that you pull off the shelf. In taking something from a study to scale, all kinds of questions come up about, “What are the underlying mechanisms? Why did we do it that way? If we change that, are we now pulling apart the whole theory of change, or is that still going to work?” I think some of the most exciting examples of times when studies have moved from evidence to scale have been when you have a really committed academic who’s willing to kind of take that journey with policymakers. There is sometimes some tension between what a programme needs from a data and evidence perspective, and what is attractive to an academic. Sometimes you really just want to replicate something, or you need a bunch of descriptive statistics — and this is not necessarily what’s going to attract the attention of a world-class development economist. At the same time, many of those world-class development economists, we’re lucky to have them involved and caring about the impact of their research. So I find it’s really useful and important to have an open conversation with academic partners about what would make it interesting for them. I have found that almost always people are absolutely willing to do the less sexy stuff, if you also allow them to ask some really interesting, innovative questions that will drive their research agenda — and nine times out of 10, that part of it is going to end up leading to new insights that affect your work anyway. So the marginal cost of adding another module to a survey, or trying a slightly different version of an intervention, doing that can create the space for intellectual engagement from top thinkers, and then brings those people to the table for all of the other things that you need. So I think we should have those conversations. James Tibenderana on the value of working with local researchers From episode #129 – Dr James Tibenderana on the state of the art in malaria control and elimination James Tibenderana: It’s very important to have local research institutions, and researchers with the capacity to identify research needs. And to participate and really lead on research — whether it’s operational research, or implementation research, clinical trials — a variety of research is really important. And we’ve seen it for seasonal malaria chemoprevention: one of the reasons why the research on the efficacy of seasonal malaria chemoprevention was very quickly adopted in the Sahel was because a lot of those studies were done by national researchers. So as the results were available, they were able to link up with their counterparts in the ministries of health to speed up the process of the adoption, so the adoption process was quite rapid and straightforward. One of the things we did in Uganda was we had regular meetings between researchers and ministry officials. In one of these meetings, there was this molecular scientist that was describing the genomics of mosquitoes and insecticide resistance. You had the decision-maker in government, who was sitting there and listening. And after this presentation, which was saying, “In these parts of Uganda, the mosquitoes are resistant to this. They have these molecular genetics…”, he looked at the scientist and he said, “So what are you saying? Are you saying that I need to buy an insecticide for that mosquito here? Another insecticide for that mosquito there and there? And yet I have money to buy one insecticide. So can you make me understand how I’m going to be able to achieve my objective with one insecticide?” And you had this conversation between the science and the implementation. And it was a really lively conversation. So you want to be able to have those conversations at a country level. You want to be able to have those conversations, ideally, in a manner that emphasises trust, in a manner that emphasises evidence and neutrality. My personal experience working in research in different settings is that having institutions with the capacity to do research in their context is a critical success factor for the adoption of some of these interventions, and maintaining their quality. Because once you’ve introduced, you still need the operational research, the monitoring and evaluation to identify what’s going wrong and what’s going well. And then tweak those in the setting to make sure that you continue to maximise and optimise the impact. Lucia Coulter on getting buy-in from both industry and government From episode #175 – Lucia Coulter on preventing lead poisoning for $1.66 per child Rob Wiblin: Is corruption ever a factor here? Maybe testing has been going on, but someone’s been getting paid off to ignore the issue, or not report that there’s lead in paint? Lucia Coulter: Not that we’ve come across. The main thing that’s going on is just this lack of resources, and then sometimes the lack of awareness as well. The other thing is that there isn’t really like a Big Lead Paint lobby. Rob Wiblin: That’s good to know. I don’t know why I’m laughing; there are so many industries where there is a lobby. Lucia Coulter: Yeah. There isn’t a huge amount of industry resistance. And in fact, industry is sometimes supportive of regulation, because they might want to switch to lead-free but they don’t want to be undercut by people that aren’t. So there’s this relationship between regulation and industry, where industry might be pro-regulation if it’s going to be enforced across the board, because then it creates this level playing field. And then also the government authority is more keen on regulation if industry is on board, because they don’t want to do something that’s going to be really difficult to enforce, or something that’s just not going to be feasible. So there’s kind of this bidirectional thing where, if it just happens, everyone’s happy with it. Rob Wiblin: I guess that’s quite different than climate change worries about fossil fuels, because the whole goal is to use less fossil fuels. If you’re an energy company that was completely indifferent between supplying oil versus wind energy, then that might be an analogous situation. But that’s of course just not how it is. And likewise with smoking, the idea is not you’re going to switch to a different cigarette: you’ve just got to stop smoking, and so their product doesn’t get consumed. But so long as people are going to continue consuming the same thing in roughly the same quantity, then regulation is much more straightforward to get up. Lucia Coulter: Yeah, exactly. Rob Wiblin: Have you found any messages in your emails or your meetings that are particularly motivating to policymakers or even to paint suppliers in these countries? Lucia Coulter: With policymakers, I think the local data is really important. So like the paint study data, or the fact that we are offering to do a paint study with them, I think that’s really appealing — that they can generate this data that’s really important for understanding the situation relevant to their role in government. Other messaging that I think is quite important is about the feasibility. As I mentioned, practically, they do really care about this being feasible for industry, because if it’s just going to be near impossible to enforce, then what’s the point? And also I don’t think they want to be causing huge economic harm or something like that. So the feasibility is important messaging. I think another one is the international precedent: the fact that other countries are doing lead paint regulation. It’s the future, basically. Rob Wiblin: People don’t want to be behind the curve. Lucia Coulter: Yeah. And I think also, people don’t necessarily want to do something that’s really unusual. So I think that’s quite helpful. Then the fact that the WHO and the UN Environment Programme are behind this, and there’s a Global Alliance to Eliminate Lead Paint, and there’s the UN model law for lead paint regulation: I think all of that brings a lot of credibility to the issue. But overall, the majority of our work is less about persuasion or advocacy; it’s a lot more about technical assistance. It’s like, they’re on board: how can we help them? So it’s about the technical assistance, providing funding where it’s needed, and that sort of thing. Rob Wiblin: I guess the general observation is that you’re having a very easy time. There’s probably a whole bunch of different reasons. One is that everyone kind of knows that leaded paint is bad, it’s been banned in lots of countries, it’s cheap to do, there’s no one arguing against it, really. All of the authorities internationally agree that this would be a good thing to get rid of. The health data is fairly clear. It’s easy to measure, at least for you to measure the lead in paint. So I guess there was just a crazy amount of low-hanging fruit here to begin with. You’ve just found an extremely easy hill to slide down, and now you just have to implement it. Is that kind of the basic situation? Lucia Coulter: Yeah, that’s pretty much it. Rob Wiblin: So all the hard work is done by finding the initial opportunity, and then I suppose there’s hard work with the follow-through. But that’s kind of the key insight, maybe, is the original thing. Lucia Coulter: Yeah, I think so. Alexander Berger on reasons neartermist work makes sense even by longtermist standards From episode #105 – Alexander Berger on improving global health and wellbeing in clear and direct ways Alexander Berger: I think arguments for global health and wellbeing are first and foremost about the actual opportunities for what you can do. I think you can just actually go out and save a tonne of lives. You can change destructive, harmful public policies so that people can flourish more. You can do so in a way that allows you to get feedback along the way, so that you can improve and don’t just have one shot to get it right. And at the end, you can plausibly look back and say, “Look, the world went differently than it would have counterfactually if I didn’t do this.” I think that is pretty awesome, and pretty compelling. But honestly, if somebody were coming to me and saying, “I buy the longtermist gospel; I’m all on board,” I would be super uninterested in trying to talk that person out of it. I think that is great. I think there’s not enough longtermists in the world. I really think the fact that longtermism is new and small is totally crazy; it should be huge, and it should be a really popular idea. The world is in a crazy place in that people don’t understand and appreciate our position in the world and the universe, and how big the future could be. I get a lot of value from seeing more concrete impacts of my work and feeling like I can work on problems where I can make progress, but I’m not at all interested in talking people out of spending their career on longtermism. Rob Wiblin: If there was someone who came to you who was on the fence between doing something that was more unusually longtermist and something that was, say, within your own umbrella of the work at Open Phil, and they said, “What’s the strongest case that you can offer for doing the latter rather than the former,” what kind of arguments would you make? Alexander Berger: I think a really central one in my mind is I think we just don’t have good answers on longtermism. The longtermist team at Open Phil is significantly underspending its budget because they don’t know where to put the money. When I think about the recommended interventions or practices for longtermists, I feel like they either quickly become pretty anodyne, or it’s quite hard to make the case that they are robustly good. And so I think if somebody is really happy to take those kinds of risks, really loves the philosophy, is really excited about being on the cutting edge, longtermism could be a great career path for them. But if you’re more like, “I want to do something good with my career, and I’m not excited about something that might look like overthinking it,” I think it’s pretty likely that longtermism is not going to be the right fit or path for you. Rob Wiblin: What are some examples of valuable feedback loops that you’ve observed that have helped people to have more impact than they might have if they weren’t able to see what was going on? Alexander Berger: I think a really central one is just being able to see what works and then do more of it, which is a funny low-hanging fruit. But I think often in other categories, where you don’t even know what intermediate metrics you’re aiming for, you don’t have that benefit. So for instance, the amount of resources flowing into cage-free campaigns in farm animal welfare has well over 10x-ed, because they were working. We have found a strategy or tactic that works, and we can scale. I think that accounts for a very material portion of the whole farm animal welfare movement’s impact over the past decade. But if you were somehow unable to observe your first victories, you wouldn’t have done it. So I think that there’s something about literally knowing if something is making progress. That’s a really, really important one. Also, on the other side: being able to notice if the bets aren’t paying off. We have a programme that’s focused on US criminal justice reform. We don’t do calculations for every individual grant, necessarily; we make the big bets on Chloe Cockburn, who leads that programme. But if after five years the US prison population was growing, we don’t observe the counterfactual, but that would raise questions for us — given our cost-effectiveness bar and the level of reduced incarceration we would need to be hitting to make this pencil compared to other opportunities for us. Being able to observe the state of the world and say, “Is the state of the world consistent with what it would need to be in order for these investments to be paying off?” is an important benefit that you can get in the neartermist global health and wellbeing side that you can’t necessarily get in the longtermist work, I don’t think. Another one is just really boring stuff, but you can run randomised controlled trials that tell you that the new generation of insecticide-treated bed nets is 20% more effective, because the level of resistance before to the old insecticide was reducing it by 20%. You wouldn’t have necessarily known that if you couldn’t do the data and improve. So none of those are necessarily order-of-magnitude kinds of things, but I do think if you think about the compounding benefits of all of those — and the ways in which basically the longtermist project of trying something and maybe having very little feedback for a very long time is quite unusual relative to normal human affairs — it wouldn’t shock me if the expected value impact of having no feedback loops is a lot bigger than you might naively think. That’s not to say that longtermists have no feedback loops, though. They’ll see, are we making any intellectual progress? Are we able to hire people? There are things along the way, so I don’t think it’s a total empty space. But then I think that there are these practical benefits of the global health and wellbeing work that I actually think can just add a lot of value by longtermist lights, because of those practical benefits. A lot of these arguments are in our original post on worldview diversification from several years ago. One of them is just optionality: if we think we’re going to go through a lot of different causes over time as our thinking changes — and I think that’s pretty plausible — global health and wellbeing gives us a lot of knowledge and opportunity to build up experience in causes that might turn out to be relevant. So we know a lot more about policy advocacy in the US than we did five years ago when we were getting started — and that doesn’t primarily come from the work on longtermism, even though it might turn out to really significantly benefit the work on longtermism. A second practical benefit that I think makes a big difference is around feedback loops. We talked earlier about some areas of just being able to see concretely what is working. I think we can learn lessons about what kind of grants go well versus don’t. I think there are generalisable lessons there. And a third example is these very concrete relationships, and grantees that have been able to move from one side to the other. So there’s an organisation that we fund called Waitlist Zero that primarily worked on advocacy around allowing compensation for living kidney donors. And during the pandemic, the person who had started that was able to pivot with some of our funding to start this organisation 1Day Sooner that was working on allowing human challenge trials for COVID vaccines. And if we hadn’t been funding the work on Waitlist Zero, I don’t think he would have been in that position to start the work on human challenge trials. Another example is a grantee from our macroeconomic stabilisation portfolio has ended up turning out to be really helpful for a bunch of policy work in biosecurity and pandemic preparedness. And it’s something I think we wouldn’t have necessarily expected a lot of in the beginning. And there’s just even more pedestrian stuff, like recruiting and fundraising. Eventually we think Open Phil will be a lot more conventionally successful and attractive if it’s a 50/50 longtermist and global health and wellbeing organisation, versus if the only thing we work on is AI risk. We think we would be leaving a lot of value on the table there. Rob Wiblin: Yeah. I think all of that makes sense. I feel a bit uncomfortable with the idea of advocating for doing this big programme and spending all this money just because it incidentally benefits this other programme. There’s something about that that feels uncomfortable. I guess you want to put the affirmative case for it first, rather than just say, “As a side effect, it benefits the longtermist work.” Alexander Berger: I think that’s right, and I do think that the affirmative case is first. The affirmative case is you get to do a lot of good. I think that’s why people do it. I think it’s why people are interested in it, and all this other stuff is the third thought. But I don’t actually think it’s that crazy as a consideration, because basically the longtermist view should just be a lot more comfortable with this, because the longtermist view is kind of like everything is these weird bank shots to influence the distant future. So anything that you might do to help somebody today according to longtermism is fundamentally towards the terminal goal of their distant impact on the far future. And so if what you think is that most of the moral value is in the far future and we should act accordingly, you’re committed to all of these weird bank shots — because there’s nothing else, in some sense. So if people were cynical about it, and they were like, “I’m going to pull the wool over people’s eyes,” I think that would be really unfortunate. But I think being honest that we worry that we might have gotten a little bit crazy-seeming here, and we like to do concrete, good things because that helps people understand that we are motivated by concrete, good action… I think there’s something about actually making your appearance in line with your true deep values there — where that’s not deceptive, that’s just honest and correct. And I find that totally compelling. Economist Shruti Rajagopalan on the key skills to succeed in public policy careers, and seeing economics in everything From episode #84 – Shruti Rajagopalan on what India did to stop COVID-19 and how well it worked Howie Lempel: If a student in India, who is an undergrad or at the end of high school, wants to eventually work on policy reform or maybe have a career similar to yours, how would you tell them to get started? Shruti Rajagopalan: I would say to read broadly. Economics, as a discipline, especially as you go higher and higher into the academy, it becomes very technical, you read only within the discipline. If you want to do public policy, if you want to do broader political economy kind of problems, one needs to read broadly. So that’s the number one requirement: be curious about the problems around you and read very broadly. The second thing I would say is pay a lot of attention to local context. And I have had more appreciation for this as I have grown older and as I have written more, because initially the tendency is to follow best practices. For instance, you look at what are the best practices abroad, and you want to copy them, right? Or you want to look at the best academics. I mean, you want to read everything Milton Friedman has written and be genuinely inspired by it — and I was too, right? But I would not make the error, I hope, of applying what Friedman says one-to-one to what is happening in India without considering local context. I think local context is just incredibly important, and academia tends to reward people who walk away from it. And I think within the culture of public policy we need to reward people who are more rooted in local context. And I think the third is epistemic humility. And this is not just for people who are starting out in public policy — actually, they might have a lot of it — I think this is more for the rest of us who’ve been doing this for a while, and some people who’ve been doing it for a very long time. The idea that there is a single best answer out there, right? And we can both get the data to confirm it or to deny it and prove it. I don’t think we have anything like that for public policy. We don’t have any of these eureka moments you might have in natural sciences or hard sciences, in social sciences. So I think we need to really carefully think about what it means when we read a study. What does an empirical paper mean? Whether it confirms our bias or it tells us something different about the world. So I think epistemic humility, combined with local knowledge and context, combined with just really broad reading — everything from literature to law, to history, to cultural mythology — I think that really helps. That’s how I was raised: to think broadly about the world, to travel, to pay attention to the arts, to pay attention to literature — and that’s sort of how I have formed my view of the world. I see economics in everything. I think it’s an incredible discipline. I think the way one sees the world, once they have been trained in economics, you can never unsee the world that way. And the greatest strength of economics is that it makes you pay attention to things that are not visible. It makes you think about unintended consequences — or to borrow Frédéric Bastiat’s great phrase, “the seen and the unseen,” right? It makes you think about the unintended consequences, the unseen effects. So economics is just a fantastic discipline. It’s got a great, very sensible methodological core in terms of rational choice, and the ability to apply it to everything from constitutions to culture, to history, to religion, to crime. In that sense, I’m certainly one of those Chicago school type economic imperialists: I see it in everything. So I think the economic way of thinking is very powerful. So any student who asked me — especially if they’re working in public policy, which has so much to do with tradeoffs and costs and benefits — I would say that an economics education is both thrilling, if it’s done the right way, and it is invaluable. So economics in everything. You can read broadly, but get trained in economics. So that is one thing I would say. I think the other moves are… It’s very means and ends linked, right? I wanted to be an academic; I wanted to be a professional economist who teaches economics. In that sense, the minimum academic degree you need to do that is a PhD, and that’s the way to think about it. But there are many ways to do good in the world which have nothing to do with academic affiliation or getting a PhD or going to law school and things like that. The effective altruism project is a great example of that. That’s something you guys do really well. So I think people just need to decide what the end goal is and accordingly pick the means to achieve the goal. So for those who want to work in formal public policy — this might be the government, think tanks, all your multilateral organisations and agencies, like the World Bank and things like that — you definitely need advanced degrees in economics, social sciences, public policy, to achieve that goal, right? If one wants to be a lawyer and think about all the constitutional issues I’m talking about, it’s really helpful to have a law degree. It’s a little bit hard to talk the talk and think the think outside of the legal training. So I would say for those things it’s useful. So it really depends on what the end goal is, but if we’re asking about really bright young people who want to be informed, engaged citizens: read broadly, but also read economics — it just takes you a very long way, in a way that I think no other discipline does. Howie Lempel: Great. And then if there were listeners from the US or the UK who are really interested in working with, or in, India, and wanting to sort of get a start, is there any different advice that you would give to them? Shruti Rajagopalan: Understand the local context, once again. It’s harder when you’re a foreigner. And sometimes I’m a foreigner in India because I don’t speak all the languages, right? I just speak two or three of them. So just really understanding, knowing the language helps. I know a lot of political scientists and economists who work in India and they all learned the language of the area where they were doing their fieldwork or running their randomised control trial and things like that. So I think that’s very important. I think having a network helps if you’re doing academic work. You need to be able to go there, raise grant money, work with local people, coauthor, publish. So try and formulate networks, go to a department where there are professors who are working on India. For the non-academics, I would say there are so many ways to support India, starting with Emergent Ventures. I might as well plug what we are doing here. But if someone just has an interest in India and understands the moonshot value of doing anything in India, and how a little bit of money and few resources can really enable people to change the world in a big way for a large number of people, I would say support philanthropic efforts in India. And it need not be Emergent Ventures, it could be anything. But Emergent Ventures supports very particular kinds of ideas. So if anyone likes those, I would definitely recommend ourselves. And overall, the best way to learn about India many people think is reading, but I think it’s traveling. So if people just want to learn more about India, just go there. Spend some time there, find some friends there, and then you go from there. It’s the way I learned about the United States. I think the reverse will be true in terms of learning about India. Howie Lempel: Makes a lot of sense. J-PAL lead Claire Walsh on her career advice for young people who want to get involved in global health and development *From episode #13 – Claire Walsh on testing which policies work & how to get governments to listen to the results Rob Wiblin: It sounds like you have a very senior role in J-PAL. But you’re only 29, right? Claire Walsh: Yes, that’s true. Rob Wiblin: Is it fair to say that your career has gone pretty quickly, maybe more quickly than you thought, or is this just typical? Claire Walsh: Definitely more quickly than I thought. Rob Wiblin: How do you think you advanced up the ladder so fast? Claire Walsh: Not sure I have the exact answer to your question, but in my experience, looking at my peers who are involved in similar international development policy work, I think we owe a lot to our master’s degree programmes. There are a lot of public policy schools that tend to launch people into rapidly rising careers in governments and international policy in research organisations. They’re called the APSIA schools. Rob Wiblin: We’ll stick up a link to those. It’s something we found in general that across the whole policy sector that people can really end up in positions of serious responsibility late in their 20s or early in their 30s. People just have to be a little bit opportunistic and hope to be in the right place at the right time that they can end up really jumping up a couple of rungs, if they’re lucky enough to get that situation. Claire Walsh: Yes, I think there’s a lot of truth to that. Rob Wiblin: In the past, you were working in nonprofits. Maybe can you give us a description of the path you’ve taken in your career to end up where you are now? Claire Walsh: Sure, so in college, I was actually an anthropology major. I had studied abroad in Uganda, and knew that I wanted to continue to work in East Africa, and so I started working part-time for two small NGOs that have now since grown a lot bigger, Educate! and Africade, working on employment opportunities for youth who face the labour market that’s just dismal in Uganda and Tanzania — where they graduate high school with all those great credentials, and find that they’re faced with a lot of really sorry job options. So trying to find ways that they can start their own businesses or become entrepreneurs. I was excited to work for them for about a year and quickly realised that I didn’t have enough skills to really make it in the international development sector. That’s why I went back to grad school so quickly just after a year of work. Rob Wiblin: What kinds of skills were you lacking? Claire Walsh: Oh, gosh. Rob Wiblin: Too many to name? Claire Walsh: So many to name, but I’ll never criticise my liberal arts education. I owe everything to it, so yes, I don’t knock that either, but I didn’t have management skills, accounting, basic business, basic finance, how to fundraise. Then on the more technical side, I was supposed to be working on monitoring and evaluation, but I had no idea about statistics, econometrics, impact evaluation methods, and so I made it a priority to get those skills in grad school. Rob Wiblin: What did you do after that? Claire Walsh: Immediately upon finishing grad school, I applied to J-PAL’s policy wing. Rob Wiblin: OK, wow. Got helicoptered in to do some really important work. Where do you think young people should start if they want to work in global poverty in a similar way to you? What majors would you recommend that they do? Claire Walsh: Economics, political science, international development. Rob Wiblin: Snappy answer. Are there rather options or paths in global poverty that you think are promising beyond the traditional ones? Like possibly doing entrepreneurships, starting a business in the developing world, or just going and trying to work there in government just directly? Claire Walsh: Definitely. You don’t have to go the research route. You don’t have to go the international aid route. I’m very pro-entrepreneurship, but I think sometimes there’s a risk of, “I have the one silver bullet solution that’s going to solve anything. It doesn’t exist. Therefore, I should create an organisation around it.” I would caution a little bit against that. There’s probably already someone who’s working on the problem. It might be more effective if you wanted to go work for their organisation. Rob Wiblin: Do you think it’s important to live in developing countries where you might be helping with policy change? This is something that people should consider doing in possibly their late teens or early 20s when they have their opportunity to? Claire Walsh: Definitely, and I regret not doing it more. I think if you want to work in international development, having at least two years of experience living in a developing country is a huge plus and can open a lot of doors. Rob Wiblin: What is it specifically that you gain from doing that? Claire Walsh: It’s so many different things you gain. It’s a deeper understanding of human difference, of cultural and ethnic diversity, of being able to live with less creature comforts than you’re used to, being able to understand how to get things done in resource-poor environments, being able to put your personal comfort behind a greater cause and mission. Then, it also gains you a lot of credibility with a lot of major players working in development to see that you’ve lived and worked and thrived in low- or middle-income countries. Rob Wiblin: Let’s say that you’re doing or you have completed a relevant undergraduate degree. How can you go about building a professional network in global poverty reduction? How can you experiment with different options that you might have in that whole field when you’re in your mid-20s? Claire Walsh: It’s not the prettiest answer. A lot of people, right out of undergraduate, first take an internship that’s often unpaid and often in a developing country. Then they convert it later into a paid position. That’s what I did. That worries me because I think internships should be paid, but it is what a lot of people do. I think what really helped me build my professional network in international development because I wasn’t lucky enough to get into an important institution right out of undergrad. I was working for a very small niche NGOs, and was going back to get my master’s. That completely exponentially grew my professional network and gave me connections to all the major organisations that I would hope to work for. Rob Wiblin: It sounds like perhaps the most important thing that we can learn from your career is the value of doing one of these excellent master’s programmes. Are there any other programmes that people could try to get into that could really help to advance their career? Claire Walsh: Yeah, if you’re interested in public policy and international development particularly, there is a group of schools called the APSIA schools. I don’t remember what the acronym stands for, but it’s a group of about 25 master’s programmes at universities around the US, and I think maybe a couple in Canada that have high quality master’s degrees in this fields. They include Princeton’s Woodrow Wilson School, Harvard’s Kennedy School, Johns Hopkins’ School of Advanced International Studies, George Washington, Georgetown, American, Denver University, a lot of universities in California. It’s a lot of great public policy schools out there. Rob Wiblin: That’s a very useful list. How do you think doing a master’s compares to doing a PhD? Have you considered doing a PhD yourself? Claire Walsh: Definitely considered doing a PhD myself. Doing a PhD in order to make policy change is not one of the top things that most people going to a PhD are trying to do. I’m really proud of the ones who are, and I would encourage more and more to think about it, but going into a PhD is really great for becoming an academic or a full-time researcher, but I knew I wanted to work in policy action rather than full-time research. Rob Wiblin: That makes sense. I guess most of the principal investigators all have PhDs at J-PAL, right? If you want to go into hardcore social science research, then the PhD is necessary? Claire Walsh: That’s correct, yes. If you want to be a principal investigator on an impact evaluation that you hope might someday be published in an academic journal, you need a PhD. Rob Wiblin: What are the differences you’ve found between working in academia versus the nonprofit sector? Do you feel like you’re a much better fit for one than the other? Claire Walsh: In academia, there’s much more room for scepticism, and I think that I am potentially more suited to that. In nonprofits, I think it was really exciting to be working directly with people, providing people directly with services, and I miss that a lot. But I enjoy that within academia, there’s more room for questioning whether what we think is helping people is or isn’t. Rob Wiblin: Is the challenge there that if internally, you’re not so optimistic about the value of the information that you’re delivering, it could be demoralising for staff, or is it that it would discourage donors from supporting you and helping you to scale up the programme? Claire Walsh: I actually think that a lot of NGOs are very introspective and are always seeking to question their assumptions and improve their model. It’s more just me, personally, that I have a really hard time working on something if I don’t know it has an impact. Rob Wiblin: A question I regularly ask people I’m interviewing is whether you think you would do more good if instead of trying to do good through policy reform, instead you’re going out and try to make a lot of money, and then donated your earnings to help tackle poverty, either doing cash transfers perhaps or providing medical services or perhaps funding a think tank or an organisation like J-PAL. Is that something that you have ever thought about or have a view on? Claire Walsh: I’ve definitely thought about it. I think in some ways, I think it’s quite likely that I could have more impact just giving all my money away to cash transfers for the poor. Sometimes, I wish I was just doing my daydreaming about that. But I have a broader goal in mind, which is helping improve institutions and having an optimism despite corruption, despite the hurdles of bureaucratic processes that governments in developing countries have the capacity and the capability to run high-quality programmes that reduce poverty — and that in some small way, by helping them test their latest innovations before scaling them or scaling up things that RCTs have shown to be effective in the past, that I’m also contributing in some very, very small way to those institutions being more effective to deliver quality services to their citizens. Rob Wiblin: Yes. It sounds like J-PAL has quite a lot of traction with developing world governments, and their budgets are often very large. It would surprise me if you could have more impact by just donating the money. Maybe you have potential to earn more money than what I appreciate, but I expect that the changes in budgets you’re getting through J-PAL are probably doing more good than that. Claire Walsh: I would just have to get out my spreadsheet. Rob Wiblin: Do you feel like your career track has been risky at all? I mean initially, you were working at a nonprofit in the developing world. Did you ever feel like you weren’t sure what your next position was going to be, or whether you’d be able to transition into wider problems if you decided that you didn’t want to work on global poverty anymore? Claire Walsh: I will say that early on in my career, I was very worried that by working for small NGOs, that I couldn’t transition into working for big NGOs, foundations, or policy organisations or research organisations. I was quite pleased that that wasn’t the case at all. Obviously, the master’s programme helped with that a lot, but it was possible to go from a smaller NGO to a bigger organisation like J-PAL. I do think that it is harder for people who go into public policy to then go into the private sector. That’s something a lot of my peers think about and struggle with a lot. Organisations value skills from the private sector and from the business world; I feel like it’s usually easier for people to switch from the private sector to the policy space or the public sector, rather than the other way around.