Latest / Quantum Potential / Impact of Micro-Investments in School Children with Felipe Barrera-Osorio
Transcript
- 0:00The mechanism is essentially that you are saying to a student, "We believe in you, and that's why we are giving you this scholarship." And mind you, the two groups that were recipient of the scholarship, both groups were very poor, but one group, we say,"OK, you will receive it because we believe your household needs support.
- 0:22We believe that you, you need this money." Basically, that's the poverty one.
- 0:28And the other one we say the implicit message was, uh, "We believe that you can do the work, that we believe in you, uh, in your ability," and that trigger a very different response.
- 0:41The merit one, uh, had, uh, impacts on cognitive even if those merit were among the poorest of the poorest.
- 0:49Yes.
- 0:50So that is fascinating.
- 0:53My guest today is Felipe Barrera, associate professor of public policy education and economics here at Vanderbilt in our Peabody school of education.
- 1:02Felipe has a Ph.D. in economics from the University of Maryland, College Park, USA, and a degree in economics from the University of the Andes, Bogota, Colombia.
- 1:23Welcome to Quantum Potential, a podcast dedicated to curiosity and discovery.
- 1:28My name is Cybele Raver, and I serve as the provost of Vanderbilt University.
- 1:32As a leader in higher education and as a social scientist, I believe that we can transform higher education for the 21st century.
- 1:39At my core, I am wildly curious about other people's work, whether it's in science, the social sciences, the humanities or the arts.
- 1:48This podcast gives us the opportunity to break open new ideas through discovery and creative expression.
- 1:55Barrera-Osorio traveled to the United States in 2006 to work as a senior economist at the World Bank in Washington, D.C.
- 2:09Between 2011 and 2020, he was an assistant professor and later an associate professor at the Harvard University Graduate School of Education.
- 2:17Since 2020, he has joined Vanderbilt University as an associate professor.
- 2:21We're so thrilled to have him, in all the ways that he really enriches our understanding of education in global context.
- 2:28He's a founding member and co-director of the Network of the Evaluation of Impact.
- 2:32One thing that's really super impressive to me is you're also a visiting researcher at the Abdul Latif Jameel Poverty Action Lab—JPAL—at MIT.
- 2:41I'm really excited to have us dig in and learn about what that means and, uh, what it, it it tells us, um, in terms of the impact of your work in the wider world.
- 2:52Um, so welcome...
- 2:53Thank you.
- 2:53...Felipe.
- 2:53So exciting...
- 2:53Thank you.
- 2:54Thank you so much.
- 2:54It's a pleasure to be here.
- 2:56It's great.
- 2:57So, Felipe, you take on these really huge questions about how nations and districts run schools and how they structure educational opportunities for students.
- 3:07You ask questions such as, are school policies and programs really effective in shaping our ability or our willingness to attend school, work hard in school and how much we learn?
- 3:20Or is it the case that those experiences that we have in school are fundamentally shaped by students' own characteristics like grit, or by family resources, like how much time or money families can spend or contribute to supporting their children's education?
- 3:34Um, these are obviously questions that were true to my own area of research in early childhood here in the United States, and I'm fascinated to learn about how you see that through a global lens.
- 3:44In countries as diverse as Cambodia, Mexico, and of course, Colombia.
- 3:50Do school policies lead us not only to stay in school, but do those policies lead us to have better jobs and happier lives?
- 3:57And I love that you ask that second question about people's happiness— not just what they're earning.
- 4:02Um, I think it's a really important contribution in the field.
- 4:06So you carry out these incredibly careful policy experiments in low- and middle-income countries to try to disentangle those many factors.
- 4:15Um, you're internationally known for your work on public/private partnerships in Bogota, in Colombia, and I can't wait to dive into that.
- 4:22But first I actually wanna back up so that our audience, so that people listening, um, feel more ... a closer understanding to what we mean when we say policy experiments and what we mean even when we say low- and middle-income countries.
- 4:37Very good.
- 4:37Great.
- 4:38So first, can you just tell us a little bit about, um, your experience in low- and middle-income countries?
- 4:43I thought perhaps we might start with Cambodia.
- 4:45What's it like to collect data there?
- 4:48What is, uh, families' experience of, uh, their own earnings, their own income?
- 4:54What do we mean when we say low and income country?
- 4:58Uh, collecting data in Cambodia, as in many other countries, is super difficult.
- 5:03It's a exercise in which you need to go usually to rural areas.
- 5:08And traveling in those, uh, countries is, uh, especially challenging.
- 5:14If things like, uh, you need to transport, uh, equipment, you need to, it's, it's a huge operation.
- 5:21Uh, usually one person like me, we hire firms to, uh, do these type of jobs.
- 5:28But usually one person like me, uh, at the beginning of any survey, uh, collection of the, uh, data.
- 5:35We got to feel and we, uh, experience firsthand, uh, what does that mean?
- 5:40So like in any country, uh, it was very complex to arrive to, to different households. And the households are very poor. Poverty is a very concrete concept, and you feel, uh, the poverty in every aspect of the, of the household.
- 5:59So you arrive to one of these households, and the households, the infrastructure of the household is uh, in some cases very precarious.
- 6:07The floor, the walls, the conditions are especially, uh, taxing, uh, for some families.
- 6:14So collecting data is a very interesting aspect, but is, uh, very challenging.
- 6:20Amazing.
- 6:21So here you are.
- 6:22Coming from, you know, Vanderbilt, uh, or Harvard or the U.S. generally, right?
- 6:28And you're arriving and you're saying, "I'm really interested in you and your child's education, um, but first I'd like to know a little bit about, uh, your family circumstance." Something along those lines, right, to be able to understand even how much resource families have.
- 6:43Yes.
- 6:44And there are two ways to think about this.
- 6:48Poverty can be an income measure.
- 6:50You can measure how much the family has in income, let's say, every month.
- 6:56Uh, but that measure is very difficult to, to get.
- 7:00You are asking people how much do they earn or the wages or, and that is very difficult.
- 7:06So usually what we do is we observe the characteristics of the household.
- 7:12And with those characteristics we were, we are able to create indexes.
- 7:17Like that there are proxies, uh, for this idea of poverty.
- 7:22And uh, actually that is a very concrete technique that a lot of governments are using, is they ... Instead of asking about income or consumption, uh, they ask about the characteristics of the house— what is the, the material of the floor, what is the, uh, how many kids do you have in the household?
- 7:40So you have different measures of physical evidence, physical, uh, attributes of the household, but also what we call a human accumulation— what is the education of the mother?
- 7:53What is the, uh, number of kids in the household?
- 7:56And from that, you can create indexes and you can have a proxy for poverty.
- 8:03Amazing.
- 8:04So you've outlined that really, uh, elegantly in a recent paper.
- 8:09Congratulations on your first authored article in the 'Journal of Policy and Management' that's just come out in 2024. And the title of that article in case anybody's interested in going to read it is Long-Term Impacts of Primary School Scholarships—Evidence from Cambodia.
- 8:26And you outline this in Table 1, um, where you suggest, you know, that this is the right way to think about what families have, um, and what they can use in their lives rather than, you know, what we would use in the U.S., which is to ask a family about their monthly earnings or to look at their tax statements, right?
- 8:44Yeah.
- 8:44So it's fascinating.
- 8:46Can you just read us a couple of items and how much, or how many people, uh, what proportion of the sample has have those things?
- 8:54Just to give the audience a bit of a sense to paint a picture of, you know, where we're thinking about education.
- 9:00So, for instance, we ask what is the number of minors in the household?
- 9:05And the number of minors in the households country is from 1.7 to 2, eh, So, two kids?
- 9:11Two minors in the, in the household.
- 9:13Great.
- 9:13So not huge families.
- 9:14Not Not huge families.
- 9:15Yeah.
- 9:16Eh, if they own motorcycle.
- 9:18Motorcycle is super important in Cambodia because it's the mean to transport yourself and if you produce something to, to drive to the market.
- 9:26So every household has less than one motorcycle.
- 9:30Some household have some motorcycle.
- 9:30If they own car or truck.
- 9:34If they own a oxen or buffalo.
- 9:37If they own a pig.
- 9:38It turns out that owning pigs in Cambodia is a very important element that households can have and create wealth.
- 9:48So every household in Cambodia has less than one pig.
- 9:53So for or only a fraction of households have at least one pig.
- 9:58Yes.
- 9:59And uh, if they have a hard roof.
- 10:03If they have a hard wall.
- 10:04If they have hard floor.
- 10:07And this capture a lot of the, as we were saying, the level of poverty of the, of the house.
- 10:13Yeah.
- 10:13Amazing.
- 10:13Okay, so here we, you've, you've given us a really clear indication that this is not the same as looking at education in U.S. districts, um, in either Baltimore or, you know, Tulsa or a name another small city or a rural area in the U.S., right?
- 10:30Very different context.
- 10:32So, um, tell us a little bit about this study.
- 10:34I think you frame it really beautifully on page 33.
- 10:38Would you mind just reading us, um, that first "this study has investigated," starting right there?
- 10:43Sure.
- 10:43It's just a great description of the study.
- 10:48This study has investigated the long-term impact of a scholarship program that increases schooling with a particular focus on links between the schooling, the development of cognitive and social emotional skills, and labor market and well-being outcomes later in life.
- 11:04To this end, we evaluate the effect of a primary school scholarship program in rural Cambodia nine years after the program inception, tracking the student participants when they were on average 21 years old.
- 11:20Amazing.
- 11:21So you take kids in school— and you randomly assign schools or kids?
- 11:28I, I, We randomize schools.
- 11:30Okay.
- 11:31So you randomly assign some schools to getting one kind of scholarship based on need, and another group of schools another kind of scholarship based on merit.
- 11:43Yes.
- 11:44Because schools are doing, kids are doing well in those schools.
- 11:47Yes.
- 11:47And some schools that don't get the scholarship right now, but they may get the scholarship some other time, right?
- 11:53Yes.
- 11:53And then you follow those same students out of school and into adulthood for nine years. And then you detect what is the impact of that investment, right?
- 12:05That offer, that essentially investment of a scholarship.
- 12:10Yes.
- 12:10Okay.
- 12:11I have to say something that, at least for me, it's very important.
- 12:15These studies require a lot of man and woman power.
- 12:19Uh, we work with coauthors especially this one, my coauthor, Deon Filmer, and Andreas de Barros.
- 12:26Andreas is a terrific.
- 12:27These are your co-authors.
- 12:29Those are my two co-authors.
- 12:31And these stories, as, as we were mentioned, they are very difficult to create.
- 12:37But in any case, so the root of the story was when I was at the World Bank, uh, Deon was, uh, working very close to the, with the Cambodia government.
- 12:48And the Cambodia government has one question, which is— What will be more effective to give a scholarship based on merit or to give it based on poverty?
- 13:02So those two scholarships are very different. And related to one point that you made that is very important is, the context is very different, Cambodia and United States, but the questions, and we believe the mechanisms are similar in both contexts.
- 13:21So we are trying to do a very difficult trick, which is investigating these questions that can be generalizable, but in a specific context.
- 13:33And you need to separate what is the specificity of the context vis-a-vis what can you be general to other countries.
- 13:43So that question of how do you target if it's merit-based or poverty is extremely relevant to United States and at that moment to the government of Cambodia.
- 13:57Absolutely.
- 13:58And I guess one of the things that I'd love you to talk a little bit about is, when we think about making that kind of investment, my guess is that it's with the assumption that if we provide families with a bit of resource, it'll motivate them to make schooling more important than all of the other things going on in their lives, right?
- 14:20That maybe they're dealing with a seasonal, uh, farming need, an agricultural need where they need kids to be working, uh, with the family in that rural context to help bring in whatever has been planted, right?
- 14:34Yes.
- 14:35Or alternately, maybe they need their kids to care for elderly family members, right?
- 14:41So the goal of that scholarship, tell me if I'm right, is to help to really highlight and motivate families to understand that education is an incredibly important thing to spend time doing.
- 14:53Yes.
- 14:54And you, you mentioned two elements that are very important.
- 14:58One is what we call the opportunity cost.
- 15:01So we are giving money, families saying,"Hey, we realize that sometimes it's very difficult to send your kid to school because there is an opportunity cost.
- 15:11You can put the kid to work the land." And again, we are in the context of rural area in Cambodia.
- 15:17And that, uh, possibility is very clear there.
- 15:21So we are saying to the family, "Hey, we are going to give you some money that at least reduce that opportunity cost," but also that money, uh, can have this effect that is motivating the household to invest more in education.
- 15:37It has this idea that, eh, we are telling the household education is important.
- 15:41It's an important investment. And that is another family of programs that that are very important in Latin America and other parts of the world, which are called conditional cash transfers.
- 15:54So this is the programs that we are trying to say to the family,"It's important to invest in education.
- 16:00We are going to help you." It is a social contract, and we are going to transfer money usually to low income families so that they can invest in human capital.
- 16:28Now we get to talk about what you found, right?
- 16:30I mean, Felipe, this is so cool because you did this so carefully, right?
- 16:35You have these two different ways of thinking about scholarship funding.
- 16:38And by the way, I noted that the scholarships were $20.
- 16:42They are very small.
- 16:43They are very small. For United States is almost two coffees.
- 16:48For Cambodia, uh, can be some money, uh, in rural area.
- 16:53Yes.
- 16:53Right.
- 16:53I mean, maybe half a pig.
- 16:55Yes.
- 16:57Right.
- 16:58I mean, we start thinking about, you know, on one hand you could say, well,$20 isn't gonna make any difference.
- 17:03But it's important in that context, you know, from the framework that you just laid out, maybe $20 is actually quite substantial.
- 17:10It is. Actually, we, we are surprised with the results in, in one way.
- 17:15Another point that, that I want to make, uh, talking about collection of data.
- 17:20This is nine years after we do the ex, uh, we did the experiment, so we have to go back to Cambodia.
- 17:27Again, we work with very specialized collecting firms, uh, collect ... survey collecting firms in the countries.
- 17:34That is something fundamental to find a good firm to collect data.
- 17:39Uh, but we went back. And to reach these households was very difficult.
- 17:45I'm sure.
- 17:46Uh, it was a tremendous effort.
- 17:48There is a lot of mobility in, in low-income, uh, households.
- 17:52But we were able to, to collect this data, and the results are amazing, even if the, if the amount is very little.
- 18:00The, uh, families who received this scholarship, both of them, increased number of years of education...
- 18:07Amazing.
- 18:07...which by itself is very important.
- 18:08Right?
- 18:09So just pause there.
- 18:10I mean, I think, right?
- 18:12That's an amazing thing that you found, right?
- 18:14That you did in fact find evidence that that scholarship increased education.
- 18:19Yes.
- 18:20So kids did stay longer in school because of that scholarship.
- 18:24And this is, uh, for both scholarship, for merit and for, uh, poverty.
- 18:31So a kid that received the scholarship because he performed well in a standardized test, uh, performed increased years of education equally as a person who received the scholarship, uh, for being poor.
- 18:47Amazing.
- 18:48Yes.
- 18:48And non-trivial, right?
- 18:50So if I read that sentence in on page 23 where you report your results.
- 18:56You have this great sentence that, uh, specifies what you just said, "Across all indicators, the scholarships have consistently, statistically positive impact on the acquisition of formal education." And then you give the 4.4% increase in the number of grades students complete.
- 19:12But then you have this great second finding—an increase in the probability of completing primary school by eight percentage points compared to a control mean of 57%.
- 19:22Or, or a 14% increase and an increase in the probability of receiving any formal education during 2011 to 2017 of 6.8 percentage points from a base of 75%.
- 19:34So roughly it says from, as I understand it, that the control group who didn't get the scholarship, they had a 57% chance of completing primary school.
- 19:44Yes.
- 19:4457% of the control group finish grade six.
- 19:49Yes.
- 19:49Okay, so just over half of the control group finished grade six.
- 19:54Yes.
- 19:55Yes.
- 19:55That also reflect one point, which is the level of school at acquisition in Cambodia is very low.
- 20:02And this experiment, this small investment...
- 20:05Yes.
- 20:06...made on the basis of either merit or on the basis of family need increase that by about 10 percentage points.
- 20:12Is that about right?
- 20:12Yes.
- 20:13Which is quite big.
- 20:14Yes.
- 20:15Really big.
- 20:15Right.
- 20:16So, you know, I think we know that economics is often referred to as the dismal science, right?
- 20:23Sometimes we give good, good news.
- 20:25Yes.
- 20:26Cause often there's not such good news.
- 20:28But this was really great news.
- 20:29Yes.
- 20:30How did it feel to find that you engaged with a group of communities, um, uh, to substantially increase kids' likelihood of completing primary school?
- 20:39It's very humbling.
- 20:41When I do these type of experiments.
- 20:43You start thinking, "Will I impact the life of people?
- 20:46Will I really move the needle?" And sometimes you do.
- 20:52And it is a quite extraordinary feeling.
- 20:56I have another program in Colombia in which ... we did a conditional cash transfer program and three years after we start the program, I visit the country and I run into one of the beneficiaries.
- 21:10Mm-hmm.
- 21:10And she remembered me because I was the person in charge of the randomization of the lottery in front of the community.
- 21:17And she said to me, "You cannot imagine the change that these program did in my life.
- 21:23The change in the conditions of my household, eh, dramatically improve.
- 21:30And it was because of, of the program that that you did." which was not my program, it was the government program.
- 21:36But those feelings are amazing.
- 21:39Ah, thank you so much for sharing that, Felipe.
- 21:41It's such a, I think, important message for us to really sit with for a minute, right?
- 21:47Yes.
- 21:48You know, I think we often think about the life of a professor as teaching students, which is super important, and we'll get to that, or running analyses on large data sets or supervising grad students or figuring out how to get a paper submitted for publication.
- 22:07And it's great to hear you really sort of zoom out from that and talk about, um, both the humbling and the deeply emotionally moving experience of changing, uh, people's lives— or at least measuring...
- 22:21Yes...appropriately how government programs can change people's lives.
- 22:26Somehow I, I am very lucky.
- 22:28Because the, the type of research that I, that I do is very, it has very concrete impact in policy, and that is a very powerful idea.
- 22:41When you try to connect research with the actual implementation of policies, is very direct the link with what you are doing and the effect on people.
- 22:54And I am in that realm, and there are other people that do other things that are important, but in this realm, uh, I can, I can see the effect on people, which is a, a great sort source of pride and happiness.
- 23:10And that is one important component of my research.
- 23:14Amazing.
- 23:15And so when you think about that, you just described that parent standing there watching you conduct the lottery.
- 23:23Hmm.
- 23:23And I would love to just take a minute to talk about the methods that you use, right?
- 23:27Because we could immediately think,"Wow, you're randomly assigning families to getting something special.
- 23:33Or not getting that thing.
- 23:35Um, is that unfair?
- 23:37Shouldn't you have distributed it to everybody?" Um, so how, how does that work?
- 23:41Um, and how does the community respond?
- 23:44Um, do they understand that there isn't enough for everybody to get the good thing, uh, and therefore lotteries are fair? Or do, is there a feeling that you are experimenting with the community?
- 23:55We do these lotteries, uh, a lot of the times, for practical reasons in the sense that we don't have resources enough for all the people that they needed.
- 24:05But there is still another important aspect of that, which is, let me go back to the example of Cambodia.
- 24:12We don't know if it's better to produce a merit base or a poverty base.
- 24:18And from the point of view of the government, the resources are limited and the opportunity costs of the resources are huge.
- 24:26So we need to start digging into these questions, and we need to start finding if one is better than the other.
- 24:34So from that point of view, uh, to have a method that for sure tell me that the impact of the policy is this, it is very important.
- 24:44So we, we do it for those two reasons, but one point is that sometimes, uh, randomization lotteries are the fairest way to assign resources.
- 24:57Why?
- 24:58Because the probability of receiving the good is equal exactly for everyone, so it is very equal.
- 25:07And that is an important component of a lottery.
- 25:11Of course I, I don't, I don't dismiss and I, I think seriously about ethical considerations of, of randomizations, of lotteries.
- 25:20Uh, I think that there are important questions there, and I think that researchers like me needs to think very carefully about the ethical considerations, but those are the fundamentals for doing these methods.
- 25:33Makes perfect sense, right?
- 25:34Because you could have found that the merit-based or the poverty based, um, scholarship had a negative consequence for families, right?
- 25:42Yes.
- 25:42So you actually, from my perspective, are, are really taking a very careful, um, and very considered look, rather than assuming that the scholarship is gonna work for everybody in the same way.
- 25:55Yes.
- 25:56And actually we find we, we found in that experiment that even though the two scholarships increase human capital acquisition, the two type of kids increase education, only the merit one produce cognitive effects.
- 26:11So that is important by itself, and that is one case in which you start thinking,"OK, how can we structure a scholarship that, let's say, maximize the benefit?" And that is one of the, of the objectives of the paper.
- 26:26Yeah.
- 26:27Fascinating.
- 26:27I read that part very carefully, of course.
- 26:30Um, and I was interested because, you know, what you're determining or describing as cognitive measures are measures of children's ability to demonstrate what they've learned, right?
- 26:40Yes.
- 26:40Vocabulary on and essentially their capacity to show what they've learned on a test.
- 26:45Yes.
- 26:45Um, and I thought it was fascinating that students who have received the signal that you know, that they, that their effort and their accomplishment on those tests is seen.
- 26:56Yes.
- 26:56You see them.
- 26:57Right.
- 26:58It's this a, a very interesting signal.
- 27:00"I see your effort, I see your hard work and, um, you're getting this scholarship because we know you're working hard." Yes.
- 27:08And it's just a very interesting ...
- 27:08um, do you have a sense of, of the mechanisms there?
- 27:13The mechanism is essentially that you are saying to a student, "We believe in you.
- 27:20And that's why we are giving you this scholarship." And mind you, the two groups that were recipient of the scholarship, both groups were very poor.
- 27:30But one group, we say, "OK, you will receive it because we believe your household needs support.
- 27:36We, we believe that you need this money." Basically, that's the poverty one.
- 27:42And the other one we say, the implicit message was "We believe that you can do the work that we believe in you, uh, in your ability." And that trigger a very different response.
- 27:54The merit one, uh, had impacts on cognitive even if those merit were among the poorest of the poorest.
- 28:02Yes.
- 28:03So that is fascinating.
- 28:05Totally.
- 28:05And then tell us a little bit about why you think you didn't find any impact on employment.
- 28:11Is it because no one in the village is working or everyone in the village is working or that there really isn't, uh, a, a detectable effect of that small investment nine years earlier on employment?
- 28:23Um, 'cause that was fascinating to me that you found null effects.
- 28:27You found no statistically significant impact of the intervention on employment.
- 28:32There is one hypothesis if I, if I can say that, which is the following, is that this is early entry in the labor market.
- 28:39And perhaps at, uh, earlier stages, uh, is difficult to detect difference in labor.
- 28:50And more, uh, at the light that we increase the education progression of these individuals.
- 28:58So perhaps these individuals enter a little bit later to the labor market than the control group, and the control group experience matter in the labor market.
- 29:07So perhaps the control group at that stage have higher wage than the kids that received the scholarship.
- 29:15That is a hypothesis.
- 29:17I, uh, we cannot truly test that, but that is an explanation.
- 29:21It's fasc...
- 29:21I mean, you know, one of those things that we always have to remind people, right?
- 29:25That null—no statistically significant difference— it doesn't mean it didn't work...
- 29:32Yeah.
- 29:33...to state three double negatives.
- 29:34Yes.
- 29:35It just means we didn't detect that it did work.
- 29:38Yes.
- 29:39Um, and I love that you are posing, maybe it's because it's too early.
- 29:43Maybe it's because at 21, as students are just entering the labor market...
- 29:47Yes.
- 29:47...things haven't evened out.
- 29:49But something that we find is that the well-being, a measure of happiness, if we can say that the measure of well-being, how do they feel about life, is higher in the people who receive the scholarship than, than the people who didn't receive the scholarship.
- 30:05So that is, uh, something that is also very promising.
- 30:08Absolutely.
- 30:09What, what possessed you?
- 30:12What led you to include a measure of happiness?
- 30:14As an economist, that is not what I expected to have seen in this, um, paper, right?
- 30:19Usually not to, you know, tease my econ friends, but usually there's a very strong emphasis on labor market outcomes and not on mental health and well-being running right parallel alongside, um, labor market outcomes.
- 30:33So what led you to include measures of happiness in this study?
- 30:36Oh, Cybele, this is a discussion that is in the profession, but, uh, I. Look, uh, economists from long, long time has been interested in well-being, in the, the well-being of individuals.
- 30:48And uh, the things that we can measure very clear are labor market, but that doesn't mean that there is no interest in well-being.
- 30:57Moreover, there are a group of, of economists working in happiness.
- 31:01So why we measure, uh, well-being?
- 31:05We thought that it was an important measure.
- 31:07We thought that, uh, we, we believe in education that, uh, create individuals that are, that can explore their own potential and, and that can grow as an individual— not only in the labor market, but as a person, as a citizen.
- 31:24And, and that's why we measured that variable.
- 31:44In another paper on families in Mexico, you actually had fascinating findings about an intervention that backfired, right?
- 31:51Yes.
- 31:51That, um, when we think about parent involvement, for example, you shifted from looking at what the impact of a scholarship is for students to what the impact of, um, support for families to be more involved.
- 32:04Um, can you tell me about the two different arms of that experiment, the two different assignments that different families or different communities received, and what you learned in terms of what works and what might not work in supporting families to support their students, their children in learning?
- 32:20So this is another experiment in in Mexico, and the fundamental idea of that experiment is families perhaps do not have the information of how invest in the education of their kids.
- 32:35We believe that the family that get involved in the education of the kid, uh, can be very powerful and can increase the education of, of an individual.
- 32:45Uh, in United States, that idea is very prominent— families that get involved in the schools are, can induce, uh, the students to, to perform better.
- 32:56And in around the world, we, we try to look at that.
- 32:59So in Mexico we have this experiment, which is one group of families.
- 33:06We invite them to receive a light touch information session about how can they, uh, help the kids, how can they, uh, go to the school and talk with the principal.
- 33:20What resources the government had that they can reach out to improve the schools.
- 33:26Uh, that was one arm.
- 33:28The other one was an arm in which we give the parents association in general terms, uh, money, uh, so that they can invest in the school.
- 33:39So parents association in these schools are again, in rural area, very tough conditions.
- 33:46These families.
- 33:47Uh, receive, uh, money from the government, and we give them extra money.
- 33:52We say, "Hey, here is some money." And we give them the same type of information to those two families.
- 33:59So one is sort of directly to parents for something like homework help or making sure that your student, your child, is able to get to school on time...
- 34:09Yes.
- 34:10...and, uh, able to get home safely.
- 34:13And able to focus on the value of learning family specific support, learning at home as well as at school.
- 34:21Yes.
- 34:21And the other.
- 34:22Is more like a PTA, right?
- 34:24Yes.
- 34:25The parent associations that you said, oh, the PTA, instead of having to raise money through a bake sale as we would in the U.S., um, we're actually gonna double the amount of funds they have to be able to be engaged and involved with school.
- 34:38Is that about right?
- 34:39It is totally correct, yeah.
- 34:42Okay.
- 34:42So the experiment runs.
- 34:45The experiment runs, and three years later we tried to detect what were the impact of, of the experiment.
- 34:52Um, and the results are not that positive.
- 34:55Uh, in one sense, nothing happened uh, with the learning of the kids.
- 35:00The second thing that we find is that nothing happened with the, of course, with how the teachers, uh, behave in the classroom.
- 35:10And let me, let me pause here three seconds.
- 35:12When you give more money to the PTA, uh, the idea was that the PTA go to the school and say, "Hey, we have more money.
- 35:20What do we do with the money?" They cannot invest in salaries of teachers.
- 35:25They can invest in any other thing.
- 35:27But the idea was like,"When the parents have more money, there is a better communication with the parents and the school." And the parents can talk with the principal and they can say,"Hey, do you have a problem?" From physical problems, that is a window that is broken and they have the money to repair the window.
- 35:45Or that they need some sort of materials to run an activity in the class.
- 35:51So.
- 35:51More school supplies?
- 35:52Yes.
- 35:52More, you know, a field trip, something like that.
- 35:56Yes.
- 35:56So that the idea was that it will change the, the dynamic of the school, and it will improve the condition of the school.
- 36:05If the teacher has to arrive to the classroom and the classroom has the window broken, that is very different if the teacher arrive and the window is repaired.
- 36:13So it just make the teaching conditions for the teacher...
- 36:16Yes.
- 36:16...better?
- 36:17Yes.
- 36:17And parents would feel more empowered and engaged.
- 36:20Yes.
- 36:21That was the hypothesis.
- 36:22Yes.
- 36:22Okay.
- 36:22So then, you know, this is the beauty of experiments, right?
- 36:25What happened?
- 36:26It didn't happen.
- 36:27The teacher didn't change.
- 36:29Uh, something that, that, that was interesting was that in the money treatment, the parents were able to, what we described, sit in the table in the decision of the school more.
- 36:42They had a seat at the table.
- 36:43Yes.
- 36:43Okay.
- 36:44But they didn't, uh, but the school didn't change in a fundamental way.
- 36:49We were able to demonstrate, we were able to demonstrate that actually parents get more involved in the school.
- 36:56But again, no change in the learning of, of the students.
- 37:00So we were puzzled, we, we will say, "Hey, how come the parents are going more to the school?" And, but we found that everything pinned down to trust.
- 37:11So in the information experiment, we were able to increase trust between the parents and the school.
- 37:21In the money experiment, the trust decrease, and that is a potential mechanism to explain that result.
- 37:31So fascinating.
- 37:32So in the condition where parents are now supported to do homework help or to have more information about how to be engaged, those parents actually increased their level of trust.
- 37:43That intervention increased their level of trust in the schools and the school's sense of their being partners.
- 37:49Yes.
- 37:50But in the op in the other, it actually was negative, right?
- 37:53Yes.
- 37:54It was.
- 37:54It was what we call an iatrogenic effect, right?
- 37:57You actually generated an unfortunate consequence, right, which is now parents trust teachers less...
- 38:05Yes.
- 38:05...than the control group.
- 38:06Yes.
- 38:07And...
- 38:07Fascinating...teacher, and teacher trust less the the parents.
- 38:11The parents.
- 38:12And do you think that that was a generalizable result?
- 38:18You were talking about making people's lives better, right?
- 38:22Um, in this case, that mechanism suggests that if you, you know, come in with good intentions, but you're not careful about measuring the results, you could actually create a little bit more chaos in a school system.
- 38:38Eh, yes.
- 38:40And that is the $1 million question, and it's very difficult to answer.
- 38:47We don't know if these particular results are very context-specific.
- 38:54Uh, Mexico, we were talking, we are talking about Indigenous population.
- 38:59And there is a, that create a very context aspect of, of this intervention.
- 39:07Usually in Mexico it's very difficult for the parents to enter the school.
- 39:12They are not welcome, and that's why we want to do this—is we wanna...
- 39:17Ah, got it.
- 39:18...induce the parents to go to the school and, and we, we try to do that.
- 39:23And again, it didn't happen and, and actually backfire in the, in the money intervention.
- 39:29Uh, you asked me, do you think that that can be generalizable to our context— and that is more difficult.
- 39:37Yes.
- 39:37And, and, and, and I don't have an answer to that that.
- 39:40Perfectly.
- 39:41Which is fine.
- 39:41Perfectly reasonable.
- 39:42It's still very fine.
- 39:43You were being a very, very careful scientist, right?
- 39:46But I think it also speaks to the context that you're describing sounds like one, what we would wanna know more from different disciplines.
- 39:55You know, does there serve a purpose then to have a sociologist or an anthropologist or, uh, you know, someone who's really versed in Indigenous culture in Mexico to help us understand that context of trust and authority and organizations.
- 40:11It's fascinating to me to think about how each of us, through my lens as a Ph.D. in psychology, your lens as a Ph.D. in econ.
- 40:20Um, we can think about all the other lenses that we would wanna understand that context.
- 40:23It's fascinating.
- 40:25And I think that that's a very important point and I, I am totally believe in that point, which is what we need is more interdisciplinary work.
- 40:36We...
- 40:37again, these type of products require a lot of people.
- 40:41I have three other coauthors here is Harry Patrinos, Paul Gertler and a fantastic student of mine Nozomi Nakajima.
- 40:49We all are economists, and, and, and we perhaps will, will have a lot of help from other perspectives.
- 40:57We didn't have it.
- 40:58But I think that that interdisciplinary work is very important.
- 41:02Amazing.
- 41:03And so tell us a little bit, Felipe, about how you ended up here?
- 41:06How did you end up as an economist focusing on education in international contexts?
- 41:12You know, this is a way in which you've shaped your expertise and your talent and your viewpoints so that you can use these amazing tools, um, these econometric and experimental tools in these policy contexts.
- 41:27But I think it's really helpful sometimes to ask— what led you from life as a student to life as a scholar?
- 41:35I am going to, to tell you two things that are very concrete.
- 41:39When I was an econ, uh, a student in economics in Colombia, I was thinking about inflation.
- 41:46I was thinking it's to ...
- 41:46something totally different.
- 41:50And I arrived to a class with Gary Becker.
- 41:52And the first class of Becker was about how families invest in education and human capital.
- 42:00And that was my Ph.D., and it changed everything in my life.
- 42:05Uh, because I went out of that class and saying "Why I was thinking that inflation was interesting?" And I started discovering all these very interesting things about how families invest.
- 42:18So from that point, I start, eh, obsessing with that question.
- 42:22But then I went back to Colombia.
- 42:25And I start working in Colombia doing consultancy and research.
- 42:30I didn't enter to a university or because I believe firmly in research that have concrete impact in countries.
- 42:39So, uh, that link, that nexus between Pol Ed research and policy, for me was very important.
- 42:47And, uh, after that, uh, I arrived to the World Bank.
- 42:52And in the World Bank, it was the same idea— not only in Colombia, but in the whole world opened to me.
- 42:59Uh, my first, uh, task in the World Bank was in Pakistan.
- 43:03Then I start working in Mexico, then I start working in Cambodia.
- 43:08But the same tools apply to everywhere.
- 43:11And the same questions are, were very present in everywhere.
- 43:15So at the World Bank, I start working with a more global, uh, perspective.
- 43:21And then accidents happen, and I end up in academia.
- 43:25So that is, it's a very, my, my, my road to here, it's not a typical road of finishing your Ph.D. and entering academia.
- 43:36I have an exposure to to policy in one fundamental way— I was, I was not, never in the government, but I work very closely with government.
- 43:46Amazing.
- 43:47I think when we think about government, we, or we think about impact, we think, you know, the way to have impact is to enter government. And it's less clear, I think it's more opaque, um, that there are many ways to have impact on policy that don't involve being necessarily in a governmental public, uh, sector role, but, um, being, uh, adjacent to those roles and actually generating a ton of knowledge for those, uh, roles.
- 44:13The power of knowledge, and that require a policymaker that is willing to listen to knowledge...
- 44:23Well said.
- 44:24...which is, uh, increasingly difficult to find.
- 44:28But anyway.
- 44:31So tell me a little bit about, um, the huge amount of work that you've done on, uh, public-private partnership in Colombia.
- 44:38Because obviously, um, you have a, a very large corpus of work, um, that's happening in that country.
- 44:46But also, um, that I think serves as this remarkable, uh, level of, uh, both draws on probably your own, uh, personal experience, uh, being from that country, but also, uh, tremendously on your expertise, um, that you've just described so, um, elegantly.
- 45:04So tell us a little bit about those partnerships and, and what you're learning.
- 45:08So, when I returned to Colombia, uh, one of my first jobs was the World Bank contract to study a new type of schools that were, uh, being implemented in Bogota.
- 45:25Uh, these schools were public schools— receive money from the public—uh, but with two characteristics that are important.
- 45:32These schools are the provision of the service, uh, whereby, well reputed, well established, high quality private schools.
- 45:42Uh, so a private school that was very, very good school in Bogota, one area, uh, decide to also provide the service in another school in a very low income neighborhood.
- 45:55And that was the program that the government of Bogota implemented.
- 45:59The second thing is these schools, public schools, uh, couldn't choose their students.
- 46:03The government give the list of students to these, uh, providers, and these were a new experiment in Bogota, a new innovation in, in in education.
- 46:15And a...
- 46:16So just to slow us down for one second.
- 46:18Yes.
- 46:18So this is the equivalent of sort of aligned with U.S. charter schools?
- 46:24This is a private school operator, shall we say, who's now being, uh, funded by the government to provide services and educational supports, uh, run a new school, uh, for public school students who are attending a separate school.
- 46:42This is not a voucher program for those lower-income kids to attend the private school.
- 46:48This is a program where the, basically the operators of the private school are now being, are extending their service delivery to a new public school.
- 46:58Is that right?
- 46:58That's correct.
- 46:59Okay.
- 46:59With, uh, let me, let me say couple of things.
- 47:03One, again, these are private providers that have proven to be of excellent.
- 47:10Got it.
- 47:11So it is kind of a charter school, but if you are going to be the provider, you have to demonstrate that you are an exceptionally good, uh, private school.
- 47:22Got it.
- 47:23So high selectivity on who gets to have these contracts.
- 47:27The second thing is that the contract itself has element of quality.
- 47:32So you are not saying to the provider, "Hey...
- 47:36here are some kids, do whatever you want." Good luck.
- 47:38Yeah.
- 47:38Uh, they are saying,"Hey, there are some kids.
- 47:41First of all, they are low-income, all of them are low-income.
- 47:44And you have to prove that they can progress, they can learn, and that the probability of entering into university is high." So.
- 47:53Got it.
- 47:54There are, there are clauses in the contract that are quality-oriented, and the, the providers are amazing.
- 48:02So, yes, it is like a charter school, however, with a heavy emphasis in the contract and in the selection of the providers.
- 48:11Okay, so fast forward.
- 48:13What do we, what do we know about these, um, partnerships—success?
- 48:18Um, right? We've just talked about those, those moments where these, uh, policy innovations work and those moments where these policy innovations backfire.
- 48:27Uh, what, what are you finding?
- 48:29So my first evaluation was a long time ago.
- 48:32And I found positive effects, but that evaluation was not with the same techniques as we were describing of lotteries.
- 48:40And I keep thinking, "OK, we need to evaluate these schools in a more, more, more scientific way." And several years later, I was able to convince the local government to do a proxy of a lottery.
- 48:55Uh, still there is some targeting, still the kids that goes there are, uh, uh, chosen based off an poverty index similar to the one that we described in Cambodia.
- 49:07But, uh, what we did, we create pairs of kids that were objects in these indexes, and one received the offer to go to this school and the other not, ah, by chance.
- 49:18Okay.
- 49:18So pairs of kids.
- 49:19So you paired kids up, not to their knowledge, I assume. They didn't know they were paired.
- 49:24They don't know. They apply.
- 49:26Uh, the government has information about these kids, about, uh, these indexes.
- 49:30Yeah.
- 49:31You make two lists.
- 49:33We make the list.
- 49:34Flipping a coin.
- 49:35Yes.
- 49:35With money from the National Science Foundation, uh, we were able to measure the, uh, one-year effect of this program.
- 49:43And the effects are quite large, in any measure that, that you can measure.
- 49:49We are measuring kids that are entering this school— very young kids, uh, about the age of, of, of the kids that, that you study, Cybele.
- 49:59Eh, but they are entering the school, and the effects are impressive in terms of cognitive outcomes, in terms of socio-emotional outcomes, in terms of the satisfaction of the family.
- 50:13Uh, and these effects are, uh, are very large, very large.
- 50:16Amazing.
- 50:17And when you say large, help us to put that in context.
- 50:21I mean, many educational interventions show, you know, one or two points change on a given measure of kids' ability to learn vocabulary, for example.
- 50:31Or one or two months greater learning, uh, as if they had spent two or, you know, one or two more months in school relative to the children in the control group.
- 50:41When you say large effects, help us to understand what you mean.
- 50:47See, you asked very complex questions.
- 50:50So let's, let me start with social emotional.
- 50:53What we measure is 0.3 standard deviation in social emotional.
- 50:580.3 standard deviation in education intervention is in the, in the size that is large.
- 51:06Yes.
- 51:06In terms of cognitive is smaller, is 0.2.
- 51:11But still, uh, is, is is an important impact.
- 51:16We measure the Peabody Picture Vocabulary Test.
- 51:20PPVT.
- 51:22That measure has a, a scale that is around the mean was 90 points in my, my sample.
- 51:31And we increase it to around 97, 99 points.
- 51:36Great.
- 51:36So if you're raising kids almost 10 points... Yes.
- 51:39...on a standardized measure of vocabulary, that means they're showing a lot more... Yes.
- 51:43...ability in their language skills, right?
- 51:45Yes.
- 51:45And and it's a fundamental measure of cognitive skill.
- 51:48Yeah.
- 51:48Yes, absolutely.
- 51:49So amazing.
- 51:50So you've opened the watch.
- 51:52You've moved a bunch of gears.
- 51:54Because now it's not just parent involvement.
- 51:56It's not just scholarship funding.
- 51:58It's actually a completely different set of school policies and school quality, and you're finding big effects, right?
- 52:08So does that indicate that we shouldn't mess around with one gear at a time?
- 52:13Or does that suggest that ... how do we think about that in terms of school reform and think about in terms of school investments?
- 52:20Oh, that's interesting.
- 52:22So.
- 52:23You are right.
- 52:24Uh, the approach, uh, the approach that a lot of economists are doing and, and we were, uh, doing is we, we change one input at a time, one of the gears at a time, and we try to see what happened.
- 52:38However, these type of interventions are very complex interventions.
- 52:43It's not one gear, it's you change the watch.
- 52:46Yes.
- 52:46So, and that goes to the topic of the organization of of the education system.
- 52:53Is, it is not only you are going to, let's say, simple one, you're going to put more computers in a school.
- 52:59It is to the question of how do you organize a education system?
- 53:04Because we are talking about these private providers that provide public education, and that changed the system in fundamental ways.
- 53:14Uh, for instance, now the kids have more choice.
- 53:17A poor kid in a neighborhood, uh, can decide, OK, would I want to go to this school or should I go to the typical, uh, public school?
- 53:28So it changed a choice, but it also changed the way the government has to think about how to assign money, and that is a more systemic type of question.
- 53:41Felipe, thank you so much for joining me today on 'Quantum Potential.' Thanks to our listeners for tuning in.
- 53:48Remember to stay curious.
- 53:50Keep exploring new ideas.
- 53:52We'll be back next time with another guest who's pushing the boundaries of discovery.
- 53:56Until then, I'm Cybele Raver and this is 'Quantum Potential.' The 'Quantum Potential' podcast comes from Vanderbilt University.
- 54:14Your host is C. Cybele Raver.
- 54:16Thank you to our guest, Felipe Barrera-Osorio.
- 54:20Production by Matriarch Digital Media.
- 54:23For more information about Quantum Potential, go to vanderbilt dot edu forward slash quantum potential.
- 54:32Stay curious.