Latest / WonkyFolk / Vol 5 – Special CREDO Episode with Macke Raymond
Transcript
- 0:01Jed: Hey, Andy.
- 0:03Andy: Hey, Jed.
- 0:04How are you?
- 0:06Jed: I am doing well.
- 0:07Excited that we're going to have our first guest here at Wonky Folk.
- 0:12Andy: I know, and this is like a poor man's version of Smartless.
- 0:15We have a mystery guest.
- 0:16We'll get to that in a second.
- 0:18So for folks who, this is your first time or you've never watched this podcast, or
- 0:23you've been in the Taylor Swift concert and lost your memory, and so you need
- 0:26to be refreshed, this is Wonky Folk.
- 0:30And with Jed Wallace of Charter Folk and a leading charter consultant.
- 0:35And I'm Andy Rotherham of Bellwether.
- 0:37And today though, this will be a Wonky Folk that's going to go deep on charters.
- 0:43And the reason is because there was a big study that just recently came
- 0:46out, about charter schools, the third in a series and it's by CREDO
- 0:53at Stanford and it's important.
- 0:55And so Jed and I tried to figure out like who would be a really good person
- 0:59to have on, who incidentally will also be our first guest ever on Wonky Folk.
- 1:08And we were trying to figure out, who would be a great person to have on to
- 1:11discuss this study and its implications.
- 1:16Jed: And so we chose no another, and we were delighted to have an
- 1:21accepted invitation from Macke Raymond, who is the leader.
- 1:24Andy: We figured go straight to the go, straight to the source.
- 1:26So who better than the lead investigator and the leader of CREDO who's been
- 1:32on this issue in doing these studies for well more than a decade now.
- 1:37So, Macke, thank you.
- 1:38Jed: Welcome Macke.
- 1:40Macke: With that kind of buildup, it's have to be downhill from here.
- 1:43Andy: Now we're excited.
- 1:44I wore my, I don't know if you can see on the screen, I wore my charter shirt that
- 1:50I only wear for like, really important discussions of things like charter
- 1:53schools that are generally really tortured conversations and that's why we're excited
- 1:58that you're here, because you really do a nice job of sort of cutting through
- 2:03the rhetoric on all sides and getting to the facts of what's actually going on.
- 2:08Macke: Well, I'm delighted to be here.
- 2:10Jed: So I'm playing straight man to Andy Rotherham.
- 2:12That's a good one.
- 2:13But Macke, delighted to have you here for sure.
- 2:17I'm going to take the first question because I do think we're going to focus on
- 2:20a lot on charter school issues, but what's great also are the implications for your
- 2:24study as it relates to public education more broadly, all sorts of decision
- 2:29points, about what society is doing in terms of our commitment and our policy
- 2:33making as it relates to public education.
- 2:35So we'll go in a lot of different directions, but I want to start with
- 2:38kind of the heart of the matter and what are the takeaways from your
- 2:42standpoint, the most important ones that we should be looking at, as it
- 2:46relates to the performance of charter schools over a multi-decade timeframe?
- 2:51There are very few studies that have such a longstanding look at things.
- 2:57And also you've been through ups and downs.
- 2:59You've been at your moment when the charter school world
- 3:01was so frustrated with you.
- 3:03And there have been also this moment of like, "Hey, can you please do more?"
- 3:08And it's always a struggle to get the quality data and now
- 3:11you've been able to do it.
- 3:12So just very intrigued to hear what your thoughts are about the
- 3:17most important thing we should be thinking about given that multi-decade
- 3:20frame that you bring to the work.
- 3:23Macke: Sure.
- 3:24So great way to start.
- 3:26Thank you.
- 3:27So as I look across all of the data from all three of the studies, the
- 3:32thing that really stands out for me is the fact that there are so many, many
- 3:37schools in our sample that demonstrate improvement over time in a timeframe
- 3:44where that wasn't the case for the nation.
- 3:47And so when I think about what the charter proposition does for schools,
- 3:53I think it creates the right kind of incentives for the adults in schools
- 3:58to do the right thing by kids.
- 3:59And that is, for me, the largest takeaway.
- 4:03Andy: So, go ahead, Jed, please.
- 4:06Jed: Well, I wonder there are so many different findings that
- 4:10you have within your story.
- 4:12This idea of a sector that's grown to serve millions of kids, to be
- 4:17able to improve at that scope, it just seems pretty un unprecedented.
- 4:22But there are all sorts of sub stories, there are concerns.
- 4:27"Hey, we are doing as well with white kids.
- 4:29Hey, we've got our challenges as it relates to special education."
- 4:36What's the right way to present the study in its totality?
- 4:40What's the right metaframe while also maintaining, or encouraging the
- 4:46movement to keep enough integrity to keep focused on the things
- 4:49that still aren't good enough yet.
- 4:51Andy: And Macke, when you answer that's a really important question, particularly
- 4:54that last part, like for charter proponents, but also just for folks.
- 4:58I feel like some folks may be feeling like they're joining
- 5:00a play in the third act here.
- 5:01Just talk a little bit about the study, the method, how you do it with the virtual
- 5:06twins, and talk about this study, what's the timeline we're looking at, thing
- 5:09like help readers also understand what specifically we're talking about here.
- 5:14Macke: Okay, I'll start by answering Andy's first and then Jed, I
- 5:18may have to circle back to you and get the question refreshed.
- 5:21So, I also have to say, as a scholar of education policy in the United States,
- 5:29I want to put on the table that our team doesn't pick, as an advocate, any
- 5:35particular policy or program to endorse.
- 5:39We let the data tell us what the answers are.
- 5:42And so when we set out to examine the performance of charter schools, we
- 5:47wanted to know really what was the impact they had on student academic
- 5:52learning and progress over time.
- 5:54And so our entire method is focused on what do kids learn.
- 5:59And in order to do that study there's a lot of concern that you don't get
- 6:05to compare charter schools to district schools head to head like that because
- 6:10they might not have the same kids.
- 6:11So we developed a methodology where we created a very complicated matching
- 6:16methodology, I'll spare you all the details, but what we end up with is
- 6:20we end up with essentially twins, the only difference between the twins
- 6:25is that one student is enrolled in a charter school and the other is the
- 6:29other example is students who enroll in district schools that the charter school
- 6:35students otherwise would've attended if they hadn't gone to the charter school.
- 6:38So very, very tight geographic comparisons and very tight demographic comparisons.
- 6:45Based on that, we've been doing this for a really long time and we've seen the
- 6:51results go from really strong negative and statistically significant, which is
- 6:56what we came up with in 2009, the first time we did the study, to now in the 2023
- 7:03study, in both subjects reading and math, students in charter schools are learning
- 7:08more than they are than their peers in traditional district schools are learning
- 7:12in the space of a year, and we think that trajectory over the three studies
- 7:18that we've done, is evidence of the fact that schools themselves are improving in
- 7:24little bits and little bits and little bits, and over time, they actually have
- 7:27created these greater gains for students.
- 7:30That's an incredibly important finding.
- 7:32It's not just a charter school finding, it's an education finding,
- 7:35and we're really excited that we were able to see that in the data.
- 7:42Andy: Jed's question.
- 7:44Jed: Yeah, I started at C C S A in 2009, and we already knew from a variety of
- 7:51other data sets that we had some serious quality issues in the movement and
- 7:57two mandates get better in advocacy.
- 8:00But the other one find a way for us to begin to nudge the entire
- 8:04charter school sector toward greater performance and your study really put
- 8:10in repose, just how important that was.
- 8:14So my question for you now, going back to the one I've asked before, there are,
- 8:18there's the meta story, this incredible story of a multi-million student sector
- 8:25actually being able to improve itself.
- 8:28But there are sub stories as well.
- 8:30Where maybe it's not as good or maybe there are sub stories are even better
- 8:34than the meta story, but, How do you think we both keep the focus on the, I
- 8:41think, very positive meta story that's happened while also have the integrity
- 8:46to keep pushing ourselves in those areas.
- 8:48And everything gets so politicized, it seems to me like charter detractors,
- 8:54all they want to do is focus on our problems and charter advocates
- 8:58can err in the other direction.
- 9:00What's the right way to keep the mix here?
- 9:01Macke.
- 9:03Macke: Well, I think I can give you the thumbnail answer by pointing
- 9:06to the title of this third study.
- 9:09We have named this study as a matter of fact, the National
- 9:13Charter School Study three.
- 9:14And the reason that we did that was because of all of the thousands and
- 9:18thousands of times when we have heard some small slice of opinion or position
- 9:25about schools that are in the charter sector that just don't hold up under
- 9:30scrutiny, they don't hold up to the data.
- 9:32And so what I would say is, along with Andy's t-shirt, it's not
- 9:39just a completely vanilla story.
- 9:41I mean, there's very, very strong positive news that we see in the study that we
- 9:48have brought forward, and we really want to celebrate that, particularly
- 9:52in this post covid world where the need for evidence-based strategies for
- 9:59student success has never been higher.
- 10:02This is good news for everybody, but we also want to be completely honest about
- 10:08the fact that there are still pockets within the charter school landscape
- 10:15where there are students that are not learning as much as they should.
- 10:19There are pockets where schools are not performing as well as they
- 10:23should and we need to be completely transparent about that, as researchers,
- 10:29because that's what the data say.
- 10:32Andy: Yeah, I want to talk about those pockets, I
- 10:34appreciate the t-shirt call out.
- 10:35I think something that will surprise people in our extensive market
- 10:38research, people think people are coming to this podcast for
- 10:41the education policy discussion.
- 10:42It's actually our fashion choice, as it turns out, is what's
- 10:45actually driving listenership.
- 10:48Yeah.
- 10:49Which was a surprise to us as well.
- 10:52So you talk about those pockets, like dig in both in terms of like
- 10:56types of schools, types of kids.
- 11:00What are some things we should be attentive to?
- 11:02Because I do think one of the things I've respected about these studies
- 11:05since the stars, as you said, they don't flinch on the bad news, these aren't
- 11:10exercises in messaging and so forth.
- 11:12And so I think that's what makes them valuable and credible over time.
- 11:16But talk about what we did see some places here where there's obviously
- 11:20a lot of room for improvement.
- 11:22Macke: Sure.
- 11:23So let me just say, across the entire student population that we studied,
- 11:30we saw that there were particular student groups that did not learn
- 11:36as much as they would have had they gone to traditional public schools.
- 11:41I would call out here Native American students, even though they constitute
- 11:45a small percentage, the learning gains that they post are just not anywhere
- 11:49near what they need to be in order to prepare those students for the life
- 11:53that they will eventually inherit.
- 11:56The same is true for students who have special education needs.
- 12:00They are just not getting the kind of education in charter schools.
- 12:04And this has been a persistent finding across the studies that
- 12:06we've done, not just these national studies, but when we drill down
- 12:09into particular areas, particular geographies, we see this as well.
- 12:13So this is sort of a collective finding for the entire movement.
- 12:19And then finally, the one that just continues to distress
- 12:23are the student results.
- 12:24For students that are enrolled in virtual charters, the learning that they get
- 12:29there is so far dramatically under their peers in traditional district schools,
- 12:36that we just have to call it out.
- 12:39There's just no way around it.
- 12:41Andy: Yeah, that's been an ongoing issue.
- 12:43Talk about that, if a virtual school proponent were here to try to give
- 12:51them their due and steelman their argument, they would say: "Well, like
- 12:54there's a lot of variants and kids don't go to virtual schools because
- 12:56they were doing well previously.
- 12:58They go because they're struggling and they're looking for a fit.
- 13:01And so they're trying this thing."
- 13:04Talk about, why do you think that is?
- 13:07And to the extent you agree with this, why do even the strongest arguments
- 13:10from virtual supporters fall short?
- 13:14Macke: Well, I want to say, I'm gonna be drawing on not just this study, but our
- 13:21earlier study of virtual charter schools.
- 13:24And there we partnered with Mathematica, and also with the Center
- 13:29for Reinventing Public Education.
- 13:31So we brought a larger body of knowledge to that study than we
- 13:34were able to do in this study.
- 13:36The results continue to be concerning because a lot of the
- 13:39claims, just are not substantiated.
- 13:41The claim that the students all come to virtual schools because
- 13:46they have had X in the past and that X has actually changed over time.
- 13:52When we first started talking about this, it was that they were bullied.
- 13:57Then it was because they had some particular categorical learning
- 14:04challenge, then it's now because they're in post-traumatic stress.
- 14:10The argument continues to be it's a student-based problem and
- 14:14therefore we should be off the hook.
- 14:16And I just want to say, If you're a charter school, your job is to figure
- 14:21out how to meet kids where they are and bring their learning forward, and
- 14:24I don't care whether you're talking about a Native American population,
- 14:27whether you're talking about Hispanics or E L L or Special Ed or white
- 14:31kids in suburbia, I don't care.
- 14:34Your job, as a charter school, the reason that you've been given a charter is to do
- 14:39exactly what I just said, take kids from where they are and move them forward.
- 14:43And so after all of the time that we've put into virtual education, the fact
- 14:48that we have gotten results this time that are even worse than last time,
- 14:53and this is before the pandemic hit.
- 14:55I'm just boggled.
- 14:56I'm boggled why this is a conversation we continue to have.
- 15:01Jed: I think that my experience in this is that, if you can come up
- 15:07with minimum performance expectations that apply to charter schools.
- 15:11The vast majority, there are some that are in prisons and there are some that
- 15:16are working with certain populations of alternative students where maybe that
- 15:21we need to have something very specific and different, but across the vast swath
- 15:26of schools, it is possible to identify minimum performance expectations that
- 15:31would apply to absolutely all schools.
- 15:33And if the, if the movement and, and, and policymakers have the fortitude to
- 15:38simply enforce those, we will address, if we have disproportionate problem in
- 15:44virtual schools, you will see that those schools will face a reckoning or improve.
- 15:51And it's important for us to do that.
- 15:54I guess my question for you, Macke, is where do you think societal opinion is on
- 16:02the use of standardized tests, generally, are we wavering in our commitment?
- 16:08Is it harder for you to get the data that you need to do these kinds of tests?
- 16:13What's the future here?
- 16:14Are we going to be able to sustain enough focus on student learning such
- 16:19that your kind of studies can continue?
- 16:22And what does that mean for society more broadly if the answer is no?
- 16:26Macke: Society more broadly.
- 16:27Let's talk about the future of CREDO.
- 16:29Both of these things align in this question because the work that we've
- 16:35done for the last 20 years, requires that we be able to access student level
- 16:40data of a preponderant number of students in each of the places that we study.
- 16:46So states necessarily need to be educating the majority of students.
- 16:51In public schools, in environments that are tested on a regular basis.
- 16:56A number of those assumptions are starting to not necessarily hold true.
- 17:01So we see in the most recent enrollment data that there are a lot of students
- 17:07who are electing not to be involved in the public sector for education.
- 17:11We see a lot of legislative activity that suggests that we're going to
- 17:15see more and more students taking advantage of alternative pathways to
- 17:19education, whether it's homeschooling or micro schools or private schools
- 17:23or whatever, through the ESA programs.
- 17:25So that in addition to a sort of a, shall I say a softening of the commitment
- 17:34to testing students on a regular basis that we see in a number of states.
- 17:38We're not convinced that the record about what kids know is
- 17:42going to continue into the future.
- 17:44And I think that's a real threat to the kinds of performance expectations that
- 17:50we need to be able to place on schools, especially now, and so I have the concern,
- 17:59not just for my team, but for the nation as a whole that, while we have a active
- 18:06conversation about whether testing is good or how to make testing better, I would
- 18:11like to make sure that we don't lose the signal that we do get from testing until
- 18:16we have something better to replace it.
- 18:18Andy: So, yeah, I agree, and on the virtual school question, it seems
- 18:21like this is the kind of schooling that works well for a real subset
- 18:24of kids, but has been scaled quite a bit and is serving more kids than it
- 18:29probably is going to serve optimally, and that may be one of the reasons
- 18:32that we're seeing these results.
- 18:34Because you do see virtual school can work for some kids, but the scale it's
- 18:38at and then the bellwether charter deck a few years ago, this was a very clear
- 18:41finding, both on virtual schools and also states that had a lot of virtual
- 18:45schools, their achievement was getting depressed overall as a result of it.
- 18:50So you do see this.
- 18:51I want to go back though to this question on Native Americans,
- 18:54because I think it's fascinating.
- 18:55You have the tribes are actually starting their own charter schools
- 19:00and a real area of concern has been native kids in district schools we're
- 19:05more than 90% of them go to school.
- 19:06Like I think people can think that the BIE schools and so
- 19:11forth are a larger section there.
- 19:15Most of these kids are in district schools and there's been a move that
- 19:18they're not being well served there.
- 19:19So how do you address that and put them in schools that are going to be
- 19:23more culturally affirming, be better and yet we're seeing these results.
- 19:27So talk a little bit more about that Mackey, like what are your
- 19:29hypotheses for why we're seeing that?
- 19:32How should advocates be thinking about that?
- 19:36Macke: So I think it's a really important sector of concern for
- 19:41the charter school community.
- 19:44My sense is that the models that have been introduced in the Native
- 19:51American serving schools that we see, are not culturally affirming.
- 19:56I would also say they're not necessarily sustainable in the
- 20:00environments in which they're placed.
- 20:02And so I think that there's a fundamental mismatch between the schools that enroll
- 20:08Native Americans in charter schools.
- 20:09I should note that the schools that have Native American students, there are
- 20:14very few where the Native Americans are themselves the majority of the students.
- 20:18So we're still seeing the same problem that you just mentioned for the district
- 20:22schools, where we're seeing small slices of the student population be
- 20:25Native American and slotting into other charter schools, across the country.
- 20:30And I think there it's very easy for students to get missed and to
- 20:37not be the center of attention.
- 20:39So the reservation and tribe based charter schools are trying
- 20:44to do a different thing, which I applaud and I'm looking forward to
- 20:48learning more about as time goes on.
- 20:50Andy: That's great.
- 20:51That's really interesting.
- 20:53Talk about one other subset of kids, you know, a big push you hear in the charter
- 20:57world politically, and something Jed and I have actually talked about on this podcast
- 21:01is, how do you build a suburban political strategy and how do we keep charters right
- 21:06now are still disproportionately sort of a function of urban politics and haven't
- 21:10really penetrated the suburbs, as much as how do you build a broad based movement,
- 21:15yet some of the results that you find seem to raise questions about that as a sort
- 21:20of efficacy strategy in terms of actual, as you said earlier, actual teaching and
- 21:23learning, leaving aside the politics.
- 21:26So how should we think about those results?
- 21:29Macke: So the suburban results that we see are literally in
- 21:35comparison to suburban schools.
- 21:37And so when we don't see a necessary advantage to going to a charter
- 21:40school in that specific achievement context, that might not be a bad thing.
- 21:45Let me just say, we would always want to see charter schools doing
- 21:50strong academic progress for kids and when you look at most of the
- 21:55suburban schools, you will see that.
- 21:57What you'll see though is that they're not quite as strong as the schools that
- 22:00they otherwise would've attended, and that's why we get the result that we do.
- 22:04Having said that, there are a large number of cases that suit a charter school
- 22:12presence in suburbia where you have huge schools, at the high school level, where
- 22:17you have a particular run of schools from elementary to middle to high school
- 22:24that may not actually provide a kind of flexibility that families are looking for.
- 22:29Those are contexts in which charter schools can actually enter and be very
- 22:34successful because they're offering choice, they're offering flexibility,
- 22:37they're offering the opportunity for a different education environment,
- 22:42and what we see, in places where the suburban education has been really
- 22:50high flying, but really high stress, that's the place where the charter
- 22:54conversation has become very vibrant.
- 22:57Andy: That's interesting and I've always thought the customization and
- 23:01sort of choice and options part is a key thing in the suburbs, it's almost
- 23:05like a Maslow's hierarchy of schooling, and so if like your basic schooling
- 23:09needs are being taken care of, then you start thinking about, "I might want
- 23:12core knowledge, I might want arts."
- 23:14And that sort of customization is a key selling.
- 23:18And actually, I think over time makes public education more resilient because
- 23:22instead of trying to keep people in who want something else, you're giving them
- 23:25lots of choices and people feel more bounded invested in the entire project.
- 23:30Macke: I think that's a great point.
- 23:32Jed: Macke, can you talk about the composition of students
- 23:35who are in virtual settings?
- 23:38My intuition is that we would have a larger percentage of those
- 23:43students be Caucasian than in other classroom based settings.
- 23:48And that could be skewing the results as it relates to white kids.
- 23:52But is my intuition wrong here or not?
- 23:54Macke: No, you're right.
- 23:55And for those of you who would like to really like totally geek out, we
- 24:00have a website of all of the results of our study that are available
- 24:05for you to go and poke around with.
- 24:06And if you look at the demography of the charter school landscape, the
- 24:11demography of virtual charter enrollees is substantially more White than
- 24:16charters or their district schools.
- 24:19Lower proportions of Black students by a great deal.
- 24:23And lower proportions of students in poverty.
- 24:26And so there is a sort of an evolving archetype of what a typical virtual
- 24:32charter school student looks like.
- 24:34I will warn that there are contexts across the country where that is not the case.
- 24:40Jed: Yeah.
- 24:40It's my sense that this is one of the responsibilities that is going to fall
- 24:43on charter school advocates, which is to continue to advocate for this
- 24:48kind of data being available and for schools to be held accountable to it.
- 24:54I think we find ourselves in a challenge because I think there have been all
- 24:57sorts of instances where data has not been used wisely and not used fairly.
- 25:03And so anytime you end up defending something like that, you run the risk
- 25:08of undercutting your own credibility.
- 25:10But more for the long term, for the long term.
- 25:15There's have to be some voice and Andy maybe you've got other ideas
- 25:18about where else, we can get some spine to keep this kind of testing
- 25:27data available in the landscape.
- 25:29Andy: Jed, that's a great question and you would think like in general
- 25:34people would want as much information and as much transparency, but
- 25:37as we know, that's not the case.
- 25:40Um, and so I do think we're under a lot of pressure here.
- 25:42We're seeing even like post pandemic, it's been a struggle to get test score
- 25:45data released in a lot of places.
- 25:47And I think in some of those places, and we talked about a few weeks
- 25:50ago, New Jersey, I think one of the reasons maybe that it told a pretty
- 25:53good story about charter schools and a less good story about traditional
- 25:56public schools and there's obviously politics around all of that.
- 26:01So I think this is a big problem and it's something that everybody
- 26:03needs to pay attention to because without good data, this just becomes
- 26:07brute force sort of politics.
- 26:11So Macke, I kinda want to ask you related to that though.
- 26:14Like I follow these since the beginning.
- 26:16I remember the 2009 study and you got a lot of grief because there's
- 26:19certainly in the charter world, there was a sort of hear no evil
- 26:22speak, no evil, see no evil faction.
- 26:25That was like, what, you know, essentially kind of argued you were like turning your
- 26:28guns on the wrong lines or something.
- 26:31And I think what gives these study credibility again, as we said
- 26:34earlier is how it arcs over time and the results have changed.
- 26:38But how like it seems like we're still heavily anchored on, I still hear a
- 26:44lot about the 2009 results when you listen to advocates, it's like they're
- 26:48like frozen amber or something, and like this one and the last one for
- 26:54that matter didn't seem to get quite as much attention, which is striking if
- 26:58you just step back and are like, we've created this enormous cohort of schools.
- 27:02If you think about sort of the history of public education, this is one
- 27:06of the more ambitious experiments and innovations historically that
- 27:10we have seen in this country.
- 27:12And these results keep coming in and they're changing.
- 27:15It's an evolving story.
- 27:16You would think they'd be more interest and curiosity, and yet I just didn't
- 27:20see a lot of attention, maybe it's the old adage, if it bleeds, it leads.
- 27:24And so good news, or somewhat good news.
- 27:27Didn't get the attention, but I'm just one observer and you
- 27:30were at the epicenter of it.
- 27:31How did you experience, like, is there a changing appetite around like, you
- 27:37know, what kind of news, the level of information, like what's, what's going on?
- 27:42Macke: So I want to make a distinction between folks who are
- 27:45actually engaged in the crafting of policy and the public media.
- 27:49And I think the statement that you just made applies pretty
- 27:53broadly to the public media.
- 27:57We have learned from our past studies that a lot of this is a slow burn that
- 28:03particularly as we continue to talk about or place op-eds in particular places other
- 28:11news vehicles start to pick up on it.
- 28:13So I'm a little bit surprised at the media reaction this past week, but
- 28:19I'm not ready to pack the tent yet.
- 28:23On the policy leadership side though, I have to say, t's been a
- 28:27very different reaction this time.
- 28:31Most of our state partners require us to do briefings with them before
- 28:36the report is released, and of course we would want to do that anyway.
- 28:39And the conversations that we've had in those settings have been
- 28:45much more constructive, I would say, than we've seen in the past.
- 28:53We've been asked to be much more available to share the results
- 28:59and to discuss without making any policy recommendations, discuss the
- 29:04implications of what we've learned.
- 29:06And so lots of folks in the policy arena are handing us off so that
- 29:11they can create a community of folks who know the findings.
- 29:15And I, I look at that as a very positive development.
- 29:18Andy: Yeah, that's actually, there's like a robust set of folks doing that,
- 29:20that's extremely encouraging just in terms of how issues evolve that and
- 29:25is that bipartisan and is that, are you seeing like, say a little bit
- 29:29more about that because like, that's probably the most encouraging thing we've
- 29:33talked about in this whole discussion.
- 29:36If that's happening in a number of places.
- 29:39Macke: So, one of the things that's happened over the long term that we've
- 29:45been doing this research and as you spoke about keeping these data relationships
- 29:49going, is that I think we've actually worked ourselves into, I wouldn't call
- 29:56it allies necessarily, but at least respected colleagues in a number of
- 30:01places that you would not normally think we would have any kind of traction.
- 30:06And so there are quiet conversations that are going on in places that you
- 30:10wouldn't expect, and surprising to me, lack of conversations in some of
- 30:15the places where you would expect it.
- 30:18So we're tracking this sort of one day at a time as it comes and really
- 30:23trying to make sure that the focus is on looking at what the evidence says
- 30:30and what it portends for future decision making for K12 in the public space.
- 30:36Jed: I like that.
- 30:37Focus on what this portends.
- 30:39Because I've had conversations with several advocates whose
- 30:42state sectors are just nailing it.
- 30:44And it could very well be that you can make a big deal about
- 30:47the results that are here.
- 30:49But they also know they are sobered by their own data about what's happened
- 30:54to charter schools during Covid.
- 30:56And a lot of those supports that charter schools were using to generate these
- 31:00kinds of results were the exact supports that were most affected by the pandemic.
- 31:06And so our folks are saying, don't make too much hay about
- 31:09it right now, cuz our data might not look as good a year from now.
- 31:12Do you have anything that you've seen as it relates to this data and heading into
- 31:18Covid that would help us understand what might be a coming, because everybody's
- 31:23got their guess on what's gonna happen with charter schools through the pandemic,
- 31:26but do you have any inclinations there?
- 31:30Macke: So we don't have the kind of longitudinal student level data that
- 31:35we have used in these studies, but during the pandemic we actually did
- 31:38do a fair amount of research about what was going on in charter schools.
- 31:43And we zeroed in on three states, three very different states, I should say.
- 31:48And what we found was that, many of the things that we had believed
- 31:55before about what made charter schools different were evident in spades.
- 32:01And what I really mean here is, what we saw was a much faster responsiveness of
- 32:07school teams to the changing conditions of the schooling that they faced, that they
- 32:16literally turned within a few days to an alternative learning model and insisted
- 32:22on learning as they went along and continuous improvement as they went along.
- 32:27That was something that we didn't see in any of the comparative
- 32:31examples that we delved into.
- 32:34I also think that there was a real commitment that they identified themselves
- 32:41without being prompted to being first responders and took on that mindset, and
- 32:46so when we talked to school leaders and when we talked to educators and we talked
- 32:50to parents, we got this consistent message of these guys are frontline fighters.
- 32:55And I would say that throughout the pandemic, I think universally
- 33:00we're gonna see learning loss when we get those data and actually can
- 33:03nail down what that looks like.
- 33:05I think we're going to see learning loss no matter where we look.
- 33:08But what I would say is I think both, the magnitude of that I
- 33:13anticipate would be smaller.
- 33:15And I think the resiliency of those teams to get back on track is probably higher.
- 33:22Those are my predictions.
- 33:24Andy: Interesting.
- 33:24And you square out that we've got some very preliminary
- 33:28data in the their findings.
- 33:30And so that's what you're sort of grounding that, as
- 33:33well as what you're saying.
- 33:35Macke: Well, and through some of the formative assessment providers that we
- 33:43partner with, we can see some of those interim results starting to come through.
- 33:48So I don't want to get too far out in front of the data, because frankly there's
- 33:53not enough of it to be able to say things with sort of with the kind of authority
- 33:57that these other studies provide.
- 33:59But the early indicators are pretty encouraging.
- 34:03Andy: So let me ask you a question about just the funding of this project and
- 34:06what of portends Macke,.You've been able to do this for a long time and it seems
- 34:11like maybe the real risk was after 2009 people just like throw in the towel and
- 34:15stuff and instead you've been able to keep these studies going, and as you
- 34:18said, and we'll put in the show notes.
- 34:20There's studies of particular states, there's studies of cities, it's a
- 34:26treasure trove of data on charters.
- 34:29And we've had over the last 20, 25 years, a problem in the sector of funders will
- 34:34quickly lose interest in something, move on often before the results are even in.
- 34:39I mean, like the Gates Small Schools Initiative is sort of Exhibit A, but
- 34:42you also saw that around some, around teacher evaluation, some other things.
- 34:46Macke: Small interval challenge.
- 34:48Andy: Yeah, exactly.
- 34:49You've been able to like, keep with this and keep it going, which is, you
- 34:56know, enormously important because the learning does happen over time,
- 34:59this is not sort of a point in time thing, it's an overtime thing.
- 35:01So talk about like, how have you been able to do that and
- 35:06what should we be doing more of?
- 35:07What are the broader implications for the field, even beyond
- 35:11charters and so forth of this kind of work you've been able to do?
- 35:16Macke: So it's a wonderful question.
- 35:19And when we put the results of the three studies together and we
- 35:23saw the upward trend across all of these three studies that emerged.
- 35:30For us, it wasn't a wow, look at that charter schools are really pushing it
- 35:37here and they're doing really well.
- 35:39Obviously we were happy to see that result and we are happy to
- 35:43bring those findings forwards.
- 35:45But for me, the real insight there was policy takes a really long time to mature.
- 35:51And it takes an endurance that I don't know is well matched to the context.
- 35:59And so you asked about the funders, I have to give credit to the legislatures that
- 36:04continue to endorse the policy over time.
- 36:07I give credit to the school teams that continue to battle on when they're faced
- 36:13with the kind of political uncertainty that they've faced over time, and I really
- 36:20appreciate the authorizers who have been diligent about applying new standards
- 36:26in increments so that, the incentives work in the directions that they do.
- 36:31So for me, what we've got here is we've got a collective mix of a lot
- 36:36of different stakeholders who are in it for the long haul and are willing
- 36:40to stand tough over the long haul.
- 36:42And that, for me is the big insight of that trend line that we see.
- 36:47Andy: It's ironic though, right?
- 36:48Because it's happening on an issue that is particularly
- 36:51contentious, has become partisan.
- 36:53I Think like a naive hypothesis would be this would be the kind of
- 36:57issue where that wouldn't happen.
- 36:58It might happen on stuff that's a little, where there's a little bit more of a
- 37:02different kind of foundation, different kind of political and policy arrangement.
- 37:06So I think it's a super encouraging story because it is a little bit of, if you can
- 37:11make it here, you can make it anywhere.
- 37:12If we can keep, if we can keep this kind of subsystem kind of working in
- 37:17terms of policy and learning on an issue like this, like that's hugely
- 37:21encouraging on other issues that we think about, like standards, accountability,
- 37:25even school finance, other, other issues where the politics are a
- 37:29little bit different to start with.
- 37:31Macke: Yeah, I think that's great.
- 37:32I also don't want to minimize how much effort it took to keep the game going.
- 37:36So, let me just say, a lot of feet on the line standing in the fire.
- 37:44So, and guys, I'm really sorry, but I have my annual meeting
- 37:48with the boss in four minutes.
- 37:51Andy: That's alright.
- 37:52It's a great note to end it on, the mixed metaphor of feet in the fire.
- 37:57But no, that note you just ended on how much hard work it was.
- 38:00I mean, just.
- 38:01I know I speak for Jed when I say this to speak for us.
- 38:03Extremely grateful to you for this work.
- 38:05This is, this, this is really substantial.
- 38:08My team knowing that the two of you are giving that kind of praise, my team's
- 38:12going to get really excited to hear this.
- 38:13Thank you.
- 38:14I'll pass that on.
- 38:15Jed: I think that multi-decade view is becoming one of the most important
- 38:21and it's the only view that helps us suss out what's really going on.
- 38:26So for everyone that's doing it, you know, we should offer special
- 38:29thanks and Macke you, no one's doing it to the level that you are.
- 38:31So thank you for that and thanks for being with us here.
- 38:35Macke: All right, Andy, it's good to see you, Jed.
- 38:36I'll see you next week, right?
- 38:39Jed: Yeah, for sure.
- 38:40For sure.
- 38:40Look forward to seeing you in Austin.
- 38:42Good luck with with your meeting here in a few minutes.
- 38:45Andy: You can tell your boss that we either loved it or hated
- 38:47it, depending on what will help you, what will help you the most.
- 38:51We found sometimes, sometimes it's our praise or our criticism
- 38:55sometimes helps people more.
- 38:57I
- 38:57Macke: I'll report back.
- 39:00Jed: See you back.
- 39:04Andy: Hey, that was great.
- 39:05What a good first guest like really substantive and important issue.
- 39:12Jed: Yeah, I've learned so much from Macke over all our decades of work together
- 39:16that to see her continuing to unearth things that no one else is finding.
- 39:22It's just really great.
- 39:24And I don't sense her own personal desire to keep going waning at all.
- 39:30It's just, can we keep the soci societal conditions together such that this really
- 39:34important work can be brought to light, or these findings can be brought to light.
- 39:39Andy: But I think it's encouraging.
- 39:40Look, she's obviously, she's someone who's disposed towards
- 39:43reform, I mean that's clear.
- 39:46But the fact they've been able to do this work, shoot it straight over the
- 39:48years, talk about the good and the bad, even in this study, it's not all good.
- 39:53It's such an encouraging thing when there's so much pressure on
- 39:56everybody to just pick a side.
- 39:58And you're seeing people on charters who are like, they're not really against
- 40:01charters, but the politics the way they are, they're definitely throwing their lot
- 40:04in with people who are against charters.
- 40:06And then likewise, you've got people in the choice community who have never
- 40:09like really stopped to question a lot, that they're just always are for choice.
- 40:13And to see someone like Macke who does bring real nuance to it, to see her be
- 40:17able to produce this work and succeed at it like she has, it's really encouraging.
- 40:23Jed: And this last report had 2 million virtual twins.
- 40:27It's 4 million data sets.
- 40:29They're students who's who are being looked at, right?
- 40:32So I think that's a part of what the credibility is for her.
- 40:35Hey, it's having done it over many, many years.
- 40:37But also to have a scope of statistical undertaking that just
- 40:42becomes very difficult to refute.
- 40:45So hopefully, those policymakers who are getting those individual briefings from
- 40:50her will take this stuff to heart and keep the resolve necessary to allow further
- 40:55studies like this to happen going forward.
- 40:57Andy: Yeah.
- 40:57Well look, just speaking as a guy who only has two twins, I can't
- 41:01even get my head around that.
- 41:03That is a lot of twins.
- 41:05Well this was great Jed.
- 41:06I look forward to seeing you in a couple of weeks.
- 41:08I hope you enjoy Austin.
- 41:10I know you're off to there.
- 41:12Jed: Yeah.
- 41:13Looking forward to conference and looking forward to talking
- 41:15on the other side of it.
- 41:15So take good care to them, Andy.
- 41:17Andy: You as well.
- 41:18See you soon.
- 41:19Jed: Okay.
- 41:19Bye-bye.
- 41:20Andy: Bye.