Latest / DiveIn: Diving into Special Education's Most Complex and Pressing Debates / The Disproportionality Series Part 1: A Conversation with Paul Morgan

Transcript

The Disproportionality Series Part 1: A Conversation with Paul Morgan · DiveIn: Diving into Special Education's Most Complex and Pressing Debates

• 0:08
  From the Division of Research of the Council for Exceptional Children,
• 0:11
  this is Dive In. I am your host, Federico Weitholler,
• 0:15
  professor at the University of Illinois at Chicago.
• 0:20
  Welcome, welcome, welcome, welcome to the ninth episode of the
• 0:24
  third season of Dive in. Today we will continue to discuss
• 0:28
  a highly controversial topic in our field,
• 0:31
  the disproportionate representation of minority students in
• 0:35
  special education. A couple of episodes ago, in episode seven, we discussed
• 0:40
  with Dr. Catherine Vulgarides the legal aspects of disproportional
• 0:45
  representation and how states are responding to legal mandates
• 0:49
  to address this issue. If you haven't listened to that show, I recommend that you
• 0:53
  go back and listen to our episode seven. Today we will begin
• 0:58
  a series that explores one of the most contentious issues surrounding
• 1:02
  disproportionality. Does it actually exist?
• 1:06
  Are minority students over or underrepresented in special education services?
• 1:11
  The idea of the series is to bring disparate perspectives and
• 1:15
  approaches to this issue and by the end of the series to have some,
• 1:18
  some sort of synthesis and ideas to bridge the disproportionality
• 1:23
  divide. And to start the series, I have a conversation with
• 1:27
  Paul Morgan. He is the Empire Innovation Professor,
• 1:31
  Social and Health Equity Endowment professor at the Department of Health Policy
• 1:36
  Management and Behavior at the College of Integrated
• 1:39
  Health Sciences at the University of Albany,
• 1:43
  suny. If you are tuned to the heated debates about disproportional representation
• 1:47
  of minority students in special education, well,
• 1:51
  his name should be familiar to you. Since the 2000 and
• 1:55
  tens, Dr. Morgan and his colleagues have published a series of articles contradicting
• 1:59
  decades of research in special education that point to
• 2:03
  the overrepresentation of minority students in special education.
• 2:06
  Dr. Morgan's work has received both praise and support,
• 2:11
  as well as severe criticism for his framing of disproportionality
• 2:15
  and his methods. In this interview, we engage in a
• 2:19
  lively and critical discussion about his findings, his methods and
• 2:23
  recommendations. But before we start, I wanted to share with you
• 2:26
  the exciting news that I started a Discord channel where
• 2:31
  we can interact and discuss past and present shows.
• 2:34
  I have started actually a specific link for the Disproportionality
• 2:39
  series where we can engage in scholarly and respectful conversations
• 2:43
  about the topic. I look forward to hearing from you and having that
• 2:46
  moment of connection. I will be putting the link in the show description.
• 2:50
  So are you ready to start with the Disproportionality series?
• 2:54
  Let's dive in.
• 2:58
  Thank you so much, Paul, for being here with us today. It's a pleasure to
• 3:02
  have you. It's the pleasure of all mine. Thank you so much for the Invitation?
• 3:05
  Yeah. Thank you. I know you've been doing a lot of work on this proportionality,
• 3:10
  some of them with supporters, some of them with people that kind of contest
• 3:14
  your work. So I wanted to have you here because
• 3:18
  we're starting a series on disproportionality and to have different approaches
• 3:22
  and different ways to see the issue. So let's start with a very simple
• 3:25
  question. We have Catherine Bulgaris a couple of weeks ago. She kind of defines
• 3:28
  proportionality, but more from a legal perspective. I want
• 3:32
  to know how you define disproportionality on your work.
• 3:36
  So we would. So my definition would be oriented
• 3:41
  towards some descriptive evidence of differences, say in
• 3:45
  percentages, shares, proportions,
• 3:49
  in terms of receipt of special education services
• 3:53
  that vary by race ethnicity. So as
• 3:57
  a simple example, if descriptively,
• 4:01
  say Black students represent 14% of the general education population,
• 4:05
  but say 17% of the special ed population,
• 4:08
  there might be some evidence there of a
• 4:13
  descriptive evidence of a difference, of a disproportion.
• 4:16
  And that's generally what the federal government means by disproportionality
• 4:21
  or significant disproportionality is some descriptive evidence
• 4:25
  of differences in proportions by
• 4:29
  race and ethnicity who's receiving special education services.
• 4:33
  Okay, then we'll get more on the. Because I know you go a little beyond
• 4:37
  that, those descriptives, right? Because I know in your
• 4:41
  research, in some step of your research, you find
• 4:45
  proportionality. But would you go a little farther on your analysis? You don't find
• 4:49
  it. So we can get into that later. But I'm thinking
• 4:53
  also on that, right. You wait to define this proportionality. Not. So I'm
• 4:57
  empty. Descriptive. My understanding. Right. I may be wrong. But minus. And not
• 5:01
  just descriptive, but you also look at ways
• 5:05
  that we can dissolve some variables and compare similar situated
• 5:11
  studio. Know how you call it. And then where you, where you. Where your.
• 5:15
  Where are your argument of this proportionality, Is that fair to say?
• 5:18
  Yeah, yeah, I think that's kind of a helpful way to start
• 5:23
  the conversation. So maybe,
• 5:27
  maybe I can give a little bit of background context and then maybe some sort
• 5:31
  of ways to approach this. So ever since special education started,
• 5:35
  there's been concerns that the way identification
• 5:39
  is functioning may have some degree of racial bias in it.
• 5:43
  And we clearly don't want that. We want children who have
• 5:47
  disabilities to receive services that they're
• 5:51
  entitled to and would benefit from, but we don't want to be misclassifying
• 5:56
  children as having disabilities if they don't. So we're trying, trying to strike
• 6:00
  this balance in which we're not identifying children as having disabilities
• 6:04
  when they don't. And we are identifying children who have disabilities and
• 6:08
  that might benefit from special education services.
• 6:12
  And so that's kind of the balance we're trying to strike now.
• 6:16
  Ever since the field started, there's been some descriptive evidence that
• 6:20
  there are differences by race, ethnicity, and who's receiving special
• 6:24
  education. And you can find that in many reports
• 6:28
  by the federal government, going back some anecdotal
• 6:32
  reporting, even the most recent reports to Congress
• 6:36
  by Department of Education, OSEP will find some
• 6:41
  descriptive evidence of differences by race ethnicity in
• 6:45
  who's receiving special education services. And there's also some
• 6:51
  attention as to identification, but also placement
• 6:55
  and discipline, which we can talk about if that's of interest. Interest. But around identification,
• 7:00
  we've had these descriptive evidence of differences. So if
• 7:04
  you look at the most recent report to Congress by osep,
• 7:07
  you'll find some evidence that black students.
• 7:12
  They. Have a greater risk, if you want to characterize it that way,
• 7:16
  of receiving special ed. So their risk ratio might be like 1.3.
• 7:20
  So numbers above 1 indicating the group is more likely and numbers
• 7:24
  below meaning they're less likely. Okay, so.
• 7:27
  And we find that too. Okay. When we look at
• 7:31
  different data sets, we find indication that in an unadjusted
• 7:36
  estimates, just descriptively, there are differences by
• 7:40
  race necessity in who's receiving special education or who's being identified
• 7:44
  as having disabilities. Okay, so that's one thing, and I'm
• 7:48
  not really disputing that. I mean, we've reported that multiple times ourself.
• 7:52
  That's been reported in many reports to Congress.
• 7:55
  That descriptively seems to be the case.
• 7:59
  It may vary, maybe in particular contexts. Most of the
• 8:03
  reporting is just using national aggregate statistics.
• 8:07
  But this is found over and over again.
• 8:11
  And we might characterize that as sort of representation evidence
• 8:16
  who's represented in special ed versus
• 8:21
  in general ed. And in terms of representation, you find repeated
• 8:25
  evidence of differences. Now,
• 8:29
  the reason I'm saying differences is because sometimes people talk
• 8:34
  about disparities, and I think they're using that term loosely
• 8:39
  and often inaccurately. So the way that term
• 8:42
  should be used according to the pseudo medicine in a 2003 report
• 8:47
  is not because of differences by
• 8:51
  clinical need or patient preferences, but instead by provider
• 8:55
  discriminatory practices or inequities in how this healthcare
• 9:00
  system is working. So I want to differentiate descriptive
• 9:04
  evidence and differences and representation from
• 9:08
  disparities that takes into account clinical need
• 9:13
  and perhaps patient preferences. I also want to
• 9:17
  introduce another term, which is identification. So when we're
• 9:20
  talking descriptive evidence over under Representation,
• 9:25
  you know, just, it's descriptive, it doesn't allow for much
• 9:29
  in the way of inferences. Okay. When we start talking about disparities or
• 9:33
  we start talking about identification over or under,
• 9:37
  for that conversation, we should be attempting to make some
• 9:41
  accounting for clinical need or comparing children
• 9:45
  who seem similar in terms of their likelihood
• 9:49
  of having disabilities. Okay. And that's where our work
• 9:53
  really comes in because we've tried
• 9:57
  to account for many indicators
• 10:01
  of potentially having a disability,
• 10:05
  most particularly relevant whether the child seems to be struggling academically
• 10:11
  in school. When you compare children who seem otherwise
• 10:16
  similar, you get very clear and replicated
• 10:20
  and repeated evidence that children of color are less likely than
• 10:25
  white or English speaking students to be identified as having disabilities.
• 10:29
  And we find that prior to school
• 10:33
  entry, we find that after school entry, including elementary, middle and high school,
• 10:37
  we find out whether we use school record,
• 10:40
  parent report of disability, we find it for
• 10:44
  disability in general, and we find it across many different specific disability
• 10:49
  conditions. Other teams also find this.
• 10:52
  So it's a very well replicated finding.
• 10:57
  Now, there's limitations, we can get into that,
• 11:00
  but that's what, that's a bit of context, that's a bit of definition
• 11:04
  and that's some evidence of what we're finding and happy to answer more questions
• 11:08
  about. So let's try to rephrase that. So there is one
• 11:12
  thing you call differences on proportions or that
• 11:16
  are mostly descriptive, which are mostly the risk of one
• 11:20
  student compared to another student to be in special education.
• 11:23
  Something as you say, it's been well documented. But when you,
• 11:27
  in your world, when you take to talk about disparities,
• 11:32
  talk mostly about looking
• 11:35
  at issues of not accessing services or biases
• 11:40
  on the identification process. Yeah, yeah. So that's
• 11:45
  right. So you know, so descriptively you can find evidence,
• 11:48
  we find that others find it. Descriptively you find evidence
• 11:52
  of differences by race and ethnicity and who's being identified as
• 11:57
  having disabilities and receiving services.
• 12:00
  Okay. And you know, no dispute about that,
• 12:05
  we find that others find that descriptively. That seems to be the case.
• 12:09
  Okay. At the national level.
• 12:12
  The question then is why is that occurring?
• 12:16
  Okay. And this, this to me has really
• 12:19
  gotten the field off track. Okay. Because there's
• 12:24
  been a lot of strong claims based
• 12:28
  on really weak empirical evidence to
• 12:31
  support such claims about discriminatory
• 12:36
  practices and bias that systemic that
• 12:40
  by, you know, teachers, school personnel,
• 12:44
  that's resulting of students of color being inappropriately placed
• 12:48
  identified as having disabilities in place in special education.
• 12:52
  That evidence to me
• 12:56
  is that inference is not supported by the
• 13:00
  strength of the evidence available. Okay, I'm all for the possibility
• 13:04
  that might be occurring. I mean, that's one of the reasons we started doing these
• 13:08
  studies, is to investigate that as a possibility. And I'm completely
• 13:12
  open to that being observed.
• 13:16
  We're not finding it, others are not finding it.
• 13:21
  We can go into some sort of detail about that.
• 13:24
  The evidence base that supports a conjecture
• 13:28
  that the overrepresentation of students of
• 13:32
  color is the result of systemic bias and discriminatory practices
• 13:36
  by schools, to me, is not empirically supported or credible
• 13:41
  as a claim. Now, I'll tweak that a little bit, okay. Because there is some
• 13:45
  on the margin, perhaps some evidence that we should attend to,
• 13:49
  but by and large, those kinds of sweeping claims about
• 13:54
  how special education is working, you know,
• 13:57
  I don't see much if any, evidence. And I've been
• 14:01
  searching for, you know, our first presentation on
• 14:05
  this was in 2008. I've been. Been looking for this kind of evidence,
• 14:08
  examining the research base since then. The evidence base
• 14:13
  to support such speaking claims to me is. Is largely non
• 14:17
  existent. Okay. Not to say it might not emerge, not to say it might not
• 14:21
  be true in some particular context, but there's
• 14:25
  some things that we have to, you know, focus on.
• 14:29
  First of all, descriptively, that kind of claim based on descriptive evidence is
• 14:33
  just not supported. So as an analogy for some of your
• 14:37
  listeners, descriptively,
• 14:40
  black students are more likely to have asthma and
• 14:44
  it's a risk ratio of 1.6, something like that.
• 14:50
  That seems to be the case. That seems to suggest a
• 14:53
  difference. Okay, now the next step is, why is that occurring? Should we
• 14:58
  then automatically include the providers or bias in how they're diagnosing asthma?
• 15:02
  Okay. From my perspective,
• 15:06
  perhaps, but that descriptive evidence doesn't allow us to
• 15:10
  make that inference. We need better, stronger evidence. We need some sort
• 15:14
  of evidence that when presented with children who seem similar
• 15:20
  in having the symptoms, say, of asthma, the providers
• 15:25
  are more likely to identify kids of color who don't
• 15:29
  have asthma based on their symptomatology as having asthma.
• 15:33
  That would be the analogy here. So it's
• 15:37
  not the descriptive evidence that I think where the debate is, it's the inferences
• 15:42
  that are being made on. Okay, now the last thing I'll say here,
• 15:46
  because I don't want to be answering questions.
• 15:51
  So how do we know whether there's bias or discriminant?
• 15:55
  There's two standards. One is differential treatment and the other is disparate
• 16:00
  impact. Disparate impact I don't think is so relevant
• 16:04
  here. So I'm going to focus more on Differential treatment, which is the traditional standard
• 16:09
  of identifying discriminatory practices.
• 16:12
  So you take individuals who are similarly situated.
• 16:15
  This is exactly the language that the US Department of Education in
• 16:20
  2016, with the equity and IDEA regulations introduced. This is
• 16:24
  how we know whether discriminatory practices are occurring amongst similarly
• 16:28
  situated, otherwise similar children, particularly in the factors most
• 16:33
  materially relevant to being identified. You have evidence that
• 16:39
  children who are black are being inappropriately identified as
• 16:43
  having disabilities. So if you
• 16:47
  want to know whether bias or discriminatory practices are occurring in schools, which is an
• 16:51
  important question, then what we need is we need studies that
• 16:55
  are looking at children who seem otherwise similar and then
• 16:59
  looking at who's being more or less likely to be identified. So the
• 17:03
  way this is handled in other fields, vignette studies,
• 17:07
  audit studies where similar identical
• 17:12
  resumes are sent to employers and there's some examination
• 17:17
  of who's more likely to receive callbacks. And you know, in these studies and
• 17:21
  in other other types of investigations, you can find evidence of
• 17:25
  racial bias that certainly may be occurring in
• 17:29
  the case of special ed, as in the case of public health,
• 17:33
  what you find is when you look at children who seem otherwise similar,
• 17:38
  you get clear, repeated evidence that white and English speaking
• 17:42
  children are more likely to be identified as having disabilities than otherwise similar
• 17:46
  children of color or who are from non
• 17:50
  English speaking households. So, okay,
• 17:55
  there's so many questions to sneak through. Yeah, sorry,
• 17:59
  this is good. So I mean the comparison with the
• 18:03
  asthma, I understand it, my knowing goes like, well, we have probably much
• 18:07
  better ways to diagnose asthma that specific learning disabilities,
• 18:12
  for example, or behavioral. So that is
• 18:15
  what it gets to me a little tricky. Right, because there is a lot of
• 18:18
  more professional judgment
• 18:22
  included in identifying a student with emotional
• 18:26
  behavioral disability or specific learning. Right,
• 18:30
  right. So that anybody gets the bias or the systemic
• 18:34
  discrimination can sneak in. Right, Yeah, I understand
• 18:38
  that. So that's one thing. The other one is there's
• 18:43
  been a lot of studies to my understanding up to now
• 18:47
  that they're not just descriptive. Right. They're also trying to control
• 18:51
  and look at issues of district
• 18:55
  variables, school variables, and see how
• 19:00
  that impact disproportionality. It seems that those kinds of
• 19:04
  studies still does not convince you when they look for example
• 19:08
  at. For example, I remember the studies English
• 19:13
  language learners and the amount of support that received in second language or
• 19:17
  issues of district funding or each
• 19:21
  district racial disparities.
• 19:26
  Those levels of analysis at controlling or looking
• 19:30
  at district variables and school parameters still doesn't convince you that
• 19:34
  that bias exists for what I understand. Well,
• 19:37
  I don't think they should convince many.
• 19:40
  Okay. And again, I'm not.
• 19:43
  I'm trying to approach this, these all as empirical
• 19:47
  questions and to, to remove any kind of,
• 19:50
  you know, inclinations that I might have myself.
• 19:54
  If anything, I was inclined when I began these investigations
• 19:58
  that, okay, there's lots of reports that there's bias
• 20:02
  in how children are identified. Let's investigate that. So I was completely expecting
• 20:07
  that we would find evidence of over identification,
• 20:11
  you know, amongst some of our children.
• 20:15
  I was expecting to find that evidence indicating black Hispanic children
• 20:20
  are being over identified as having disabilities. That's not what we found.
• 20:25
  To answer your question, I think as
• 20:29
  a field we need to adopt methodological
• 20:34
  perspectives that allow us to conduct as
• 20:38
  strong investigations as possible. This issue is really important to figure out.
• 20:43
  And. To remember
• 20:48
  kind of continuums of evidence. Okay.
• 20:51
  Just like, you know, in our tradition where the usually
• 20:56
  expect randomized control trials are sort of
• 20:59
  like, you know, typically characterized as the gold standard, as long as they're well implemented
• 21:03
  and things like that are going to provide us pretty strong evidence of
• 21:07
  the impact of a treatment. Okay. They're just like that.
• 21:11
  You're gonna get well designed RCTs, poorly designed RCTs
• 21:15
  all across a continuum. You can have that also in observational studies.
• 21:19
  Okay. So what we need in terms of well conducted observational
• 21:24
  studies is we need to control for the factors that result
• 21:29
  in disability identification. Okay. So you
• 21:34
  know, just like in gifted ed, you know who, who gets gifted ed?
• 21:37
  Typically not children who are struggling academically.
• 21:41
  All right, who gets gifted ed are children who are displaying a
• 21:44
  high level of academic proficiency. Okay. If you
• 21:48
  showed me evidence among similarly high achieving
• 21:52
  students that children who are white are more likely to receive gifted
• 21:57
  ed services, I'm going to be like, okay, that seems to warrant some,
• 22:01
  some concern that maybe there's bias in how the system is operating.
• 22:04
  Okay. So when you talk about district level studies,
• 22:08
  they're not controlling for the factors that are most materially
• 22:12
  relevant to whether students are identified as having disabilities.
• 22:16
  What are those factors? Chief of those factors is how they're doing
• 22:20
  in school. What's the biggest indicator of how well children are doing in school?
• 22:23
  Whether they're learning their academic
• 22:27
  achievements. Those are the kind of studies that look
• 22:32
  at children who are displaying similar levels of academic
• 22:37
  achievement that I think, just like in the gifted ed example, I think
• 22:40
  are more credible.
• 22:43
  Okay. Now I'm not going to go into a lot of weeds.
• 22:46
  There are ways that we could do this even better. So,
• 22:50
  you know, other researchers have looked at using birth data so
• 22:54
  they Collect data from kids birth certificates. They link that with
• 22:59
  educational data. They know whether, for example, the child was born
• 23:02
  with low birth weight, their level of cognitive functioning at
• 23:07
  birth, whether there were any complications, things like
• 23:10
  this. When they use those kinds of data much more prospectively
• 23:14
  than many of the studies available to us or data sets available to us
• 23:19
  in special education, they find results similar to
• 23:23
  ours, evidence of under identification of students of
• 23:27
  color, particularly black and Hispanic students.
• 23:30
  So, you know, there are ways to add additional rigor to
• 23:34
  this question, not just looking at concurrently
• 23:38
  levels of academic achievement. I understand that,
• 23:43
  you know, that allows for even stronger evidence
• 23:46
  and from my perspective, but the district
• 23:50
  level studies that have no control for student level academic
• 23:55
  functioning or even district level differences
• 23:59
  in academic functioning across student populations to
• 24:03
  me are not particularly compelling in terms of answering the question are
• 24:08
  schools biased in how they're identifying children as having disabilities?
• 24:13
  But it could be more. I understand what
• 24:17
  you're saying, but I wonder if there could be more of a systemic
• 24:23
  disparities that are not based just on teacher biases.
• 24:27
  Right. It could be more larger structural issues
• 24:31
  from like funding kind of quality of education,
• 24:36
  the training of teachers,
• 24:40
  the history of the district. I mean, there are other factors where
• 24:45
  you can see disparities that not just be the individual teacher who say,
• 24:49
  oh, you're black and we're going to send you this particular
• 24:52
  educational. Yeah, 100%. I think we're in complete agreement
• 24:57
  about that. Public health has adopted
• 25:01
  a social determinants of health perspective for some time.
• 25:05
  That means there's a recognition that sort of like societal
• 25:10
  inequities, including economically, may be resulting in
• 25:14
  some of the differences by health conditions we're observing. So we should be trying to
• 25:18
  address those. Okay, I'm completely with you on that.
• 25:23
  There are many reasons to believe the way our society is organized
• 25:30
  is resulting in greater exposure to harm or
• 25:35
  risk factors, maybe by race ethnicity.
• 25:40
  And you find that across many, many different studies.
• 25:43
  And that starts very early. So, you know, children of color
• 25:47
  are more likely to be born with low birth weight, for example. They're more likely
• 25:51
  to experience extreme economic disadvantage. They're more likely to
• 25:55
  live not only in impoverished households,
• 25:59
  but in addition in impoverished neighborhoods.
• 26:03
  Okay. There are many different ways that our. There's differences
• 26:07
  by, in terms of lead exposure by racisticity.
• 26:11
  So there are many societal inequities that
• 26:15
  we should be trying to address. You know,
• 26:19
  I'm completely on board with that and in my mind
• 26:24
  what you're saying is exactly right. There are multiple influences here that
• 26:28
  could occur across the lifespan, including during
• 26:33
  early, middle and later childhood, Some of the factors that
• 26:37
  might be resulting in what we're observing in terms of under identification
• 26:42
  might be occurring in schools. So teachers
• 26:45
  may not be as likely to consider that
• 26:50
  the academic struggles of a child of color might indicate an underlying disability
• 26:55
  as they would a child who's white, for example.
• 26:59
  And there is some research to suggest that does occur.
• 27:04
  There's federal policy that's probably disincentivized school personnel
• 27:08
  from identifying children of color as having disabilities. But there
• 27:12
  are many upstream factors that
• 27:16
  might be resulting in these disparities
• 27:20
  that we're observing. So early exposure to poverty
• 27:24
  might be 1, or early exposure to lead or prematurity
• 27:29
  or low birth weight. Lack of early learning opportunities prior
• 27:34
  to school entry. Dismissive health providers
• 27:39
  might be another. There are many different factors that might be acting
• 27:44
  as essentially headwinds that are working
• 27:48
  against families of color in terms of being able to receive
• 27:54
  services for their children. And by the way, last thing I'll say.
• 27:58
  Go ahead. Many of the factors you're mentioning are outside of
• 28:02
  school, a lot of things. But I'm also thinking
• 28:06
  things inside school. For example, one of the things you do is
• 28:10
  much, as you say, similar situated students academically. Right. But it could
• 28:14
  be that a student that, you know, you have a certain test
• 28:18
  scores same match with another one, that one of the reasons that
• 28:22
  students have lower academic achievement is because certain factors
• 28:26
  between the school and the districts, meaning, you know, lower resources, lower opportunities to learn,
• 28:29
  and that student may have fall behind and therefore,
• 28:33
  you know, and that to me, it's a systemic disparity. Right,
• 28:36
  okay. Yeah. And you know, I agree
• 28:40
  with you that that could be happening. It's likely to be happening.
• 28:46
  A couple things I would say is
• 28:50
  that there's quite a bit of evidence going back that achievement
• 28:57
  gaps are evident by school entry and that they are largely stable across
• 29:01
  time. In fact, we have two studies, one that just came
• 29:05
  out last week, another that came out maybe three weeks ago,
• 29:09
  both of them looking at academic difficulties, one in reading and
• 29:13
  one in math and science. Across both sets of analyses, you find
• 29:17
  large differences by kindergarten in terms
• 29:21
  of the risk for academic difficulties.
• 29:24
  Math, science and reading. Those are large at kindergarten.
• 29:29
  They're stable across time. In fact, there's some evidence saying the risk increases
• 29:33
  for some racial ethnic groups. Along the lines you're saying that
• 29:37
  maybe there's things that are happening in schools that are increasing the likelihood of
• 29:42
  academic struggles for students of color. But another
• 29:46
  point to be made is the gaps are quite large at school entry,
• 29:51
  so they pre exist. Schooling And I think it would
• 29:55
  be helpful if more researchers adopted sort of a life course perspective
• 29:59
  to understand this problem and, and view these differences.
• 30:03
  The overrepresentation that we're talking
• 30:07
  about is likely a downside event, a downstream
• 30:11
  event, excuse me, a downstream event. So there are many events
• 30:15
  that are occurring prior to school entry that
• 30:19
  are upstream events that might be leading to
• 30:23
  these racial, ethnic differences in disability identification that are being observed.
• 30:28
  And secondarily, I would encourage your listeners to start
• 30:32
  to look at some of the public health literature on here around
• 30:36
  disability, racial and ethnic disparities and disability. You will find very
• 30:40
  strong repeated evidence that children of
• 30:44
  color and their families are less likely to be identified as having disabilities,
• 30:49
  including across a range of different specific conditions. And you also have a lot of
• 30:53
  which I think is important for, for our field to start adopting a lot of
• 30:56
  well designed qualitative studies that are trying to understand the mechanisms that
• 31:01
  might be underlying these disparities.
• 31:05
  I don't think we've done quite that kind of work here in
• 31:09
  special education anyway. We can go into that qualitative studies.
• 31:14
  I mean, what about the, I mean there's been repeated qualitative
• 31:18
  studies showing descriptively at least how for example in IEP
• 31:22
  meetings or teachers decisions there is certain bias
• 31:27
  towards particularly for example black low income families
• 31:31
  or Latinxas family. I'm thinking about the extensive work that
• 31:35
  Jeanette Klingler and Beth Harry did in large
• 31:39
  qualitative studies and there's been other works on examining
• 31:44
  penis when they have seen some indices of that biases,
• 31:48
  some maybe. Dr. Yeah, what I understood
• 31:52
  and you know, that work also is helpful for
• 31:56
  us to gain more knowledge about what's happening. What I haven't seen so much are
• 32:01
  direct interviews of families of color as they navigate
• 32:05
  a complicated system of disability identification
• 32:09
  and service receipt. Now I have seen quite a few studies like that
• 32:12
  in public health, including particularly around autism.
• 32:16
  And what you find in those studies repeatedly is that families of color,
• 32:21
  they report great difficulties in terms of accessing
• 32:25
  services for their child, they experience stigma
• 32:30
  in regards to their child's disability condition and
• 32:34
  they often feel unsupported in trying to navigate these systems.
• 32:38
  You also see quite a bit of other reporting in our field that the way
• 32:42
  that schools provide special education services are
• 32:47
  very difficult to access even for well resourced families.
• 32:51
  Dual income, high wage earners or high income
• 32:55
  earners will report great difficulty in trying to secure
• 32:59
  services for their child, even when that child has a documented
• 33:03
  disability from an outside health professional or some type of provider.
• 33:07
  So this system is quite difficult to access
• 33:11
  it requires on a high level of resources.
• 33:16
  Our due process materials are written.
• 33:19
  The disadvantage, less resourced families. It typically requires
• 33:23
  at least some college level work to understand even the due
• 33:27
  process materials that schools are providing. Families will
• 33:31
  report that schools will just not be responsive
• 33:35
  to their child's documented disabilities.
• 33:38
  So anyway, there's a lot of evidence that the way that
• 33:42
  the system works is
• 33:45
  the services are very scarce, they're very hard
• 33:49
  to access, and families are often reporting having
• 33:53
  to go to court to secure services for
• 33:58
  their child. So there's lots of indicators
• 34:02
  across special ed in public health that I think suggest to us that
• 34:07
  the way the system is operating could be improved and
• 34:11
  that there are a lot of inequities operating in the system that in my mind
• 34:15
  disadvantage students of color, non linguist speaking families,
• 34:19
  you know, children, families from historically marginalized communities I
• 34:23
  think are having. There's lots of reasons to believe that the
• 34:28
  way the system is operating is a disadvantage in those families. And we should be
• 34:32
  trying to fix that. We should be trying to ensure that all children and
• 34:36
  their families who have disabilities have access to these services. That's what's supposed
• 34:40
  to be happening and I
• 34:44
  don't think that's presently happening. We should be trying to understand that and trying to
• 34:48
  remedy that. So you were making a lot of comparison with the healthcare system
• 34:52
  and schools. Right. I'm imagining that some of your research,
• 34:56
  you've been inspired by the work on healthcare and trying to translate
• 35:00
  that to special education. Sometimes I feel very hesitant, particularly in the
• 35:04
  U.S. comparing the two because there are such a different system. Right. In other countries,
• 35:07
  like with more socialized health
• 35:11
  care system, like Spain, for example, to get special education
• 35:15
  services in schools, you've got to go through the public health system. You don't
• 35:19
  get in the defined schools, but in the U.S. right, right. That's such a
• 35:23
  different structures and I think
• 35:28
  they are effective for many different issues. Even their identification tools.
• 35:32
  Right. In the US to be identified with disability through the healthcare
• 35:36
  system, it's probably with much more discernible
• 35:42
  sometimes disabilities, we may have better accurate
• 35:46
  tools to identify them. In the case of schools in disproportionality,
• 35:50
  some of those disabilities are, you know, are, as I
• 35:54
  said before, they're very tied to professional judgment.
• 35:58
  Right. And some tools not even in the case of learning disabilities. Right. We've been
• 36:02
  discussing for decades accurate ways to identify students with specific learning disabilities.
• 36:07
  Same thing with behavioral disorders. So I'm kind of hesitant
• 36:11
  to think that the healthcare system may operate,
• 36:15
  that schools in 90, 50 may operate similar to the healthcare
• 36:19
  system make those assumptions. Right. Okay, okay, that's helpful.
• 36:23
  I understand what you're saying. I find it
• 36:27
  helpful for a couple reasons. One, public health
• 36:31
  tends to have a stronger tradition of large scale data analysis
• 36:36
  than special education. Two, they've been studying this problem for
• 36:40
  longer than we have with the kind of data that I think is
• 36:44
  useful for understanding what
• 36:48
  might be, might explain the observed differences.
• 36:54
  And then the third thing I would say is,
• 36:57
  I hear what you're saying, but there are some overlaps.
• 37:01
  So for example, pediatricians often are
• 37:05
  diagnosing adhd, a relatively subjective
• 37:09
  condition, or autism or other
• 37:13
  types of health conditions, including those related
• 37:17
  to learning or behavior. The other
• 37:22
  thing I would add is, from my perspective,
• 37:26
  I'm not saying it's easy, but I would say it's probably
• 37:29
  relatively easier to be identified as having a disability
• 37:33
  through a healthcare provider than through a school because
• 37:38
  the school, in order for the child to be found eligible for special
• 37:41
  education services, is going through, going to bring
• 37:47
  together a multidisciplinary team. They're going to do a lot of observations
• 37:51
  of the child. There's going to be a lot of consideration
• 37:56
  of alternative explanatory factors. It's going to be a longer process
• 38:01
  with many more people involved and there's
• 38:05
  going to be a lot more attention to sort of legal
• 38:09
  definitions that schools are supposed to be following.
• 38:12
  So I, I would say that it's, you know, in the
• 38:16
  case of adhd, it's, my conjecture is it's going to be relatively easier
• 38:21
  for a, for a family to receive a diagnosis from,
• 38:24
  of ADHD from a pediatrician than
• 38:29
  from a school. And in
• 38:33
  public health under those conditions, we're still finding, even though
• 38:37
  the system is arguably easier to navigate
• 38:42
  for families, you still find repeated evidence of racial
• 38:46
  and ethnic disparities, including say an ADHD diagnosis or
• 38:51
  autism or other sorts of conditions. So even in conditions where families,
• 38:55
  it might be relatively easier for families to navigate,
• 38:58
  it's still incredibly difficult. And the system seems to
• 39:02
  be more responsive to white and English speaking families in public
• 39:06
  health. So these are some reasons I think it's helpful for us to attend
• 39:10
  to what's being observed in public health. And I guess the last thing is it's
• 39:14
  still disability, right? And we're supposed to be concerned about whether children
• 39:18
  with disabilities are receiving appropriate care or supports.
• 39:22
  So I think it's helpful to look at other fields besides just special education.
• 39:28
  That's interesting. You know, I wanted to go back also to,
• 39:32
  you know, your, your methods to, you know, to, to just
• 39:36
  for those similar situated Students. Right.
• 39:40
  That it's your way to discern whether there
• 39:44
  is bias or not bias. Right. You have two similar students. If one gets identified
• 39:48
  and the other. No. Well, there may be some inspired. That's. That's rational. One of
• 39:52
  the ways your work and your colleague, not just you do that,
• 39:57
  it's by controlling variables like achievement and behavior.
• 40:01
  Right. So, for example, if two students have
• 40:05
  a similar score in
• 40:10
  a behavioral rating scale, you will pair those two students.
• 40:14
  Now, my question with that is, by controlling
• 40:18
  that, are you not taking
• 40:22
  for granted on the bias that may be inherent on those
• 40:27
  judgments, those professionals made with those rating skills,
• 40:31
  which would be a form of bias. Right. So you would be saying,
• 40:36
  well, that's true, but you're avoiding to think, okay, maybe why
• 40:39
  they're situated is because there were some bias on this.
• 40:42
  The. The process of situating the students similarly. Right,
• 40:46
  right. Yeah, I understand. It's a good point. Let me maybe
• 40:51
  respond a couple. Couple ways. One, it does kind
• 40:55
  of depend on the health condition. Okay. So, for example, in the case of
• 40:59
  adhd, it is a behavioral condition.
• 41:04
  You should be experiencing difficulties in
• 41:08
  attention or impulse control. So in
• 41:12
  some conditions, not accounting for differences in symptomatology
• 41:17
  for the condition is going
• 41:21
  to introduce bias in and of itself. Okay. So for some conditions, it seems to
• 41:25
  me especially important to include behavioral controls, say,
• 41:29
  in the case of adhd. That's one
• 41:33
  point. The second point is sometimes we're using parental reports of behavioral
• 41:37
  functioning, or others have. So I think the argument it's all teacher
• 41:42
  bias is a little harder to make in those studies. The third
• 41:45
  point is, when you actually delve into the evidence base
• 41:49
  for racial bias in behavioral ratings by teachers,
• 41:53
  it is a little bit more mixed. In fact,
• 41:56
  one study that I'm aware of compared
• 42:01
  teacher behavior ratings to direct observation of children's behavior
• 42:05
  and found did not find evidence
• 42:08
  indicating strong teacher racial and ethnic bias. And then the
• 42:13
  fourth thing I'll say is, oftentimes in our work, we've not
• 42:17
  included behavioral ratings. So in a 2017 study, an educational researcher,
• 42:22
  we did not include behavioral ratings if we controlled only
• 42:26
  for academic achievement. We found evidence of undratification
• 42:31
  across many different disability conditions. So why is
• 42:35
  that important? Because I think sometimes there's a concern about teacher
• 42:39
  bias in behavior ratings. I understand that evidence of
• 42:43
  identification is robust to analyses that
• 42:47
  do not include teacher behavioral ratings.
• 42:53
  So the other thing is, sometimes they'll be
• 42:57
  concerned that you've overcorrected. You've included too many
• 43:01
  controls. How does it even represent reality? That's another
• 43:06
  point I'LL make is simply adjusting for academic achievement
• 43:09
  on well designed achievement measures
• 43:13
  with strong psychometric properties that we've been using in the United States for,
• 43:18
  you know, since the 60s to assess for academic achievement.
• 43:22
  Even control for simple, simple control for
• 43:26
  academic achievement differences is sufficient enough to provide repeated evidence
• 43:30
  of undratification across many different disability conditions.
• 43:34
  Can we clarify because I know we've been talking a little bit generalizing about many
• 43:38
  different overrepresentation, underrepresentation, different disability categories. Can you be
• 43:43
  more specific? Remind us of in the case of your work and your colleagues
• 43:48
  in what disabilities and what racial groups have you find under
• 43:53
  instead of overrepresentation? Okay, good.
• 44:00
  In terms of we
• 44:05
  have yet to find over identification in comparisons
• 44:09
  otherwise similar children. Okay. Which goes back to the start of the conversation.
• 44:13
  There's just not much evidence if at all of over
• 44:17
  identification of children for racial and ethnic bias in how schools identify children
• 44:22
  that have disabilities. In fact, the opposite seems to be occurring.
• 44:25
  Schools seem more responsive to white and English speaking students
• 44:31
  and their families. In terms of the racial ethnic
• 44:35
  groups, Our results we
• 44:40
  find that across many racial and ethnic groups. It depends on the studies
• 44:44
  or the data sets I should say that we're analyzing.
• 44:47
  We've looked at students who are black, students who are Hispanic,
• 44:50
  students who are American Indian, students who are Asian,
• 44:53
  students of other race ethnicity. It varies by data set that
• 44:58
  we're able to analyze. For some racial ethnic
• 45:02
  groups, the sample sizes can get particularly small.
• 45:05
  For American Indian, Native, Pacific Islander,
• 45:10
  the sample sizes can get pretty small, especially if you're trying to
• 45:14
  do subgroup comparisons within that category.
• 45:18
  But we find repeated evidence across multiple
• 45:23
  racial and ethnic groups relative to white students of
• 45:27
  color are repeatedly less likely to be identified as having disabilities. We find
• 45:31
  that for special whether a child has an iep, yes or no
• 45:35
  as a variable. So for special education generally.
• 45:39
  And we also find it across many different specific
• 45:43
  disability conditions. So our most recent study in educational
• 45:47
  researcher that came out a few months ago examined
• 45:51
  this across eight specific disability conditions.
• 45:55
  Autism, adhd, speech, language impairments, specific learning
• 45:59
  disabilities, other health impairments, these kinds of things.
• 46:03
  We find repeated evidence of undeterredification both for special and
• 46:07
  generally and across a range of different specific conditions.
• 46:12
  Were you clear? I mean, what I'm not clear is on the Native
• 46:17
  American category because in many, in some of those studies kind of group them
• 46:21
  with Asian studies with Asian students or which
• 46:27
  I imagine will. Will not give you a real
• 46:31
  representation of that group. No, a real idea. Yeah,
• 46:35
  it's true. It does get more difficult.
• 46:41
  Nevertheless, you know, sometimes we're analyzing Data
• 46:46
  sets of 200,000 children, or close to that,
• 46:51
  representing who are attending thousands of schools across the
• 46:55
  United States. It's true
• 46:59
  that our work has limitations,
• 47:03
  all work has limitations.
• 47:07
  But at the same time, we are analyzing very large samples
• 47:12
  of children, both cross sectionally and longitudinally.
• 47:16
  It's true that some of the specific sub groups,
• 47:21
  the specific racial and ethnic groups get a little harder
• 47:25
  to, to study. But nevertheless, across the
• 47:30
  studies that we've conducted, and at this point it's something like, I don't know,
• 47:33
  20 to something like that, 20, 20 studies plus,
• 47:37
  I. Think. We find repeated evidence of undrage
• 47:42
  identification and just to make sure the
• 47:46
  listeners understand. So generally what we're doing is we're taking data collected by the federal
• 47:49
  government. The samples tend to be large. Sometimes they're cross
• 47:53
  sectional, sometimes they're longitudinal. A lot of information is collected about
• 47:58
  those kids. They're administered student level academic achievement
• 48:02
  measures. Sometimes those scores are not shared with the teachers.
• 48:05
  So it's an independent assessment of the children's true. I would say,
• 48:09
  you know, based on the best available evidence, you know,
• 48:13
  how well they're doing academically in schools. We often know a lot
• 48:16
  about the children's health conditions, whether they were born with low birth weight,
• 48:20
  their economic conditions as reported by the families themselves,
• 48:24
  other indicators of their background characteristics, as well as
• 48:27
  kind of functioning variables like academic achievement, executive functioning,
• 48:31
  sometimes classroom behavior as well.
• 48:34
  When we analyze the students in terms
• 48:38
  of identification, when we include some other explanatory factors that
• 48:42
  are most relevant to whether the child has a disability, including academic
• 48:46
  achievement, you find repeated indication of under identification.
• 48:50
  We find that repeatedly across many data sets and
• 48:54
  many different kinds of conditions, and across many different sort of reporters
• 48:58
  of disability status, as I was saying earlier, as well as different time periods,
• 49:02
  that finding is consistent with public health as well.
• 49:05
  So I think what our. It's not like there
• 49:09
  isn't more work to do, you know, I think there is. And our studies have
• 49:13
  limitations, which we could talk about if helpful for your listeners.
• 49:17
  But I think because of the repeated replicated
• 49:22
  nature of the findings, including by us as well as researchers,
• 49:26
  I think the field needs to start grappling with these findings more.
• 49:32
  And I know there's been attention to the findings,
• 49:35
  but oftentimes I see pretty quick dismissal of the findings,
• 49:39
  despite the fact that they're replicated, despite they're
• 49:43
  based on high quality data, despite being based on longitudinal
• 49:47
  studies, despite being replicated by other research teams,
• 49:50
  despite being consistent with findings in other in other
• 49:54
  fields. So I think what we need to do is start
• 49:58
  giving some more attention in my mind to are we meeting the needs
• 50:02
  of all children and their families? And there's many reasons to believe that
• 50:06
  we're not. Okay. Our results suggest that we're not. Findings by others
• 50:10
  suggest we're not. So what are we going to do about that?
• 50:14
  Now we give a lot of recommendations based on
• 50:18
  including public health and by researchers in other fields about
• 50:23
  what should be done. Better due process materials
• 50:28
  with greater readability that are accessible by all families. Greater dissemination
• 50:32
  efforts, including to historically marginalized communities
• 50:36
  who often don't have access to evidence based information about
• 50:40
  disability, including signs, symptoms, available treatments,
• 50:44
  adoption of school, to community partnerships,
• 50:48
  making sure that there's providers from the community that are familiar
• 50:52
  with the disability condition in which there's evidence of a disparity are
• 50:58
  going out, including to sort of non traditional settings to
• 51:02
  at least disseminate the information to ensure families across all
• 51:05
  race, ethnicities, across all socioeconomic backgrounds have
• 51:10
  access to the same information. There are many things we can
• 51:14
  be doing to ensure greater equity in the system that ensures that children
• 51:18
  with disabilities across all race and ethnicities are being
• 51:21
  appropriately identified. And I think the field will benefit
• 51:25
  from greater attention to what our
• 51:29
  results repeatedly suggest is a large,
• 51:32
  long standing set of inequities that
• 51:37
  seem to be disadvantaging students of color
• 51:40
  who have disabilities, who may have disabilities to which they
• 51:47
  may be legally entitled to services for. And the last thing I'll say
• 51:51
  is there's also evidence that the services that are being provided are effective.
• 51:55
  Okay. Not that there's not more work to be done, but this
• 51:59
  wholesale dismissal of special education as
• 52:04
  just a way to segregate students and
• 52:08
  that what's being provided by special education is inherently
• 52:13
  harmful, I think is not credible.
• 52:16
  Okay. And there is re.
• 52:20
  There is work that I think is strong,
• 52:23
  quasi, experimentally that finds evidence of
• 52:27
  benefit of services, including for students of color.
• 52:31
  So if the services are beneficial and they aren't being
• 52:35
  provided equitably, then I think there's some.
• 52:39
  We need to be doing a better job to making sure that
• 52:43
  we're addressing these headwinds that children
• 52:47
  and their families may be experiencing in trying to access these services.
• 52:51
  Yeah, I think the question of services. Yeah, I think we have mixed
• 52:56
  evidence on that. That's my understanding. I think there are studies that points to that,
• 53:00
  some other studies that points that, you know,
• 53:04
  that the effectiveness of special education are
• 53:10
  not as effective as one may
• 53:14
  want to argue to place students there.
• 53:18
  Okay, let me just.
• 53:21
  I'm sure there's mixed Evidence. Okay. It's what's, it's the quality
• 53:26
  of the evidence that's also important to attend to. Okay. So if you're taking
• 53:30
  a descriptive study, that's comparison comparing, say, the academic
• 53:34
  achievement of kids with disabilities, the academic achievement of kids without disabilities,
• 53:39
  and finding evidence that kids with disabilities display
• 53:43
  lower achievement, that's descriptively, that's okay.
• 53:46
  That's maybe the case. That's not a strong study
• 53:50
  in terms of inferring effectiveness of services. The best
• 53:54
  quality studies that I know of that have tried to assess special
• 53:59
  education's effectiveness have looked at kids, the same kids,
• 54:03
  when they receive and don't receive special education.
• 54:07
  So many things are accounted for in those studies as
• 54:10
  potential confounds. Those studies repeatedly find that
• 54:15
  students seem to benefit, including academically,
• 54:19
  when they're receiving special education services. That finding has been
• 54:24
  replicated across different researchers analyzing different
• 54:28
  data repeatedly. And in fact, there's a recent
• 54:31
  review in remedial and special education by o'
• 54:35
  Hagan and I think reporting
• 54:41
  on the evidence of impacts, potential impacts. Now there's more
• 54:45
  work to be done. Right? There's always limitations to study this kind of question.
• 54:48
  But, but I. To me, I would say the best
• 54:52
  available evidence seems to suggest the services are largely beneficial,
• 54:57
  including to students of color. Yeah. And. Well, I mean, that's. That's a
• 55:01
  whole. I mean, I will have it for another show because the other question is.
• 55:04
  Okay, well, if it's that if,
• 55:07
  yes, special education may be beneficial some of students, but also is that because
• 55:12
  we have a general education classrooms that are completely obivious
• 55:17
  to students with disabilities? Well, that's the other side.
• 55:20
  That's a question. Yeah, that's a good question. Yep. That's the other side of the
• 55:23
  question. We have this system that may be working to certain extent,
• 55:27
  but is doing a job that I think it
• 55:32
  should be done by much more by
• 55:36
  the general education system. But that's a much larger discussion.
• 55:40
  Right. And I imagine that we have a lot of agreement around
• 55:44
  the idea that general education can be improved, including ways
• 55:48
  that benefit kids with disabilities. So I know we're kind of right off time,
• 55:52
  but I have two more for me, like pressing questions. So I'm going to say
• 55:55
  if we can discuss. The one is, you talked a lot about socioeconomic
• 55:59
  status. When you talk about disparities, you talk about, when you were talking
• 56:03
  mostly about the healthcare, that a lot of
• 56:06
  things about poverty impact. When I see your studies, the impact
• 56:10
  of SES is not as clear to me.
• 56:15
  I can think, for example, one of your first studies I was going through your
• 56:19
  method, kind of did my homework, went back to your studies, start seeing, pick up
• 56:23
  your methods. There was one, for example,
• 56:28
  in one of your first, I think was in the RER.
• 56:31
  Ariano Educational Researcher yeah. SES only was predict only
• 56:35
  like on 3 of like 20 something possible tests that they were doing.
• 56:39
  So I'm wondering if you find SES such an important
• 56:43
  factor or honestly you can say, you know, it's still,
• 56:47
  still to be the term. Okay, it's a good question.
• 56:51
  And it's true that we
• 56:55
  find, you know, SES can
• 57:00
  sometimes be a predictor and sometimes not. But I think the explanation
• 57:04
  for that is we're including the more direct,
• 57:08
  directly relevant indicator. So you can think of SES
• 57:13
  as sort of a little bit of a distal indicator or marker
• 57:17
  of whether children have a disability and their actual academic
• 57:20
  achievement as a more proximal indicator. So in our regressions, we're including
• 57:25
  the student's own level of academic achievement as
• 57:29
  well as the family's ses. And what I believe is happening is the
• 57:34
  proximal indicator of disability in terms
• 57:38
  of the direct nature of children's academic achievement is driving selection
• 57:44
  in the special ed. And we find this over
• 57:49
  and over again. So the strongest indicator of whether
• 57:54
  children are receiving special education is that their academic
• 57:58
  achievement, that's what schools are largely seem to be selecting on.
• 58:02
  If children are struggling, the schools are more likely to consider the
• 58:06
  child as possibly having a disability. And when the child is not struggling,
• 58:10
  they're less likely.
• 58:13
  And this is a good way to illustrate maybe a, a pattern. So if
• 58:17
  we look at the academic distribution, if you're, if you're in the, if you're
• 58:21
  in the lowest 10% of the academic distribution,
• 58:24
  you're more likely to receive special education than if
• 58:28
  you're in the top 10%. So it's something like for
• 58:33
  children who are white, for example, if you're in the top, if you're the lowest
• 58:36
  10% of the achievement distribution among children who are white,
• 58:40
  about 75% of those children are receiving special education.
• 58:43
  If you look at the top 10%,
• 58:47
  it's in the low single digits, 1 or 2%, something like that.
• 58:51
  So achievement seems to be the really drive whether children
• 58:55
  are being identified as having disabilities, if they're struggling. And the other pattern
• 58:59
  that we should. Clarify for our audience, sorry, that low achievement
• 59:04
  should not be the reason for special education services.
• 59:08
  Just to clarify that, because I know we're talking a lot of achievement associated
• 59:12
  to identification, as you're saying, but in
• 59:17
  terms of, if a student should be in a special education or not. Should not
• 59:20
  be determined by their academic achievement alone.
• 59:24
  Correct. Okay. Well. Okay, let me.
• 59:27
  I would. Maybe the
• 59:32
  most kids are identified as having learning
• 59:37
  disabilities, speech, language impairments or other health impairments.
• 59:41
  That's the majority of kids. In terms of who's being identified as having disabilities,
• 59:46
  as they. For those conditions, there should be some indication of
• 59:51
  difficulties in school,
• 59:54
  particularly in learning. Okay.
• 59:59
  I'm not saying that if a kid has low achievement,
• 1:00:03
  they have a disability, but if
• 1:00:07
  you're not struggling in school, it's not clear
• 1:00:11
  how well you're fitting, sort of definitionally, what disability is
• 1:00:15
  characterized or operationalized in schools, which is some indication
• 1:00:19
  of academic struggles, difficulty in school.
• 1:00:23
  So maybe we could talk about that more at
• 1:00:28
  some point. I'm not. Just because a
• 1:00:32
  kid is struggling academically doesn't mean they have a disability. Yeah, but if it's.
• 1:00:36
  It's a little less clear whether a child has a disability if they're not experiencing
• 1:00:40
  some kind of difficulty in school, including academically.
• 1:00:43
  And if you look. Yeah. It could be other things. Okay.
• 1:00:46
  But that becomes hard for schools to figure out.
• 1:00:50
  You know, that. Which is another conversation we could have.
• 1:00:54
  But the other thing I would say, just to close the loop, if, if,
• 1:00:58
  if, if. What we'll find in our analysis is like,
• 1:01:02
  of kids who are in the lowest 10% of the score,
• 1:01:05
  reading achievement distribution. Right. These kids are really struggling in
• 1:01:10
  reading. They're in the low. 90% of their peers
• 1:01:14
  read better than they do. So they're really struggling.
• 1:01:19
  If we look at the composition of that among
• 1:01:22
  kids who are white, about 75% of those kids are getting special
• 1:01:26
  education services. The percentages for
• 1:01:30
  students of color are always lower. It's something like 40,
• 1:01:34
  45%. It's a large difference.
• 1:01:38
  So there seems to be some indication just
• 1:01:41
  by. Across the achievement distribution on who's more or less likely to get additional
• 1:01:46
  help, including through special education. But anyway, I'm not sure
• 1:01:50
  if that answered the question, but. Yeah, no, that's. Yeah.
• 1:01:53
  Sonya had another question. Yeah. I have one more pressing question. 1. I gotta let
• 1:01:57
  you dream a little bit. Okay. The.
• 1:02:00
  The other one. You know, it's a lot of your studies, I think the majority
• 1:02:04
  of them and your colleagues are based on large national
• 1:02:08
  data. Correct.
• 1:02:11
  So thinking that a lot of overrepresent,
• 1:02:15
  a lot of disproportionality. Research points out that it's very contextual. Right.
• 1:02:18
  Is very geographically located. It depends on many
• 1:02:22
  different factors that are driven by local histories,
• 1:02:26
  by local politics, et cetera, et cetera.
• 1:02:29
  Do you think there a danger to say,
• 1:02:33
  okay, nationally students are underrepresented and
• 1:02:37
  try to do a blanket policy according to that, if the context
• 1:02:43
  and disproportionality in different states
• 1:02:46
  may show different evidence, and therefore states maybe
• 1:02:51
  doing policy driven by national agenda that does not represent
• 1:02:54
  the particular equity issues that they're facing. Okay,
• 1:02:58
  okay, so it's a good question. First of all, we've already
• 1:03:02
  pushed to make this a federal issue. So that's what the
• 1:03:06
  equity and IDEA regulations are. That's why IDEA
• 1:03:10
  was amended to give more focus, because there was
• 1:03:15
  concerns across the United States that the way that schools were identifying students
• 1:03:19
  of color was in some way biased because of this descriptive
• 1:03:23
  evidence which we were talking about at the start. So the
• 1:03:28
  data that was used to inform federal policy was not sufficient to make the
• 1:03:33
  claims, from my perspective, that bias was driving
• 1:03:37
  those racial and ethnic differences. We needed stronger evidence
• 1:03:41
  for that. And that's what we've been doing. So, and we've been analyzing
• 1:03:45
  nationally representative data in part because those data
• 1:03:49
  sets often have what and
• 1:03:53
  what we need, from my perspective, to better evaluate
• 1:03:57
  for potential bias. And that's indicators
• 1:04:01
  of clinical need, right? Direct measures of, say, academic achievement.
• 1:04:07
  Those data have often been nationally representative
• 1:04:11
  data. So the ECCLES K, the 1998,
• 1:04:14
  1999 cohort, the ECCLES K 2010, 2011 cohort,
• 1:04:18
  the National Assessment of Educational Progress, the NAEP.
• 1:04:23
  It may be that in particular context there is
• 1:04:27
  some evidence of over identification among
• 1:04:31
  similarity, similarly situated students. There may be evidence of over
• 1:04:35
  of suggestive of a potential bias.
• 1:04:38
  And if that's the case, we should be trying to do something about that.
• 1:04:41
  Okay. And if that's observed, I would like to know and
• 1:04:46
  read about it and learn about it so I can update my own knowledge
• 1:04:50
  about what might be occurring. I'm completely open to the possibility it may
• 1:04:54
  be occurring, but I think what your listeners should know is there's really no evidence
• 1:04:59
  of that. Okay. So for example, on this local context
• 1:05:04
  question, one of our studies was
• 1:05:08
  published in Exceptional. Trudeau looked at the local context of the US south,
• 1:05:12
  which has a very strong, you know, tradition of
• 1:05:15
  de jure, de facto racial segregation. We didn't find
• 1:05:20
  evidence of over identification even in a, in local, state specific,
• 1:05:24
  local context where you would expect racial bias to
• 1:05:28
  be especially likely to occur. I was at a presentation
• 1:05:32
  in Tree, which followed a sample
• 1:05:36
  of low income children across time in
• 1:05:40
  more of a local sample. Once they compared
• 1:05:45
  children who are otherwise similar, they also observed evidence suggestive
• 1:05:49
  of undocentification of students who were black. That's another
• 1:05:53
  sort of local context. So when we analyzed
• 1:05:56
  the. When we, when we
• 1:06:01
  linked the Stanford Educational Data Archive and the
• 1:06:04
  Office of Civil Rights data and looked at districts across the US Once
• 1:06:11
  we accounted for district level achievement gaps, we found very little evidence of,
• 1:06:15
  you know, over. Over representation, over identification.
• 1:06:18
  There were some specific schools that were
• 1:06:22
  anomalies, and I would suggest that they be further investigated.
• 1:06:26
  Okay, so it certainly is possible. It's an empirical
• 1:06:30
  question and it should be investigated. I think the weight of
• 1:06:34
  the evidence runs to the other side of the coin,
• 1:06:38
  which is headwinds that keep
• 1:06:42
  families, particularly those less resourced or families
• 1:06:47
  from historically marginalized is from having the same access to
• 1:06:51
  these services. And as we've been talking about over the past hour, I think we
• 1:06:55
  at least need to give that serious, sustained scholarly attention to make
• 1:06:59
  sure that the way that the system is operating is indeed equitable
• 1:07:03
  in terms of access to these services. So I was thinking,
• 1:07:07
  for example, studies like I'm thinking, for example,
• 1:07:11
  Styles on Arizona by Amanda. I think Sullivan, this artillery
• 1:07:17
  is dead back study in 2005, I think. I don't.
• 1:07:21
  BAU did some studies in Wisconsin also, if I'm not mistaken,
• 1:07:25
  my understanding of your position on this is that those studies are descriptive
• 1:07:31
  and therefore doesn't provide sufficient evidence to say
• 1:07:35
  that there is overrepresentation. Am I reading you correctly? Okay, I wouldn't quite
• 1:07:39
  put it like that. Let me revise that bit a little. Little bit. So when
• 1:07:42
  I see the reporting by OSEP to Congress,
• 1:07:46
  that is just raw descriptive
• 1:07:51
  statistics. There's no attempt to meet this sort of otherwise
• 1:07:55
  similar standard. Okay, so it shows
• 1:07:59
  overrepresentation, but we don't know why that's occurring.
• 1:08:03
  It could be bias, but it could be other things. Maybe it's differential exposure
• 1:08:07
  to risk factors for disability. So if we want to know
• 1:08:11
  whether bias is occurring, we need to move
• 1:08:15
  down the continuum of studies from simple descriptive
• 1:08:19
  to some sort of control for
• 1:08:24
  factors that are most directly relevant to whether students are being identified
• 1:08:27
  as having disabilities. Now, some of the other
• 1:08:31
  work that you mentioned, it will include other.
• 1:08:38
  It won't be just simply descriptive. It'll use some kind of regression
• 1:08:41
  equation and they'll use other explanatory factors.
• 1:08:45
  Often they're not including the factors most
• 1:08:49
  directly relevant to whether children have a disability, which is whether they're
• 1:08:54
  academic achievement. So in those studies,
• 1:08:58
  you'll see no control for academic achievements.
• 1:09:01
  Maybe you'll see some sort of course analysis
• 1:09:06
  whether pass, no pass on achievement measure.
• 1:09:10
  What you'll see is that the risk will
• 1:09:14
  decrease as Other explanatory factors are included.
• 1:09:18
  And in a case where you have direct measures of academic
• 1:09:23
  achievement, there, the evidence base is to
• 1:09:27
  me quite consistent that you observe evidence of
• 1:09:31
  under identification. The only thing that you sometimes
• 1:09:35
  see in the literature that I kind of alluded to earlier that maybe I
• 1:09:39
  shouldn't mention here is there is a
• 1:09:44
  very small research literature that's tried
• 1:09:48
  to account for other explanatory factors of
• 1:09:53
  the nature that we've been describing that are relevant to whether a child has a
• 1:09:56
  disability. In some of
• 1:10:00
  that work, including some work that we've conducted, you can find a
• 1:10:04
  school level kind of an interaction between
• 1:10:08
  the student in the school. So the racial segregation
• 1:10:12
  of the school may also matter. So when you're
• 1:10:16
  going to a school where. Which is heavily racially segregated
• 1:10:20
  and you're of the same race ethnicity of the.
• 1:10:24
  The. The. The type of segregation that's occurring.
• 1:10:29
  So if you're in a mostly black school, a school mostly
• 1:10:32
  attended by black students, and you're black, that further
• 1:10:37
  seems to dampen your likelihood of being identified.
• 1:10:41
  Whereas if you're. If there's some. Right.
• 1:10:44
  Yeah, you can find it. Now, we're talking about,
• 1:10:48
  you know, there's a lot of racial and ethnic as well
• 1:10:52
  as economic segregation in US Schools. So we're
• 1:10:56
  talking about, you know, sort of a percentage of. A percentage of percentage.
• 1:11:00
  It's getting. We're getting into a very, you know,
• 1:11:04
  on the. Very much on the margin for those students
• 1:11:09
  who seem to, you know, distinct from
• 1:11:14
  the racial composition of the school, you can find some
• 1:11:18
  evidence of reporting that those students are
• 1:11:22
  more likely to be identified. So on the margin, you can
• 1:11:26
  find some reporting that if you are a black student
• 1:11:30
  attending a mostly white student, you might be more likely to be
• 1:11:34
  identified. Or if you're a white student attending a mostly.
• 1:11:38
  A school attended by mostly black students, you might be more likely to be identified.
• 1:11:42
  So on that rare on the margin case, you can find some evidence of it.
• 1:11:46
  The reason I'm so I'm trying to characterize the nature of the finding,
• 1:11:50
  but there's a lot of racial and ethnic
• 1:11:54
  segregation that's occurring in school. So,
• 1:11:57
  for example, most students of color go to mo schools mostly
• 1:12:02
  attended by students of their same race ethnicity. So that's why
• 1:12:05
  I'm characterizing a little bit on the rare case on the margin that might be
• 1:12:09
  occurring. And that certainly warrants further investigation because again,
• 1:12:13
  as we started talking, we don't want to identify kids with disabilities who
• 1:12:17
  don't have disabilities, and we don't want to not identify kids with Disabilities who do.
• 1:12:21
  And that's the balance we should be trying to strike
• 1:12:24
  now. Our analyses repeatedly find that it's
• 1:12:28
  way tilted for students who are white or English speaking being
• 1:12:33
  more likely than otherwise similar students of color or those from non English speaking families
• 1:12:37
  to be identified. That seems to be the average
• 1:12:41
  effect, one might say. And so
• 1:12:45
  we should be, that's, you know, to me, we should be having a lot more
• 1:12:49
  attention to that, including in terms of federal policy, which is.
• 1:12:53
  DOC does not recognize this problem despite repeated evidence that it's occurring.
• 1:12:58
  So anyway.
• 1:13:02
  No, but national policy asks states. Right. To identify either
• 1:13:06
  over or under representation. My understanding. Right. It's not just over
• 1:13:10
  representation. They asked us to identify significant
• 1:13:14
  disproportionality, which could go either way. Right.
• 1:13:17
  Okay. Let me maybe just remark on what a master
• 1:13:22
  is. Okay. From my perspective. Okay.
• 1:13:25
  Okay. What, what, what, what. What are states
• 1:13:29
  asked to do? Okay. Under the regulations, equity IDEA
• 1:13:33
  regulations, they adopt a standard threshold. Okay.
• 1:13:39
  That still, you know, varies somewhat by states.
• 1:13:42
  But the regulations ask states to
• 1:13:46
  adopt a standard threshold that they then monitor. And they monitor it
• 1:13:51
  across identification, placement and discipline and across different
• 1:13:55
  racial and ethnic groups. So the state set
• 1:13:59
  the standard. And what you'll find is if you go in, if you look at,
• 1:14:02
  you know, state of Illinois, significant disproportionality.
• 1:14:07
  State of Oklahoma, significant disproportionality. You know, whatever state,
• 1:14:10
  they'll have regulations, and the regulations will set some type
• 1:14:14
  of risk ratio. And the number that you generally see is
• 1:14:18
  something like if students of color, you know, black or Hispanic
• 1:14:22
  students in the, in the district, if they're three times. If they're,
• 1:14:26
  if, if, if they're three, essentially, if they're three times as likely to be in
• 1:14:30
  special ed. Okay, what. And you'll see
• 1:14:34
  that monitoring over identification, placement and discipline,
• 1:14:36
  using some numerical target.
• 1:14:40
  That sets, which is set by each state. It's not safe by
• 1:14:44
  state, but they have to get it approved by
• 1:14:48
  the department. And it's somewhere around risk
• 1:14:53
  ratio of 2.53. And then usually some indication
• 1:14:57
  of over some consecutive time, say three consecutive
• 1:15:02
  years or something like that. There's some interesting anecdotes because I
• 1:15:05
  work earlier on looking at states and back on my PhD years and
• 1:15:09
  at the beginning when the policy got implemented, you will see thresholds
• 1:15:14
  like 8 times more likely, 9 times more likely. Yeah, it was completely
• 1:15:18
  outrageous. But anyway, the reason I
• 1:15:22
  think it's a mess is because we're not able to understand
• 1:15:26
  the nature of the problem that is motivating the
• 1:15:30
  focus. Okay. So.
• 1:15:35
  If I'm Making this up. But if
• 1:15:39
  the state of Illinois flagged a set of districts that have significant disproportionality,
• 1:15:44
  all that means. I mean, I'm not downplaying it, but the
• 1:15:49
  problem is we don't know anything about the appropriateness of de
• 1:15:53
  identification. Okay? All we have is the same thing we
• 1:15:57
  went into this conversation with is some descriptive evidence of a difference.
• 1:16:01
  That's all we have. The federal government,
• 1:16:04
  the regulations, the states are not reporting on kids.
• 1:16:08
  You know that in terms of their,
• 1:16:12
  say, academic achievement. The analogy would
• 1:16:16
  be if we were concerned about gender disparities
• 1:16:21
  in suspension. You know, boys are five times as likely to be suspended
• 1:16:26
  by schools as girls. Okay?
• 1:16:31
  We should be concerned. Let's monitor that problem.
• 1:16:35
  Anyone who studies these issues from a social scientific
• 1:16:39
  standpoint is going to go, hey, that difference is not sufficient evidence
• 1:16:44
  of bias. We don't know anything about the appropriateness
• 1:16:47
  of the response by schools. So one
• 1:16:52
  explanation is schools are biased against boys. Another explanation
• 1:16:57
  is boys are acting out and engaging in the behaviors that make it
• 1:17:01
  more likely they're suspended. Now, I'm not commenting on the appropriateness of
• 1:17:05
  suspension. I'm not supporting suspension as a
• 1:17:09
  tool for managing behaviors. I'm not. I'm saying it's a policy issue,
• 1:17:13
  okay? And when we have this raw, descriptive evidence, we don't know
• 1:17:17
  anything about the appropriateness of the response. Now, to go back to the significant
• 1:17:20
  disproportionality regulations, states report this.
• 1:17:24
  They don't report anything that provides any indication of the appropriateness
• 1:17:28
  of. Of the disability determination.
• 1:17:33
  So it could be that the schools are biased. It could
• 1:17:37
  also be that there's baseline differences in
• 1:17:41
  rates of disability. For example, we don't know from the evidence being
• 1:17:45
  provided. Okay? So it's
• 1:17:50
  important to know whether there's bias happening in how schools are responding to
• 1:17:54
  behavior or responding in terms of disability identifications.
• 1:17:57
  What we need is some evidence that allows us
• 1:18:01
  to know whether bias is occurring. And really the way to do that is this
• 1:18:05
  differential treatment standard. The significant disproportionality
• 1:18:10
  regulations do not provide evidence that allows us
• 1:18:13
  to know whether bias is occurring.
• 1:18:17
  So every time I see a report, state of Illinois, state of Oklahoma,
• 1:18:21
  whoever these districts got flagged, I know nothing
• 1:18:25
  about the appropriateness of the disability notification that's occurring. And nobody
• 1:18:29
  else does either. But yet the districts are told,
• 1:18:33
  stop doing that. Okay, well, the districts
• 1:18:37
  are told to. Address the issue through different causes.
• 1:18:40
  It could be like improving MTSs, like pouring my resources on
• 1:18:46
  more early opportunities to learn. Could be. Some people
• 1:18:50
  are for more cultural responsive instruction with communities of color, more engaging
• 1:18:54
  with family, et cetera, et cetera, to improve the
• 1:18:58
  experiences of students before they get to that point of
• 1:19:02
  identification. Right. That could happen. And I'm all for that. But what I
• 1:19:07
  also think could happen is schools are saying we can't adopt.
• 1:19:10
  We can't identify any more students of color. Right. And what's
• 1:19:14
  happened is now we've artificially limited access to
• 1:19:19
  services, including by students who may have disabilities who
• 1:19:23
  are of color. And that's not a situation we want.
• 1:19:26
  So I'm all for making sure that teachers are culturally
• 1:19:31
  responsive. I'm all for. For optimizing the educational
• 1:19:35
  experiences of students, including across many,
• 1:19:39
  you know, sort of different backgrounds, diverse backgrounds, all for that.
• 1:19:43
  You know, what I'm worried about is
• 1:19:47
  that the significant disproportionality thresholds become artificial
• 1:19:53
  limits to access to services that are disadvantaging
• 1:19:57
  particular student populations, including those of color.
• 1:20:00
  And to me, we don't want that.
• 1:20:05
  If families don't want these services, that's their choice.
• 1:20:09
  That's largely their choice. Have we met the condition
• 1:20:12
  that we know that all families have been appropriately informed?
• 1:20:17
  Have we met the condition that schools are doing everything that
• 1:20:21
  we can to make sure that all students have access to these services
• 1:20:25
  who have disabilities? My position is, no, we have not. We have
• 1:20:29
  a lot of work to do to ensure that. But we don't have
• 1:20:33
  a study. We don't have a study that points to that. Right. I know it's
• 1:20:36
  a concern, and I understand the concern, and I will be concerned about, too,
• 1:20:39
  but I don't think we have evidence to say states are
• 1:20:44
  responding to these policy measures by denying
• 1:20:48
  services. Do we have enough evidence to make that statement?
• 1:20:51
  We have evidence that when states artificially restrict.
• 1:20:56
  When they artificially restrict access to services,
• 1:21:00
  particularly by students of color, and this is reported as
• 1:21:04
  a result of the Texas special education cap, that those students seem to have worse
• 1:21:08
  outcomes over time. We do have some descriptive reporting that
• 1:21:12
  schools seem to be responding to the monitoring,
• 1:21:16
  and there is some suggestions that
• 1:21:20
  they may be less likely to identify students of color.
• 1:21:24
  We have a lot of evidence there, too. Right. Because like in the
• 1:21:28
  case of Texas, and I can think in Chicago, too, I've been in Borman in
• 1:21:31
  Chicago, which they have said, oh, there is this proportionality. Come the same argument you
• 1:21:35
  say, and they're trying to address the proportionality, but what they're really trying to do
• 1:21:39
  is cut money. Right. They're trying to cut funds and not provide the services.
• 1:21:42
  Right. So more than addressing this proportionality,
• 1:21:45
  more than Trying to address this proportionality issue. What they're trying to do is justify
• 1:21:50
  the budget cut, basically. Right. Well, the services are expensive.
• 1:21:54
  They're about twice as light, twice as costly to provide as
• 1:21:58
  general education services. This is one of the reasons so many families have it difficulties
• 1:22:02
  accessing the services because schools know they're costly
• 1:22:06
  to provide. So,
• 1:22:10
  so this, I mean to me this is another issue that we should
• 1:22:14
  try and address. Of course, we could go down to the whole other question
• 1:22:18
  of like are these, should the federal government do any more to cover the cost
• 1:22:22
  of these services? Yes. You know, like there's all other things we could talk about.
• 1:22:25
  Absolutely. So let me build up on that question because we're going to wrap up.
• 1:22:29
  You were being very generous with time and I appreciate it. So the last question,
• 1:22:32
  you know, we are in a very interesting context right now, as you probably know,
• 1:22:36
  with special education office. We may have a
• 1:22:40
  special education office, but with nobody sitting there. So thinking about
• 1:22:45
  all the things that are going on right now, the research that you're doing,
• 1:22:47
  what would you wish to see in a special education in the next 10 years
• 1:22:51
  if you have to dream big?
• 1:22:55
  Okay, that's a good question. You know,
• 1:22:59
  I'll start just as a researcher. We've been cutting
• 1:23:03
  left and right support for data collection. So it's
• 1:23:07
  going to be harder to understand and investigate this issue.
• 1:23:11
  Yeah, that's, that's cutting our
• 1:23:15
  nose off. The spider our face. That's making things worse. Like we should be collecting
• 1:23:20
  data to better understand this issue, particularly the kind of data that typically
• 1:23:24
  the federal government has supported like the NAEP or
• 1:23:28
  the Eccles k data collections. So speaking for myself as
• 1:23:32
  a researcher, we should be trying to invest in further study of
• 1:23:37
  this issue, including as a result of rich
• 1:23:41
  data collection, large nationally representative samples of children followed across
• 1:23:45
  time. Those seem to be cut. I think those are making things worse off,
• 1:23:50
  you know, more, more, more generally,
• 1:23:54
  I have a lot of concerns about where things are going.
• 1:23:57
  Like the cuts to IES have been and
• 1:24:01
  from my perspective, a disaster. The cuts
• 1:24:05
  to nces a disaster. It's just,
• 1:24:09
  it's just making things much harder for us to
• 1:24:13
  understand. These data collections efforts
• 1:24:17
  are really important to provide an evidence base for special education and education.
• 1:24:23
  So you know, I have a lot of concerns about where the country is moving
• 1:24:27
  in terms of kind of the federal government's role in
• 1:24:31
  supporting scientific investigation
• 1:24:36
  and special education services, including oversight.
• 1:24:39
  We, from my perspective, we need the federal government to be providing
• 1:24:43
  oversight in house localities. States are providing special education
• 1:24:48
  because there's a lot of reason to believe, as we've been talking about,
• 1:24:51
  that these services are not being provided,
• 1:24:56
  including to less resourced families, as they should be.
• 1:25:00
  And so a declining oversight role in that
• 1:25:04
  regard gives me a lot of concern, and I feel like we're moving in the,
• 1:25:07
  in the wrong direction. So, anyway,
• 1:25:11
  I'll. I would say that of
• 1:25:16
  course, we can always optimize things better. There's always room for improvement.
• 1:25:20
  But let's not throw out the baby with the bathwater and let's,
• 1:25:23
  let's, let's try and support scientific investigation.
• 1:25:27
  That's that, that focusing, that focus of trying to
• 1:25:31
  follow the evidence has been very beneficial in many
• 1:25:35
  ways for understanding how to help children, including in terms of their health
• 1:25:39
  and also their educational experiences. Well, thank you so much,
• 1:25:42
  Paul, for your time. This was great. It's been a pleasure to have you on
• 1:25:45
  the show. I appreciate your generosity. I appreciate the invitation.
• 1:25:49
  Federico, I think this is great that you're having this kind of discussion.
• 1:25:53
  If there's any way I can be helpful, please don't hesitate to reach out again.
• 1:25:57
  And thanks so much for coming up to live.
• 1:26:01
  Thank you for listening to Dive In. I hope you learned from
• 1:26:05
  this episode as much as I did. Please help us to spread the word
• 1:26:09
  about the show. See you next time.