Latest / The Reading Instruction Show / The Latest NCTQ Technical Report Does Not Meet Basic Academic Standards

Transcript

The Latest NCTQ Technical Report Does Not Meet Basic Academic Standards · The Reading Instruction Show

• 0:00
  This is the Reading Instruction Show. I'm your
• 0:03
  host, as always, Dr. Andy Johnson. Topic of today's
• 0:06
  podcast, the latest NCTQ technical report does
• 0:12
  not meet basic academic standards. And it's an
• 0:17
  excerpt from my book, Making Sense of the Science
• 0:19
  of Reading, Teachers Matter, that will be published
• 0:23
  by Guilford sometime in 2028. Now the National
• 0:27
  Council on Teacher Quality, NCTQ, recently put
• 0:31
  out another technical report. This technical
• 0:36
  report graded 700 teacher preparation programs
• 0:40
  across the country. However, a critical evaluation
• 0:45
  shows why this report can't be taken seriously.
• 0:50
  It contains an abundance of misinformation, disinformation,
• 0:55
  and convenient inaccuracies. And it's not known
• 1:00
  if these were intentional on the part of the
• 1:02
  writers or a result of a lack of knowledge. And
• 1:07
  this chapter contains my initial analysis, but
• 1:12
  first a little context. Part one, the necessity
• 1:16
  of analyses and critical evaluations. Academia,
• 1:21
  or the academy, refers to the community of scholars
• 1:25
  and researchers who seek new knowledge. And within
• 1:31
  the academy, members seek to maintain standards
• 1:35
  within their respective academic fields. This
• 1:40
  means we critically evaluate each other's work.
• 1:45
  And this is an important part of the academic
• 1:49
  process. This type of critical evaluation along
• 1:55
  with accompanying academic discourse keeps our
• 1:59
  various academic fields evolving. Now academic
• 2:04
  discourse has traditionally occurred through
• 2:07
  our books, journals, and academic conferences,
• 2:12
  but today increasingly these conversations also
• 2:17
  occur on various social media platforms. This
• 2:23
  is a healthy development as it opens up the conversation
• 2:27
  to a wider audience and more participants. However,
• 2:34
  the one thing missing on social media platforms,
• 2:37
  as you know, is any form of peer review or checks
• 2:41
  and balances. Anybody can write things or say
• 2:45
  things. And in this sense, The conversations
• 2:50
  taking place on social media are no different
• 2:53
  from this NCTQ technical report. Literacy instruction
• 3:00
  is a field within the Academy, and I am a scholar
• 3:04
  in the field of literacy instruction. As such,
• 3:09
  I will be analyzing and critically evaluating
• 3:13
  this technical report on its merit. Nobody gets
• 3:18
  a free ride in the academy. In the academy we
• 3:22
  must all be held accountable for our words. And
• 3:27
  I would expect that someone will do the same
• 3:30
  type of evaluation with this chapter. And that
• 3:35
  is a wonderful thing. Chapter 2, there is no
• 3:40
  reading crisis. The first sentence of the second
• 3:45
  paragraph of this report says that, quote, 40
• 3:49
  % of fourth graders in the United States cannot
• 3:53
  read at a basic level. The worst results in over
• 3:57
  two decades, unquote. Now this sentence would
• 4:03
  have you believe there's a reading crisis, but
• 4:07
  this would be a convenient inaccuracy. There
• 4:12
  is no irrefutable evidence that we are experiencing
• 4:17
  a national literacy crisis. NAEP reading scores
• 4:24
  for fourth and eighth graders from 1971 to 2020
• 4:28
  shows that there have been natural fluctuations
• 4:32
  as we would expect, but there have been no massive
• 4:37
  declines in reading test scores. In both fourth
• 4:43
  grade and eighth grade, average scores are slightly
• 4:47
  higher than they were in 1971, up to the COVID
• 4:51
  dip in 2020. Also, pre -COVID scores are basically
• 4:57
  the same as they were 20 years ago. Because the
• 5:04
  same norm reference test versus criterion reference
• 5:08
  tests are given over time, NAP data enables comparisons
• 5:13
  to be made between states and various subgroups
• 5:17
  and over time. Also, the populations tested are
• 5:22
  representative and large enough for the results
• 5:25
  to be generalized with a fair amount of confidence.
• 5:29
  That's why we use NAEP data and not state criterion
• 5:34
  reference tests. In both fourth and eighth grade,
• 5:40
  the 2024 scores were lower than they were before
• 5:44
  the COVID crisis in 2019, as we would expect,
• 5:48
  and this difference is statistically significant.
• 5:52
  Statistical significance means that the differences
• 5:56
  are large enough to eliminate chance or error
• 6:00
  as the cause of the difference. By 2022, 37 states
• 6:07
  and territories had passed reading laws. We might
• 6:12
  have expected to see some impact on the national
• 6:17
  trends of fourth grade students by 2024, but
• 6:20
  this has not been the case. As of 2026, 44 states
• 6:27
  and Washington DC have passed legislation mandating
• 6:32
  evidence -based reading instruction aligned with
• 6:36
  the science of reading. And if this news scientifically
• 6:42
  based Reading instruction works as promised.
• 6:47
  In 2028 and later, we should see NAP scores of
• 6:52
  fourth grade students to be at least where they
• 6:56
  were in the pre -COVID era. That means 220 or
• 7:00
  higher. Check back in 2028. Part three, a basic
• 7:06
  understanding of basic level. So let's get to
• 7:10
  the crux of the matter. As stated above, the
• 7:14
  NCTQ report claims that 40 % of fourth grade
• 7:19
  students in the United States cannot read at
• 7:24
  a basic level. And if this were even a little
• 7:29
  bit true, it would be a horrible thing. But the
• 7:33
  problem with this technical report is that it's
• 7:36
  not even a little bit true. So that means that
• 7:41
  everything that follows is based on a false premise.
• 7:45
  Let me explain. Basic level. NAEP test data shows
• 7:51
  that in general, teachers have been doing a pretty
• 7:53
  good job. So to find some failure in 1992, the
• 7:58
  NAEP decided to add three levels of proficiency.
• 8:04
  Basic, proficient, and advanced. However, These
• 8:10
  proficiency levels were arbitrarily defined.
• 8:17
  In addressing NAEP levels, Rinking Ruby and Risco
• 8:21
  said, and I quote, who had been contracted to
• 8:41
  advise the department of this approach, but who
• 8:45
  were overruled when they were argued against
• 8:47
  doing it at all." These NAEP proficiency levels
• 8:55
  should be given very little credence. The National
• 9:01
  Center for Educational Statistics website provides
• 9:06
  three reasons for this. They said, this is NCEES,
• 9:14
  proficiency levels are based on expectations,
• 9:17
  meaning that these levels are aspirational, not
• 9:21
  mathematical. Two, these proficiency levels are
• 9:26
  currently being used on a trial basis, since
• 9:29
  1992 apparently, but that's what it says on their
• 9:33
  website. And three, Further evidence is needed
• 9:37
  to determine if these proficiency levels are
• 9:40
  reasonable, valid, and informative. So let me
• 9:47
  put it plainly. A group of people got together
• 9:51
  in 1992. They saw that the average score of fourth
• 9:56
  grade students was 217. So if that was average,
• 10:03
  they decided That basic had to be something a
• 10:07
  little below that. And a score of 208 was a little
• 10:12
  below 217. So 208 was declared to be the basic
• 10:18
  level. It had nothing to do with basic skills.
• 10:23
  It had nothing to do with what was or wasn't
• 10:27
  learned. The sole determination of basic level
• 10:33
  was that it wasn't quite 217. Imagine that. If
• 10:40
  you arbitrarily decide that the basic level will
• 10:44
  be slightly below average, you are guaranteeing
• 10:47
  that at least 40 % of a normal population will
• 10:52
  be reading below the basic level. In other words,
• 10:57
  if the basic level is at the 40th percentile,
• 11:00
  40 % of the population will be below a basic
• 11:05
  level. That's how percentiles work, you see?
• 11:11
  Let me show you from another perspective. A normal
• 11:15
  population is represented by the familiar bell
• 11:19
  -shaped curve. And in any normal population,
• 11:24
  50 % are going to be above the mean and 50 %
• 11:28
  below. This means that there always be 50 % who
• 11:33
  score below average. This is called normal. And
• 11:39
  if you arbitrarily define basic level as slightly
• 11:44
  below the mean, you'll have at least 40 % of
• 11:48
  the population reading below the basic level.
• 11:53
  Imagine that. So, final word. In this chapter,
• 12:01
  I've addressed only the first three paragraphs
• 12:03
  of this 107 -page technical report. But based
• 12:09
  on this, one should carefully consider whether
• 12:13
  to spend time reading the next 106 pages. However,
• 12:18
  six big ideas here. Number one, literacy instruction
• 12:23
  is a field within the greater academy. Critical
• 12:28
  analyses and evaluations are an essential part
• 12:32
  of the academy. It keeps academic fields evolving,
• 12:38
  maintaining a set of academic standards, and
• 12:40
  provides a set of checks and balances. Two, there
• 12:45
  never was a literacy crisis. Three, the basic
• 12:50
  level is an arbitrarily defined level, guaranteeing
• 12:55
  that approximately 40 % of the population will
• 12:58
  always be below the basic level. Four, the basic
• 13:04
  level is not based on a set of skills mastered,
• 13:08
  rather it's based on a score being a little below
• 13:11
  average. Five, we should all be held accountable
• 13:17
  for our words. Nobody in the academy should get
• 13:21
  a free ride. And if the NCTQ enters into an academic
• 13:30
  field, such as literacy instruction, if they
• 13:33
  enter that discussion, they should be held to
• 13:38
  the same academic standards as the rest of us.
• 13:43
  And number six, the misuse of endnotes in citations
• 13:49
  has not been addressed. However, it's abundantly
• 13:53
  clear after reading the first three paragraphs
• 13:56
  that this NCTQ technical report does not meet
• 14:02
  basic academic standards. In the next podcast
• 14:07
  or chapter, we'll examine this NCTQ report as
• 14:12
  it relates to teacher preparation programs. This
• 14:15
  has been The Reading Instruction Show. I am your
• 14:18
  host, Dr. Andy Johnson.