Latest / The Psychology Undergrad Podcast / Formative Assessment: How Teachers Measure Learning in the Classroom
Transcript
- 0:00Welcome to the psychology undergrad. Hey, everyone.
- 0:03So I want you to think back to the most unfair
- 0:05test you ever took, like the one that made your
- 0:08stomach completely drop and just made you want
- 0:10to scream because it felt like a total trap.
- 0:12Oh, I have definitely taken a few of those. Right.
- 0:15But what if the problem wasn't actually your
- 0:17studying? What if it was like? a hidden psychological
- 0:20flaw in the instrument itself. That is a wildly
- 0:23comforting thought, honestly. I mean, it really
- 0:25is. So grab a seat and pull up your notes because
- 0:28whether you are actively prepping for a psych
- 0:30exam with us today or you're just insanely curious
- 0:34about how your own teachers historically measured
- 0:36your brain. we are doing a deep dive into the
- 0:39psychology of testing. Yeah, we are pulling apart
- 0:42chapter 11 on teacher -made assessment strategies,
- 0:45which honestly, as undergraduate psychology students,
- 0:48we spend so much time looking at the brain in
- 0:50a vacuum, you know? Yeah, totally. But the moment
- 0:53you put a human being in a classroom and hand
- 0:56them a graded paper, you trigger this massive
- 0:59avalanche of psychological responses. It's basically
- 1:02a highly controlled behavioral experiment. Exactly.
- 1:06Every test you or I have ever taken wasn't just
- 1:08a piece of paper, it was a behavioral instrument.
- 1:10So our mission in this study session is to really
- 1:13look at the cognitive impacts of grading. And
- 1:16to explore the often very messy mechanics of
- 1:19assessment, plus build some handy memory aids
- 1:22to make sure this all sticks in our heads forever.
- 1:24Right. So to get to the bottom of how testing
- 1:27rewires the human mind, we really have to start
- 1:30with the vocabulary. because everyday language
- 1:33just completely fails us here. Oh, absolutely.
- 1:36People use words like assessment, measurement,
- 1:38and evaluation as if they are all just synonyms.
- 1:41Like they all mean the exact same thing. Yeah,
- 1:43but looking at the actual mechanics of it, they
- 1:45form a very specific sequence. First, you have
- 1:48assessment. Which is the big picture, right?
- 1:50Right. It's the massive integrated process of
- 1:52gathering information, like portfolios, observations,
- 1:55quizzes, all of that. Then comes the second step,
- 1:58which is measurement. And that's the part everybody
- 2:00hates. Exactly, because measurement assigns the
- 2:03cold, hard number. It answers the question, how
- 2:06much? Like, you got an 85 out of 100. But the
- 2:09real friction, the part that actually impacts
- 2:11the student, happens at the third step, which
- 2:14is evaluation. This is where it gets subjective.
- 2:18Yeah, it's where the human element enters the
- 2:20equation. Evaluation is making a judgment about
- 2:22that measurement. From a psychological standpoint,
- 2:25this is where a plain number transforms into
- 2:27a verdict on a student's actual capability. Right,
- 2:30because a measurement just says, OK, Jacob scored
- 2:32a 60%. But the evaluation looks at the context.
- 2:36Like, did Jacob start at a 20 % and show massive
- 2:39cognitive growth? Is that 60 % actually a triumph
- 2:42for him? Exactly. You know, it almost feels like
- 2:44a moral or even a theological judgment. Like,
- 2:48measurement is just objectively counting sins
- 2:50or good deeds, but evaluation is like St. Peter
- 2:53at the pearly gates weighing your soul. That
- 2:56is an intense way to look at it, but yeah, it
- 2:58fits. You're looking at the whole human. Right.
- 3:00But there's another layer to this that dictates
- 3:02how our brains react to tests, and that's when
- 3:05the assessment actually happens. We have assessment
- 3:09for learning and assessment of learning. And
- 3:12the timing completely changes the cognitive purpose.
- 3:15Assessment of learning is summative. So it happens
- 3:18at the end. Yeah, it happens after the instruction
- 3:20is over to certify competence. Think of your
- 3:23big final exam. It is basically an autopsy of
- 3:26what you learned. An autopsy. Wow. Well. It is.
- 3:31But assessment for learning is formative. It
- 3:33takes place during instruction. Right. Our textbooks
- 3:36always use that super cliche analogy for this.
- 3:39Like when the chef tastes the soup, it's formative.
- 3:42But when the customer tastes it, it's summative.
- 3:44I mean, it works, but it's a bit tired. It is.
- 3:48Honestly, a better way to think about it cognitively
- 3:50is like a GPS navigation system. Formative assessment
- 3:53is the GPS actively tracking your car. Oh, I
- 3:56like that. Yeah, like when you take a wrong turn,
- 3:58the GPS recalculates and tells the driver, who
- 4:01is the teacher, to adjust the route immediately.
- 4:04Summative assessment is just getting a printed
- 4:06receipt at the very end of the road trip telling
- 4:08you that you drove to the completely wrong city.
- 4:10That is a great memory aid. And that navigational
- 4:13mechanism is deeply rooted in cognitive psychology,
- 4:16too. How so? Well, the brain requires constant
- 4:18feedback loops to correct errors in working memory
- 4:21before they get permanently encoded into long
- 4:24-term memory. Oh, right. Because if you learn
- 4:25it wrong, unlearning it is a nightmare. Exactly.
- 4:28So if a teacher relies solely on a summative
- 4:31final exam, they're essentially driving blind.
- 4:34Formative assessment requires establishing really
- 4:37clear goals so the student's brain knows exactly
- 4:40what to encode. Give me an example of that. So
- 4:42if the goal is just, hey, learn about the Civil
- 4:44War, the cognitive load is just way too vast.
- 4:47The brain doesn't know what to prioritize. So
- 4:48just noise at that point. Right. But if the goal
- 4:50is analyze the economic factors of the Civil
- 4:52War, the student's reticular activating system
- 4:55filters information specifically for economics.
- 4:59So the brain has a target. But wait, if a teacher
- 5:02is using this formative GPS to steer their entire
- 5:05curriculum, they are completely at the mercy
- 5:07of that test's data. Oh, 100%. So if the test
- 5:10is lying to them, the whole system collapses,
- 5:13which really forces us to look at the anatomy
- 5:15of the test itself. We have to ask, is the instrument
- 5:18actually fair? And that brings us to the holy
- 5:21trinity of good test design. Validity, reliability,
- 5:25and the absence of bias. Let's start with validity,
- 5:28because that one seems like the most important.
- 5:30It really is. Let's break down validity first,
- 5:32because it answers the most fundamental question.
- 5:34Does the test actually measure what it's supposed
- 5:37to measure? Right. And there are three types,
- 5:39right? Yeah. We categorize it into three types.
- 5:41Content validity asks if the test actually covers
- 5:44the syllabus. Like, did you even teach me this
- 5:46stuff? Exactly. Then criterion -related validity
- 5:50asks if the test can accurately predict a future
- 5:53outcome, like, say, an SAT score predicting freshman
- 5:57college grades. Got it. But the most psychologically
- 5:59complex one is construct validity. Oh, construct
- 6:03validity is fascinating because it asks if we
- 6:06are measuring the intended psychological trait
- 6:08or if we are accidentally measuring a completely
- 6:11different variable. Yeah, the classic example
- 6:13is giving a complex mathematical reasoning test
- 6:16to a recent immigrant. Right. If the word problems
- 6:19are written in dense archaic English, a low score
- 6:22doesn't necessarily mean they lack mathematical
- 6:24reasoning. Exactly. The test is accidentally
- 6:27measuring their English language proficiency.
- 6:30The construct is completely corrupted. It's so
- 6:32unfair. Or consider test anxiety. Say a student
- 6:36understands the material perfectly in a low stakes
- 6:38environment, but then their working memory gets
- 6:42hijacked by their amygdala during a timed exam.
- 6:45So they just freeze up. Right. A rigid test might
- 6:49lack constructive validity for that specific
- 6:51student. It's measuring their stress response,
- 6:54not their actual historical knowledge. Wow. OK,
- 6:57I want to introduce a memory aid here because
- 6:59it is so easy to confuse validity with our next
- 7:02term, which is reliability. They do sound really
- 7:04similar. They do. But reliability is simply the
- 7:07consistency of the measurement. Like, if you
- 7:10take the test on Monday, do you get the same
- 7:12score on Friday? OK, so what's the memory aid?
- 7:15Think of a broken bathroom scale. If I step on
- 7:17a broken scale, it might say I weigh 50 pounds
- 7:20every single time I step on it. Which would be
- 7:22amazing. Right. But it's incredibly reliable
- 7:25because it's consistent. It says 50 pounds every
- 7:27time. But it is completely invalid because it's
- 7:29not measuring my actual weight. That is a perfect
- 7:32way to look at it. Think of an archer shooting
- 7:34arrows at a target, too. If the archer hits the
- 7:37exact same spot on the outer edge of the target
- 7:39five times in a row, they are incredibly reliable.
- 7:42Their grouping is perfectly consistent. But they
- 7:45are completely invalid because they missed the
- 7:47bullseye. They missed the actual target they
- 7:49were supposed to hit. Exactly. You can have a
- 7:51reliable test that is completely invalid, but
- 7:54you cannot have a valid test without it first
- 7:56being reliable. That makes total sense. And this
- 7:59mechanical failure directly ties into the third
- 8:02pillar, which is the absence of bias. From a
- 8:05purely data -gathering perspective, bias introduces
- 8:08noise that ruins both reliability and validity.
- 8:11And the sources outline two major forms of bias,
- 8:14right? Offensiveness and unfair penalization.
- 8:17Yeah. Offensiveness is pretty straightforward.
- 8:19If a test uses negative stereotypes, it triggers
- 8:21an emotional response that derails the student's
- 8:24focus. Which again, hijacks the amygdala. Right.
- 8:27But unfair penalization is much more insidious.
- 8:29It happens when a test disadvantages a group
- 8:31based on differing background experiences. Like
- 8:34the symphony orchestra analogy from our notes?
- 8:37Yes, using a symphony orchestra seating chart
- 8:40to measure mathematical proportions. That heavily
- 8:43favors students from more affluent backgrounds
- 8:45who actually have a schema for how an orchestra
- 8:48is seated. Or think of a word problem built around
- 8:50the rules of American football. Right. If a student
- 8:53isn't familiar with the point values of a touchdown
- 8:55versus a field goal, like maybe they are a recent
- 8:57immigrant, their math data is corrupted by their
- 9:01lack of cultural trivia. Wait, but I have to
- 9:03play devil's advocate for a second. Isn't some
- 9:05level of background knowledge basically always
- 9:08required? I mean, yeah. How can you possibly
- 9:11strip all cultural context out of a test? Every
- 9:15word problem has to be about something, right?
- 9:17You can't just have a blank void. It's true.
- 9:19You can't build a perfectly sterile test. The
- 9:22cognitive rule of thumb is asking this question,
- 9:25is the background knowledge required to answer
- 9:27the question explicitly taught in the curriculum?
- 9:30Or is it assumed based on students socioeconomic
- 9:33status? Oh, OK. So if it's the latter, the test
- 9:36is fundamentally flawed. you are basically measuring
- 9:40their upbringing, not their learning. Exactly.
- 9:43It becomes an ethical issue at that point. So,
- 9:45with all these potential pitfalls, invalid constructs,
- 9:49broken scales, cultural bias, what do these tests
- 9:54actually look like when we build them? The toolkit
- 9:56has to start with informal assessments, right?
- 9:59Yeah, like a teacher just observing the room.
- 10:01But observation is wildly prone to confirmation
- 10:04bias. Though hugely. A teacher might assume a
- 10:06quiet student is paying attention, while an energetic
- 10:09student is off task, purely based on their own
- 10:12cultural expectations of what learning is supposed
- 10:14to look like. Which is exactly what necessitates
- 10:16formal assessments. Specifically, selected response
- 10:19and constructed response. So, selected response
- 10:21items are your classic objective tests. Multiple
- 10:24choice, matching, true or false. Right. And from
- 10:27a cognitive processing standpoint, these rely
- 10:29heavily on recognition memory. The answer is
- 10:32literally right in front of the student. You
- 10:34just have to identify it. You know, there is
- 10:35a massive irony here that really jumped out at
- 10:37me in our notes. Yeah, the source material gives
- 10:40examples of multiple -choice questions. And one
- 10:42of them asks students to identify who developed
- 10:45the morality of justice. And the correct bubble
- 10:48is Lawrence Colbert. Right. The stages of moral
- 10:51development. Exactly. We are taking profound,
- 10:55highly subjective moral frameworks about the
- 10:57nature of human justice, and we are reducing
- 11:00them to a bubble sheet. A, B, C, or D. It does
- 11:04feel a little reductive. It feels intellectually
- 11:06dissonant. Like, using rigidly objective tools
- 11:09to measure deeply subjective philosophical concepts
- 11:12just feels inherently broken. Well, there is
- 11:15a severe trade -off. Selected response tests
- 11:17are incredibly efficient to grade, obviously.
- 11:20But they often fail to capture deep cognitive
- 11:22synthesis. Because you're just picking from a
- 11:24list. Right. And honestly, they are surprisingly
- 11:27difficult to write well. The human brain is easily
- 11:29tripped up by poor test design. Oh, like double
- 11:32negatives and a true or false question? I hate
- 11:34those. They are the worst. They artificially
- 11:36inflate the cognitive load. Or you get teachers
- 11:39testing trivial facts instead of core concepts
- 11:42just because the trivia is way easier to turn
- 11:44into a multiple choice option. That makes a lot
- 11:46of sense. So on the flip side of that, you have
- 11:48constructed response. That's your essays, short
- 11:51answers, portfolios, and performance assessments.
- 11:54Yeah, where the student actually has to create
- 11:56the answer. Right. Like, if you want a student
- 11:58to synthesize the moral, political, and economic
- 12:01causes of the US Civil War, have them write an
- 12:05essay. It's definitely a better measure of deep
- 12:07learning. It is. But if I'm a teacher with 150
- 12:09students, I kind of have to use multiple choice.
- 12:12I physically cannot reliably grade 150 nuanced
- 12:16essays without losing my mind. Or letting your
- 12:19mood dictate the scores. Exactly. If I grade
- 12:22the first essay after my morning coffee and the
- 12:25150th essay at midnight when I'm exhausted, my
- 12:28gradem reliability is zero. And that exact human
- 12:31limitation is why educational psychology developed
- 12:34scoring rubrics. Thank goodness for rubrics.
- 12:37Rubrics act as cognitive anchors for the greater.
- 12:40They pull that subjective evaluation back into
- 12:42the realm of objective measurement. And there
- 12:44are two main types, right? Holistic and analytical.
- 12:48Yep. You have holistic rubrics, which look at
- 12:51the overall quality of the work as a single score.
- 12:54And then you have analytical rubrics, which break
- 12:57the assignment down into specific weighted crates.
- 13:00Like grading historical accuracy, organization,
- 13:03and argument strength all separately. Exactly.
- 13:05Yeah. It brings fairness back to the essay. And
- 13:08you know, analytical rubrics also serve as a
- 13:10formative GPS for the student. If you hand the
- 13:13student the rubric before they even write the
- 13:14essay, you totally eliminate the guesswork. They
- 13:17know exactly how their cognition is going to
- 13:19be measured. Right. But the mechanism of grading
- 13:21doesn't end when the rubric is filled out. Not
- 13:24at all. The moment that graded paper is handed
- 13:26back, we enter a really critical psychological
- 13:30threshold. How does that measurement actually
- 13:32impact the student's future motivation? This
- 13:35is where we run into Carol Dweck's research.
- 13:37which is probably one of the most misunderstood
- 13:39concepts in modern education. The growth mindset
- 13:42stuff. Yeah. She identified the incremental view
- 13:44of ability versus the fixed view of ability.
- 13:48A fixed mindset assumes your intelligence is
- 13:51just a static quantity. Like a bucket that's
- 13:53already full. Right. But an incremental mindset,
- 13:56which we now call a growth mindset, assumes intelligence
- 13:59actually expands through cognitive effort. Okay,
- 14:03but the mechanical failure happens in how teachers
- 14:06deliver praise, doesn't it? Like, it is incredibly
- 14:09common to hand back a test and say, wow, you
- 14:12got an A, you are so smart. That is actually
- 14:15incredibly dangerous to say. Wait, praising kids
- 14:17is bad now. That sounds completely counterintuitive
- 14:19to most people. How can telling a student they
- 14:22are smart be a bad thing? It's because of how
- 14:24the brain interprets identity. When you praise
- 14:27inherent intelligence, the student internalizes
- 14:29the word smart as this fragile core trait that
- 14:33they now have to protect at all costs. Oh, I
- 14:35see. So if they encounter a really difficult
- 14:37problem on the next test and they struggle, their
- 14:40brain processes that struggle as a direct threat
- 14:43to their identity. Because the internal logic
- 14:44is, if success means I am smart, then struggling
- 14:47must mean I am dumb. Exactly. They will actually
- 14:50avoid challenging work just to preserve the label
- 14:53of being smart. It creates a psychological glass
- 14:55ceiling. That is wild. It really is. You have
- 14:58to praise the strategy and the effort, not the
- 15:01inherent trait. Which feeds directly into the
- 15:04concept of wise feedback, researched by Cohen,
- 15:06Steele, and Ross. Yes, wise feedback is crucial.
- 15:09It has three components, right? You point out
- 15:12what they do well, you give clear criticism of
- 15:14what failed, and crucially, you provide an explicit
- 15:17statement that you actually believe the student
- 15:19has the capacity to reach the standard. It bridges
- 15:22the gap between cold data and human belief. And
- 15:25that combination is particularly vital when giving
- 15:29feedback across racial, ethnic, or socioeconomic
- 15:32lines where there might be some historical distrust
- 15:35of academic institutions. Absolutely. The student
- 15:38needs to know the criticism is based on high
- 15:39standards, not on some biased expectation of
- 15:42their failure. Speaking of expecting failure,
- 15:44we really need to talk about grading mechanisms,
- 15:47specifically the fundamental difference between
- 15:49absolute grading and relative grading. Oh, boy,
- 15:52the curve. Yes, the curve. Absolute grading is
- 15:56based on a fixed criterion. If everyone masters
- 15:58the material, everyone gets an A. But relative
- 16:02grading is grading on the curve. And I have to
- 16:04say, looking at the psychology of the bell curve,
- 16:07it seems incredibly toxic. It is entirely designed
- 16:10to artificially distribute scores along a normal
- 16:13distribution, regardless of actual learning.
- 16:16It literally pits students against each other.
- 16:19Our sources point out that relative grading gives
- 16:21students a personal, vested interest in persuading
- 16:24their peers not to study. Because if the average
- 16:27goes down, your grade goes up. Yes. Think about
- 16:30how you felt if you ever got an A simply because
- 16:33the rest of the class completely bombed the midterm.
- 16:35It felt like a win. But cognitively, the system
- 16:39just incentivized selfishness. It really does.
- 16:41If my friend failing makes it statistically easier
- 16:43for me to succeed, you have actively destroyed
- 16:46classroom collaboration. It is morally messed
- 16:48up. We run that through the lens of self -determination
- 16:50theory. We can see exactly why the curve destroys
- 16:53motivation. Remind me what that theory is again.
- 16:56Self -determination theory states that human
- 16:58beings require three psychological needs to be
- 17:01motivated. Competence, relatedness, and autonomy.
- 17:06Okay. Grading on a curve immediately shatters
- 17:09relatedness. Your peers are no longer your community.
- 17:13They are obstacles to your grade. That is so
- 17:15depressing. And how do we fix the autonomy piece?
- 17:18Because most tests feel incredibly dictatorial.
- 17:21You sit down, you shut up, and you fill in bubble
- 17:22C. You fix it by introducing choice. When an
- 17:26assessment allows a student to choose their method
- 17:28of demonstration, their intrinsic motivation
- 17:31just skyrockets. Give me an example. Instead
- 17:34of a rigid essay. What if the student can choose
- 17:36to explain the Bill of Rights through a board
- 17:38game they design? Or a recorded podcast? Or written
- 17:41play? Oh, that's awesome! Right? When the student
- 17:44has autonomy over the vehicle of assessment,
- 17:46the effort they expend increases dramatically
- 17:48because the work finally has personal relevance.
- 17:50That makes total sense. You know, we've spent
- 17:52this entire time analyzing the students' minds.
- 17:55But what about the teacher standing at the front
- 17:57of the room? That's a great question. If a teacher
- 17:59recognizes their tests are flawed or their rubrics
- 18:02are biased, how do they systematically fix their
- 18:05own psychology? They utilize what's called action
- 18:08research. This is when the teacher basically
- 18:11becomes a scientific investigator. within their
- 18:13own classroom. Instead of just waiting for some
- 18:15university researchers to publish a study five
- 18:18years from now. Exactly. The teacher treats their
- 18:20own instructional methods as a live experiment.
- 18:23The cycle for this is actually really easy to
- 18:25remember with the acronym PADC. PADC. Plan, act,
- 18:30develop, communicate. Right. First, you plan.
- 18:34You identify a specific problem. Like, why do
- 18:37my students keep failing the constructed response
- 18:39section? Second, you act. You collect data, observe
- 18:42behaviors, and try a new instructional approach.
- 18:45And then third, you develop a refined action
- 18:47plan based on what the data actually showed you.
- 18:49Finally, you communicate your findings with other
- 18:51teachers. P -A -D -C. It's an incredibly powerful
- 18:55tool for pedagogical growth. But it does introduce
- 18:58a very firm ethical boundary. A classroom is
- 19:01a captive audience. Right, a teacher has the
- 19:03institutional authority to force a student to
- 19:06complete a worksheet. But they cross a severe
- 19:09moral and legal line the moment they turn that
- 19:12student into an involuntary research subject.
- 19:15Informed consent is not negotiable. Exactly.
- 19:18If a teacher wants to take the data from their
- 19:20action research and publish it, or use it for
- 19:22a university thesis, they must obtain voluntary
- 19:25informed consent from the parents and the students.
- 19:28You cannot compel participation in an experiment.
- 19:32That ethical line respects the autonomy of the
- 19:34student. And, you know, when action research
- 19:37is done correctly, it builds what we call teacher
- 19:39efficacy. Which is what exactly? It's this profound
- 19:41internal belief that a teacher's actions caused
- 19:45student learning. Teachers with low efficacy
- 19:47tend to blame the students home lives, their
- 19:49genetics, or their socioeconomic status for poor
- 19:52test scores. They pass the buck? Right. But a
- 19:55teacher with high efficacy looks at a failing
- 19:57grade and says, my instructional approach didn't
- 20:00work for this brain. I need to change the input.
- 20:03Action research is the mechanism they use to
- 20:05prove to themselves that they can make a difference.
- 20:08Exactly. It brings the whole concept of assessment
- 20:11for learning full circle. It completely flips
- 20:13the script. The test isn't just grading the student,
- 20:16it's grading the effectiveness of the teaching.
- 20:18That is a really profound way to look at it.
- 20:21We have unpacked a massive amount of cognitive
- 20:24architecture today. I mean, we moved from the
- 20:27GPS navigation of formative assessment to the
- 20:30postmortem of summative exams. We visualized
- 20:34reliability and validity through our archer missing
- 20:37the bullseye. And we looked at how cultural bias
- 20:40completely corrupts data. We also explored the
- 20:43mechanics of multiple choice versus analytical
- 20:45rubrics, the danger of praising inherent intelligence
- 20:48over effort, and how the bell curve fundamentally
- 20:51breaks human collaboration. It is honestly enough
- 20:54to make you rethink every grade you ever received.
- 20:56Seriously. But before we wrap up, I want to leave
- 20:58you with one final lingering thought from our
- 21:00sources. We learned that portfolios and performance
- 21:03assessments are incredible for capturing complex,
- 21:06authentic human learning. They give students
- 21:08autonomy and they measure deep cognitive synthesis.
- 21:11Right. They are great for validity. But we also
- 21:14learned they suffer from terrible reliability.
- 21:17Just look at the famous example of the Vermont
- 21:19State Portfolio Program from our note. Oh, right.
- 21:23Even highly trained raters couldn't agree on
- 21:25how to score the fourth and eighth grade portfolios.
- 21:28The human element made it entirely unreliable.
- 21:31So what's the takeaway? The takeaway is this
- 21:33question to mull over before your next exam.
- 21:36Are we fundamentally forced to choose between
- 21:38a standardized test that is perfectly reliable
- 21:41but completely shallow and an authentic assessment
- 21:44that is deeply meaningful but inherently biased
- 21:46by human judgment. Wow. It is the ultimate paradox
- 21:50of educational psychology. It really is. Keep
- 21:53questioning how your mind is being measured.
- 21:55Thank you for pulling this study session with
- 21:57us. We'll see you next time on The Psychology
- 21:58Undergrad.